Home >> December 2019 Edition >> Managing the High Volume of Mission-Critical Data for Defense Ops
Managing the High Volume of Mission-Critical Data for Defense Ops
By Ken Chadder, Defence Business Development Consultant, Hexagon’s Geospatial Division


“We’re currently faced with a security environment that is more complex, inter-connected, and volatile than we’ve experienced in recent memory – one which will require us to do things differently if we’re to navigate ourselves through it successfully.”

These words were spoken at the GEOINT 2019 Symposium by Vice Admiral Robert Sharp, Director of the National Geospatial-Intelligence Agency, as he was discussing the organization’s new strategy and goals.

The requirement to “do things differently” touches multiple facets of the Defense and Security environment, including the management of the information and intelligence that powers missions and critical decision-making processes. As Lt. Gen. James M. Dubik, U.S. Army Retired, pointed out in his 2018 article, “Decision-Making Process May Need an Update,” a shift from an industrial model of decision-making to a network model is taking place at the tactical operations level.

In an industrial model, the structure is more hierarchical, with decision-making remaining with select few, and “making decisions, taking coherent action, and adapting is predominately sequential.”

The network model is certainly a shift from the traditional industrial structure. It requires more dependence on leadership from different levels of command and lessens the gap between information availability and action. In the network model, according to Dubik, “Mission Command and intent predominates rather than directive command, staff work is more simultaneous and continuous than sequential and episodic, and overall the organization is more decentralized than centralized—all without loss of coherence in execution.”

The network model is “the military’s response to the Information Age,” according to the Department of Defense’s Office of Force Transformation in their paper, “The Implementation of Network-Centric Warfare.” As the capabilities of information technology have transformed the way commercial industries do business, a similar effect has taken hold on Defense and Intelligence operations. The network model means embracing technology such as sensors, communications systems, and other intelligence products for true situational awareness across the organization.

According to the paper, major impacts of this trend in U.S. operations were particularly beginning to be noticed during the 2001-2002 period of Operation Enduring Freedom (OEF) and in 2003 during the Iraq War (Operation Iraqi Freedom, OIF).

When speaking about his observations on a net-centric approach, retired U.S. General Tommy Franks, commander of coalition forces for OEF and OIF, noted that the approach bypassed previous “stove-piped” processes, and instead provided a wealth of invaluable information. For instance, when discussing a force tracking system that let him know the location of his forces in near-real-time, he stated, “I’ve died and gone to heaven and seen the first bit of net-centric warfare at work!”

Aligning with modern information technology trends and having the necessary information for informed decision-making is at the heart of the modern military. Having the correct information faster than opposing forces — which are also experiencing increased accessibility to smarter technology, due to globalization in the Information Age —can be critical to success.

To break down intelligence siloes and to empower more distributed, network-enhanced decision-making, each level of leadership requires the relevant intelligence to be timely and accurate. The necessary data and information can be hard to collate and analyze not because of a lack of sources, but due to the number of data sources are growing exponentially, forming a data deluge that can hinder accuracy, timeliness, and efficiency.

Commanders need to be able to act in minutes, which requires a greater need for situational awareness and data sharing across the entire Defense network – from the geospatial intelligence specialists who integrate and analyze data, to the planners who need to understand the battlespace, to the operators who must make decisions and act based on timely and accurate information.

Facing an Abundance of Data in the Defense Space
It is well-known that an influx of data is being generated every minute. Futurist, author, and business and technology consultant Bernard Marr quantified that data in a 2018 article. He said that we produce 2.5 quintillion bytes of data each day, and that 90 percent of the data in the world was generated in the previous two years.

Defense and Intelligence organizations have not been spared from this abundance of data. As Nathan Houser, a principal at Deloitte Consulting LLP, reported in his article, “Data-driven deployments: How analytics can transform military positioning,” it’s estimated that from 2001 to 2011, “the amount of data gathered by military drones and other surveillance technologies alone rose by 1,600 percent.”

Houser also referenced research from the New York Times, which specifically looked at data surrounding U.S. Air Force (USAF) intelligence. According to Hauser and New York Times, “Every day across the Air Force’s $5 billion global-surveillance network, cubicle warriors review thousands of hours of video, thousands of high-altitude spy photos, and hundreds of hours of (foreign communications).”

The volume, variety, and velocity of Intelligence, Surveillance, and Reconnaissance (ISR) data available today has exponentially increased, creating many challenges that include:

• A time-consuming and labor-intensive process of sifting through huge geospatial data sets to find the small percentage of data that is relevant for tipping and cueing.
• Difficulty in quickly visualizing and analyzing large geospatial data sets, such as LiDAR and hyperspectral imagery.
• Integrating, analyzing, and visualizing large amounts of disparate data, including real-time data, into a common operational picture streamed/shared across a network.
• Supporting mission-specific requirements that may require a custom solution.
• Finding alternative data sources to remedy gaps in situational awareness.

In this age of Big Data, the acquisition of data is no longer the challenge. It is in fact the organization that can process this data the fastest that will win the future battle

To overcome these challenges, technological solutions are needed to better discover, manage, integrate, visualize, and analyze large amounts of mission-critical data.

Simplifying Visualization and Analysis
Before visualization and detailed analysis of geospatial Big Data, commanders and their staffs have to first sift through and analyze huge geospatial data sets to find the relevant data in order to develop the intelligence required for decision-making and action. For decades, intelligence agencies have relied on tipping and cueing to streamline this process, where intelligence analysts are “tipped off” by the presence of something suspicious and “cue” the AOI for higher resolution imagery, data, or information and further analysis.

The key to streamlining this process in the data-era lies in embracing the latest computational capabilities, such as machine learning and artificial intelligence. Introducing these capabilities into the tipping and cueing process will help reduce the time and resources required, by filtering non-relevant data and alleviating the need to manually sift through large amounts of data. For example, being able to tip the analysts or automated algorithm towards a particular region/person/object in the data is useful in reducing search and processing time.

For geospatial analysis, AI and machine learning can already be taken advantage of in proven remote sensing solutions, adding a higher level of speed and accuracy to robust imagery analysis tools that are capable of processing vast amounts of geospatial data every day — consolidating remote sensing, photogrammetry, LiDAR analysis, analytics, and radar processing in a single product.

Those same automation capabilities are also available in the latest smart analytics platforms, bringing a new level of analysis to dynamic visualization of data, in a way that can be securely hosted on networked systems. These systems similarly can provide advanced visualization and integration for multiple types of data, including vector and point cloud data, or even non-geospatial operational data, allowing operators to quickly filter through mounds of information in order to apply the necessary analytics required for decision making.

Similarly, accelerated visualization and analysis solutions can support not only simple data filtering, but additional analytics and querying methods such as styling, heat maps, geofencing, and distance calculations, bringing geospatial Big Data to life to support operations.

Fusing Relevant Data into a Common Operational Picture
A Common Operational Picture (COP) provides common situational awareness for all levels of command and staff across an operation facilitating the decision-making process. Shared situational awareness of friendly and enemy forces, resources, assets, and infrastructure can mean the difference between mission success and failure.

Jim Garmone, a reporter for Department of Defense News, explored the modern-day requirements of military decision-making in his feature on Marine Corps Gen. Joe Dunford, who served as the 19th chairman of the U.S. Joint Chiefs of Staff.

In the article, called, “Dunford: Speed of Military Decision-Making Must Exceed Speed of War,” Dunford said that “information operations, space and cyber capabilities, and ballistic missile technology have accelerated the speed of war, making conflict today faster and more complex than at any point in history.”

To make certain decision-making can be effective and just as timely, Dunford said that there must be “a common understanding of the threat, providing a clear understanding of the capabilities and limitations of the joint force, and then establishing a framework that enables senior leaders to make decisions in a timely manner.”

He went on to say that the joint force “depends on leaders who anticipate change, recognize opportunity, and adapt to meet new challenges.”

For military leaders and their staffs to meet these requirements, they must have access to a rich and dynamic COP, one that can integrate and visualize large amounts of data, including live air, maritime, and land tracks, using military symbology and providing real-time analytics for better situational awareness.

Market-leading platforms that support COPs can accomplish even more. From handling millions of static and live feeds or tracks, to connecting to a range of data sources without the need for translation or conversion, these solutions can interact with and visualize diverse types of data feeds (air, land, sea, space, and cyber) across a network. The latest technology can also visualize and drape data feeds and sources in 3D, including radar and (aerial) video in real-time, providing additional versatility within the COP.

NATO was an early adopter of such technology, beginning back in the 90s. After discovering a customizable and advanced visualization platform, they used that platform to build the interim Geo Spatial Intelligence Tool, or iGeoSIT.

While looking for a platform suitable for their COP, their requirements included finding an affordable solution which could leverage an agile development concept, facilitating a timely development to deployment cycle. They also required a solution that would be easy to learn in the field. Most importantly, they sought a solution that could directly read multiple large data sets in various formats, and would easily visualize a wide array of geographical references while overlaying important operational data (both static and real-time).

Now, almost 20 years after they found the correct platform for bringing iGeoSIT to life, iGeoSIT is the preferred COP tool for NATO and many NATO member countries, supplying the necessary data for military and humanitarian operations.

Filling In the Gaps with Alternative Data Sources
An increasingly common manner of supplementing traditional data sources is through imagery collection using unmanned aerial vehicles. Technological advances are occurring within the UAV space that further extend the tactical edge intelligence gathering capabilities in order to provide a more complete picture to support decision making.

Drones provide a level of resolution and accessibility that satellite data can’t compete with, collecting and providing imagery that broadens the scope of situational awareness. Modern drone solutions can also be deployed in low or no-bandwidth environments, collecting data offline for viewing in the field, command centers, and across military networks.

Other advances include more efficient design, meaning smaller and more compact drones can easily and quickly be deployed by hand, to gather imagery of specific targeted mission areas prior to and during an operation.

Turning Location Intelligence Data into Decisions
As military operations shift to favoring a network model where decision-making is a more distributed process that lends itself to simultaneity, it’s more important than ever that the right data, information, and intelligence be readily available at the tactical, operational, and strategic levels. This is especially crucial as the Information Age continues to produce intelligence solutions with increasing capabilities, which can be accessed from either side of a conflict.

There is a requirement for a real-time integrated COP in order to provide a common shared situational awareness for operations across the levels of command. This requires increased data exploitation, integration, analysis, visualization, and dissemination capabilities that need to leverage AI and machine learning to help overcome the Big Data challenges of today. By achieving this, we will be able to provide that timely and relevant intelligence for decision making.

As Vice Adm. Robert Sharp said, it’s time to “do things differently.” Let’s start by embracing the latest situational awareness technology to better manage operational data and act on it to achieve mission success.

Colonel (retired) Ken Chadder is a Defence Business Development Consultant for Hexagon Geospatial. He joined Hexagon in 2014 after 36 years as a Military Engineer in the Canadian Armed Forces. A graduate of the Royal Military College in Kingston where he received a Bachelor of Engineering, Colonel Chadder held command appointments at all levels during his extensive military career. In his role as Director of JOINTEX, Colonel Chadder gained extensive experience in C4ISR and modeling and simulation.