Sensor and Data Fusion Training Bootcamp

Sensor and Data Fusion Training Bootcamp

Sensor and data fusion technology refers to the use of multiple sensors to collect data from the same target, analyze and synthesize the collected data using computer technology, and form data with high accuracy and low redundancy to support the decision-making process.

In general, the objective of data fusion is to improve overall system performance, including:

• Improved decision making
• Increased detection capabilities
• Diminished number of false alarms
• Improved reliability

Different data fusion methods have been developed in order to optimize the overall system output in a variety of applications for which data fusion might be useful: security (humanitarian, military), medical diagnosis, environmental monitoring, remote sensing, robotics, etc.

The concept of sensor fusion attempts to replicate the capability of the central nervous system to process sensory inputs from multiple sensors simultaneously.

Sensor and data fusion has a variety of applications such as in GPS/INS. In this applications, Global Positioning System and inertial navigation system data is fused using various different methods like the extended Kalman filter (EKF), the nonlinear version of the Kalman filter which linearizes about an estimate of the current mean and covariance.

Sensor and data fusion is also useful in determining the attitude of an aircraft using low-cost sensors.

Additionally, a data fusion approach can determine the traffic state (low traffic, traffic jam, medium flow) using road side collected acoustic, image and sensor data.

Although technically not a dedicated sensor fusion method, modern Convolutional neural network based methods can simultaneously process very many channels of sensor data (such as Hyperspectral imaging with hundreds of bands and fuse relevant information to produce classification results.

The fusion of radar sensor and multi-purpose camera data is also highly relevant for automated driving. The Bosch road signature makes it possible for automated vehicles to determine their precise position and enables highly accurate and robust vehicle localization based on road features.

Sensor and data fusion has also become prominent in artificial intelligence (AI).

The AI algorithm can employ sensor fusion to use the data from one sensor to compensate for weaknesses in the data from other sensors.

Consequently, the AI algorithm can classify the relevance of each sensor to specific tasks and minimize or ignore data from sensors determined to be less important.

The AI/sensor fusion relationship has several benefits, including:

• The AI algorithm can employ sensor fusion to use the data from one sensor to compensate for weaknesses in the data from other sensors.
• The AI algorithm can classify the relevance of each sensor to specific tasks and minimize or ignore data from sensors determined to be less important.
• Through continuous training at the edge or in the cloud, AI/ML algorithms can learn to identify changes in system behavior that were previously unrecognized.
• The AI algorithm can predict possible sources of failures, enabling preventative maintenance and improving overall productivity.

Sensor and Data Fusion Training Bootcamp Course by Tonex

Sensor and Data Fusion Training Bootcamp covers technologies, tools and methods to automatically manage multi sensor data filtering, aggregation, extraction and fusing data useful to intelligence analysts and war fighters.

Learn about application of artificial neural network technology to data fusion for target recognition, airborne target recognition, activity-based intelligence, C4ISR, Electronic Warfare (EW), radar and EO-IR thermal imaging sensors, missile defense, cyber warfare, air, space and maritime surveillance, net-centric warfare, Activity-based Intelligence, effects-based operations process control, proactive maintenance and industrial automation.

Data fusion is a data analysis technique that combines and correlates data about a single subject from different sources, to be able to derive additional insights and intelligence from that data.

The intelligence extracted from data fusion makes it easier for analysts to identify potential adversaries and their targets. By organizing and collecting large volumes of collected data, relevant patterns can be recognized using data fusion.

The artificial neural network based fusion architectures are discussed along with multi-source and multi-sensor data fusion, alternative learning algorithms, parallel distributed processing, real-time automatic target recognition decisions based on performance, and large volumes of structured, unstructured data from disparate sources and low vs. high-level fusion.

Learning Objectives:

After completing this course, the participants will be able to:

• Define what data fusion approach is
• Identify the motivating factors behind data fusion
• Explain value proposition of data fusion
• Define the key features of data fusion and sensor integration
• List the functional requirements of multisensory fusion
• List the four pillars of data fusion and Multi-Sensor Data Fusion
• Understand what data fusion and its applications are
• List the principal components of data fusion systems
• Define Extracting, Transforming and Loading (ETL) operations
• Practice using the data fusion
• Select the right data fusion systems for your mission and application
• Assemble and manage information gathering and integration from a variety of sources
• Apply cutting-edge tools, methods and techniques for multi-sensor integration
• Explain application of multisensory fusion applied to identification, target tracking, net-centric, TDL, situational and threat assessment

Who Should Attend

• Project managers, product managers, software and systems engineers, EW and TDL operators, scientists, R&D, military and law enforcement.

Course Agenda

Introduction to Data Fusion

• Basic Concepts
• Data Fusion: who, why and when
• Data Fusion Application
• Data Fusion Modeling Technique
• Multisensor Fusion
• Simple example illustrating the data fusion principle
• Data Base A
• Data Base B
• Data unique to A and B
• Common Characteristics
• Data Alignment and Association
• Linking via Common Characteristics
• Fused Database
• Data from A and B
• The Fusion Process
• Enhanced targeting
• Linking variables
• The key to a reliable data fusion
• Data Fusion Validation
• Key Elements of the Validation Process Split-sample Validation
• Data Fusion Architectures
• Report-to-Track
• Track-to-Track
• Associated Measurement Reports
• Fusion of Data From Multiple Radars
• Fusion of Data From Radar and Angle-Only Sensor

Intelligence Analysis and Data Fusion

• Collate, Evaluate and Analyze Multiple Types of Information
• Data Integrating and Interpreting
• Information into usable Intelligence
• Manipulating Query Expressions
• Describing Data Sources
• String Matching
• Schema Matching and Mapping
• General Schema Manipulation Operators
• Data Matching
• Query Processing
• Data Warehousing and Caching
• Taxonomy, Ontologies and Knowledge Representation
• Incorporating Uncertainty into Data Integration
• Perception and Sensing
• Sensor and Data Fusion
• Dynamic World Modeling
• General Framework for Data Fusion
• Principles for Integrating Multi-Sensor Information
• Techniques for Fusion of Numerical Properties
• Prediction
• Discrete State Transition Equations
• Matching Observation to Prediction
• The Mahalanobis Distance
• The Kalman Filter Update Equations
• Fusion of Symbolic Properties
• Principles for Symbolic Fusion

Principles of Sensor Fusion

• Sensor Fusion Overview
• Fusion Applications
• Terminology for Fusion Systems
• Sensor Fusion vs. Data Fusion
• Information Fusion
• Multi-sensor Data Fusion
• Multi-sensor Integration
• ​Track objects in real time with Sensor Fusion
• Errors in Raw Data
• Multisensor Fusion Architecture
• Architectural Taxonomy
• Fusion Methods
• Bayesian Networks
• Probabilistic Grids
• The Kalman Filter
• Markov chain Monte Carlo
• Alternatives to Probability

Real-time Sensor Fusion

• Technical Strategic Intelligence Data
• Robotics
• Autonomous Cars
• Fusing Lidar and Radar Data
• Multi-Sensor Data Fusion
• Tactical Data Link (TDL)
• Signals Intelligence (SIGINT)
• Electronic Warfare (EW)
• Advanced Radar Signals Collection and Analysis (ARSCA)
• Hyperspectral and Multispectral Sensing
• Fundamentals of Synthetic Aperture Radar Signal Processing
• Electro-Optical and Infrared EO/IR Systems
• Autonomous Unmanned Systems
• Battlefield Acoustics Signal Processing

Multi-Sensor Data Fusion

• Target Tracking and Identification
• Situation and Threat Assessment
• State of the art
• Test and Evaluation issues and methods
• Cognitive Error and Bias
• Advanced Analysis Techniques
• Threat Analysis
• Fusing information and Building Intelligence

Multi-level Data Fusion Modeling

• Planning and Tools
• Tasking
• Real-time Monitoring
• Secondary Exploitation
• Level 1 Fusion
• Object Refinement
• Fusing multiple heterogeneous data sources
• Higher Level Fusion dealt with Levels 2-4
• big conceptual jump from Level 1 to Level 2
• Situation Refinement
• Threat Refinement
• Process Refinement
• Sensors and Platforms (manned and unmanned)
• Effects options
• Networks and Terminals
• Information Exchange Requirements (IERs)
• Encryption and Data Integrity

 Multi-Sensor Data Fusion Architecture, Design and Implementation

• Sensor Types
• Sensor Common Characteristics
• Fusion System Design
• Architectures for Multi-Sensor Data Fusion
• Joint Directorate of Laboratories (JDL) Architecture
• Observe-Orient-Decide-Act (OODA) Loop
• Foundational Technologies
• Theory of Probability and Statistics
• Statistical Distributions
• Conjugate Distributions for Bayesian Inference
• Monte Carlo Simulation Techniques
• Bayesian Belief Networks
• Intelligent Agents
• Software Tools
• iDAS and Text
• Bayesian Belief Network Engine
• Argumentation Engine
• MATLAB
• Kalman Filters
• Lidar and Radar Fusion with Kalman Filters
• Hyperspectral and Multispectral Sensing
• Semantic Methods
• Decision Fusion Techniques
• Semantic Information Fusion

Classification of Data Fusion Techniques

• Classification Based on the Relations between the Data Sources
• Classification Based on the Abstraction Levels
• JDL Data Fusion Classification
• The JDL data fusion framework
• Classification Based on the Type of Architecture
• Data Association Techniques
• Nearest Neighbors and Means
• Probabilistic Data Association
• Joint probabilistic data association (JPDA)
• Multiple Hypothesis Test
• Distributed Joint Probabilistic Data Association
• Distributed Multiple Hypothesis Test
• Maximum Likelihood and Maximum Posterior
• The Kalman Filter
• Decision Fusion Methods
• The Bayesian Methods
• The Dempster-Shafer Inference
• Abductive Reasoning

Data Fusion and Activity-Based Intelligence (ABI)

• Process Flow for Intelligence
• Collect, characterize and locate activities and transactions
• Target Motion and Measurement Models
• Activity-Based Intelligence Methodology
• Intelligence Discipline of Activity-based Intelligence (ABI)
• Activity-based Intelligence (ABI) Applied
• ABI and Effective Tactical Data Science
• Multispectral and Hyperspectral Imaging Techniques
• Medical Engineering & Physics
• Wearable sensors
• Sensor placement
• Data Fusion Algorithms
• Signal level algorithms
• Weighted averages
• The Kalman filter (KF)
• Particle filtering (PF)
• k-Nearest Neighbor (k-NN)
• Decision Trees (DT)
• Support Vector Machines (SVM)
• Artificial Neural Network (ANN)
• Gaussian mixture model (GMM)
• k-Means
• Decision level algorithms
• Bayesian inference
• Fuzzy logic
• Applications of data fusion for health monitoring
• Activity recognition
• Fall detection and prediction
• Gait and ambulatory monitoring
• Analysis of Multiple Sensors and JDL Data Fusion Model
• Decision on Levels of Fusion

Sensor and Data Fusion Training Bootcamp