Length: 3 Days
Verification, Validation, & Accreditation Training, VV&A Training Course
Verification, Validation, & Accreditation Training, VV&A Training Course provides all aspects of acquisition and systems engineering disciplines employing rigorous methodologies for evaluating and assessing evidence to determine whether a model or simulation should be used in a given situation and establishing its credibility.
VV&A consists of three tightly integrated processes that address verification, validation an accreditation of M&S. Verification and Validation (V&V) functions are performed during the M&S development process and are similar in concept to quality control practices in manufacturing. Accreditation is a formal decision to use a specific M&S and its results for a particular application. Properly performed and documented V&V are essential to the accreditation process. However, it is equally essential that the Accreditation Authority understand the scope and limits of the specific M&S capabilities before concluding that these are applicable and appropriate for the intended use.
M&S techniques and tools assist developers and decision makers throughout the acquisition, operational test and evaluation (OT&E), training, and doctrine development process.
In particular, M&S capabilities support evaluating requirements, performing trade-off studies, understanding and demonstrating system capabilities and performance, training and educating military and civilian personnel, and developing, assessing, and analyzing operational courses of action.
To ensure the efficient and appropriate use of this technology, reference (a) defines M&S and directs the issuance of common VV&A practices and enterprise wide policies that maximize the benefits of M&S and reduce the risk of making a faulty decision based upon M&S results.
The purpose of VV&A is to assure development of correct and valid simulations and to provide simulation users with sufficient information to determine if the simulation can meet their needs.
VV&A processes are performed to establish the credibility of the models and simulations.
Credibility depends on simulation and also depends on the accuracy of a simulation. The decision on whether or not a simulation provides the necessary degree of accuracy depends not only upon the inherent characteristics of the simulation, but also upon how the simulation will be used, and upon the significance of any decisions that may be reached on the basis of the simulation’s outputs.
Credibility for a simulation depends on its correctness, the level of confidence that its data and algorithms are sound, robust, properly implemented, and that the accuracy of the simulation results will not substantially and unexpectedly deviate from the expected degree of accuracy. Credibility also depends, on its usability.
Usability are the factors related to the use of the simulation, such as the training and experience of those who operate it, the quality and appropriateness of the data used in its application, and the configuration control procedures applied to it.
Verification: The process of determining that a model implementation and its associated data accurately represent the developer’s conceptual description and specifications: process of determining whether or not the products of a given phase of the system/software life cycle fulfill the requirements established during the preceding phase.
Validation is the process of determining that the requirements are the correct requirements and that they form a complete set of requirements this is done in the early stages of the development process. Validation of the end product or system determines if the system meets the user’s needs: The process of determining the degree to which a model and its associated data provide an accurate representation of the real world from the perspective of the intended uses of the model.
Accreditation: The official certification that a model, simulation, or federation of models and simulations and its associated data is acceptable for use for a specific purpose. VV&A is a process used to ensure that the application of M&S results is appropriate for a specific purpose, e.g., supporting a system acquisition decision, training an operator, or developing tactical mission plans or doctrine. While sometimes referred to as a single process,
Learning Objectives
Verification, validation, and accreditation (VV&A) seeks to address simulation credibility and acceptance. Focused on simulation structural integrity, verification evaluates whether simulation specifications have been met Validation focuses on simulation representations. Accreditation is a management judgment that simulation credibility (its perceived correctness) is appropriate for use in the application under consideration In other words, verification determines if the simulation was built right, validation determines if the right thing was built, and accreditation determines if it is appropriate for a particular use.
The concept of verification and validation (V&V) is not unique to DoD modeling and simulation (M&S). Historically, V&V has been pivotal in the development of space shuttle software, medical software, and software and simulations that support nuclear reactors.
After the completion of this course, the students will be able to:
- List Verification, Validation, & Accreditation (VV&A) definitions and concepts
- Describe key Verification, Validation, and Accreditation (VV&A) features
- Discuss the distinction between Verification, Validation, and Accreditation (VV&A)
- Analyze VV&A methods and techniques
- Describe VV&A processes in software engineering, Systems Engineering (SE) and military acquisitions
- List VV&A Techniques and Methods
- Describe the modeling, simulation, testing and inspection process and its role in VV&A
- Explain Life Cycle VV&A Tasks
- Explain static analysis as a verification technique
- Creating Accreditation Plan, V&V Plan
- V&V Report and Accreditation Report
Course Topics
Introduction to Verification, Validation, and Accreditation (VV&A)
- VV&A Definitions and Key Concepts
- VV&A Techniques and Methods
- Development of Modeling and Simulation (M&S)
- Planning for VV&A Life Cycle VV&A Tasks
- Measurements Applied to VV&A
- V&V Plan
- V&V Report
- Verification Assessment
- Validation Assessment
- M&S Description
- Accreditation Assessment
- Accreditation Plan
- Accreditation Report
- Verification Validation & Accreditation (VV&A) strategies
- Execution of VV&A Plans
- Supporting V&V data products
- M&S Units Under Test (UUTs)
- Associated V&V activities, agents, and resources for cost-effective VV&A program
Why VV&A
- Important to conduct VV&A
- Planning for VV&A
- Scope of work
- Software Integrity Levels
- Development of the Software VV&A Plan (SVVAP)
- Cost of VV&A
Life Cycle VV&A Tasks
- Acquisition VV&A
- Supply VV&A
- Development VV&A (Concept, Requirements, Design, Implementation)
- Development VV&A (Test)
Role of VV&A in Systems Engineering (SE)
- ConOps
- System Definition
- Requirements Development [System and Sub-system Level Requirements]
- High Level Design [Project Level Architecture]
- Component Level Detailed Design
- System Development and Implementation
- Hardware/Software Development and Unit Test
- Integration [Sub-system and System Level Integration]
- Verification [Sub-system and system level verification]
- Initial System Deployment
- Validation, Operations & Maintenance, Changes & Upgrades
- System Validation
- Validation and assessment of the operational system
- Ensuring the intended purpose and needs of system’s owner and stakeholders
- Importance of developing a simulation model
- Operations & Maintenance
- Changes & Upgrades
- System Retirement/Replacement
Verification, Validation, and Accreditation (VV&A) Roles and Responsibilities
- Modeling and Simulation (M&S) VV&A Recommended Practices
- VV&A process, roles, and responsibilities
- VV&A effort and User/Accreditation Authority
- Data verification and validation (V&V)
- Acceptability criteria
- Accreditation
- Accreditation Authority
- Accreditation Agent
- Accreditation Plan
- M&S Program Manager (M&S PM)
- V&V Agent
- M&S PM’s
- Developers
- Accreditation Agent
V&V Plan
- V&V Plan Executive Summary
- Problem Statement
- Intended Use
- M&S Overview
- M&S Application Accreditation Scope
- V&V Scope
- M&S Requirements and Acceptability Criteria
- M&S Assumptions, Capabilities, Limitations, & Risks/Impacts
- Accreditation Plan Outline
- Accreditation Plan Executive Summary Problem Statement
- Intended Use
V&V Methodology
- Planned Data V&V Tasks/Activities
- Data Verification Tasks/Activities Data
- Validation Tasks/Activities Required Validation Data
- Planned Conceptual Model Validation Tasks/Activities
- Planned Design Verification Tasks/Activities
- Planned Implementation Verification Tasks/Activities
- Define Suite of Tests
- Implementation Verification Tests
- Planned Results
- Validation Tasks/Activities
- Define Suite of Tests
- Results Validation
Issues with V&V
- Key Participants
- Accreditation Participants V&V Participants
- Other Participants
- V&V Report Outline
- V&V Report
- Intended Use
- M&S Overview
- M&S Application
- Accreditation Scope
- V&V Scope
Introduction to Accreditation
- Accreditation Methodology
- Accreditation Information
- Needs Information
- Collection Plan Assessment Plan
- Accreditation Issues Key Participants
- Accreditation Participants
- V&V Participants Other Participants
- Planned Accreditation Resources
- Accreditation Resource Requirements
- Accreditation Milestones and Timeline
- M&S Requirements Traceability Matrix
- Accreditation Programmatics
Accreditation Plan
- Accreditation Plan Executive Summary Problem Statement
- Intended Use
- M&S Overview
- M&S Application
- Accreditation Scope
- M&S Requirements and Acceptability Criteria
- M&S Assumptions, Capabilities, Limitations, & Risks/Impacts
- M&S Assumptions
- M&S Capabilities
- M&S Limitations
- M&S Risks/Impacts
Accreditation Methodology
- Accreditation Information
- Needs Information
- Collection Plan Assessment Plan
- Accreditation Issues Key Participants
- Accreditation Participants
- V&V Participants Other Participants
- Planned Accreditation Resources
- Accreditation Resource Requirements Accreditation Milestones and Timeline
VV&A Case Studies
- VV&A of Complex Modeling and Simulation System
- Methodologies and Case Studies
- Conceptual modeling
- Design and the development of Modeling and Simulation Systems
- Verification and Validation phases
- A simulation model that identifies the probability of a ship to be detected by a submarine dynamically into a specific naval scenario
- Validation and verification methodologies
- Procedures and techniques
- Conceptual model
- Design and implementation
- Validation (structural and results) processes
- VV&A (Verification, Validation & Accreditation) phase
- Modeling, Simulation and VV&A
- Role of MBSE and SysML in VV&A
MBSE and SysML support for the Modeling, Simulation, Specification, Analysis, Design and VV&A: A Case Study
- Problem Summary
- Should results from the model or simulation be believed?
- System and System of SYstems (SoS) Modeling and SysML Diagrams
Package Diagram – Applying the SysML Profile - Establishing Structure of the User Model using Packages and Views (Package Diagram)
- Setting the Context (Boundaries and Use Cases)
- Internal Block Diagram – Setting Context
- Use Case Diagram – Top Level Use Cases
- Elaborating Behavior (Sequence and State Machine Diagrams)
- Sequence Diagram
- Establishing Requirements (Requirements Diagrams and Tables)
- Block Definition Diagram
- Defining, Decomposing, and Allocating Activities
- Analyze Performance (Constraint Diagrams, Timing Diagrams, Views)
- Parametric Diagram – Dynamics
- Parametric Diagram – Measures of Effectiveness (MoE)
- Modeling, Simulation and Analysis
- Modeling and Simulation of Dynamic Systems
- Modeling, Simulation, and Optimization
- Model-based Modeling and Simulation
- Realization of a Complex system using a Conceptual Model
- V&V Challenge
- Verification Validation, and Accreditation (VV&A) Credibility
- Modeling and Simulation Verification and Validation (V&V) Challenges