Skip Navigation

4.3: Introduction to T&E of System-of-Systems and Interoperability

Difficulty Level: At Grade Created by: CK-12
Turn In

Lesson Objectives

  • Describe a system-of-systems
  • Explain the differences between a system and a system-of-systems
  • Describe network-centric
  • Describe interoperability


A collection of interacting or interdependent components organized to form an integrated whole and accomplish a specific function or set of functions. Their parts must be related. They must be designed to work as a coherent entity, otherwise they would be two or more distinct systems. Generally, a system will produce a single function or result. Your MP3 player is a system in that it consists of various components that are combined with the purpose of working together to perform a specific function.1
system-of-systems (SoS)
A set or arrangement that results when independent and useful systems are integrated into a larger system that delivers unique capabilities. A system-of-systems can result in unexpected interactions and unintended consequences. Test and evaluation (T&E) of a system-of-systems must not only assess performance to desired capability objectives, but must also characterize the additional capabilities or limitations due to unexpected interactions. Systems-of-systems is a new and evolving area for development, acquisition, and T&E.2

Check Your Understanding

In previous lessons, you gained insight into how live, virtual, and constructive (LVC) simulations can be applied to test and evaluation (T&E). More specifically, you learned how a distributed LVC environment can be built by connecting various geographically distinct capabilities needed for testing. The interaction of these capabilities, or systems, must now be understood before we can examine the direct applications of using distributed LVC simulation in T&E.


This lesson will address the concepts of the interaction needed of systems to form a system-of-systems in a network-centric environment, and then introduce the very important concept of interoperability. Interoperability between systems is a fundamental requirement for what makes a system-of-systems actually work, so it is a natural focus for testing a system-of-systems.

Lesson Content

What Is a System-of-Systems?

A system can be defined as a functionally, physically, or behaviorally related group of regularly interacting or interdependent elements that form a unified whole. Test and Evaluation (T&E) is used to assess a system for its ability to achieve a desired effect or to perform a set of tasks under specified standards and conditions. T&E on a system can be either technical T&E or user T&E.

When independent systems are integrated into a larger system, it is known as a system-of-systems, or SoS. A SoS delivers unique capabilities and interactions that are not found in individual systems. Both individual systems and SoS conform to the accepted definition of a system, in that each consists of parts, relationships, and a whole that is greater than the sum of the parts; however, although a SoS is a system, not all systems are SoS.3 For instance, a Nintendo Wii is a system (Figure below). The individual parts of a Nintendo Wii include the console, wireless controller, a separate display screen, and the Wii remote, which can be used as a handheld pointing device and detects movement in three dimensions. But all these parts are components that are integrated into a larger system. Complex, yes, but these components cannot stand alone. Each component needs the rest of the components to be able to function properly.

Your Nintendo Wii is a system. It is a collection of interacting components organized to form an integrated whole and accomplish a specific function. (Courtesy of Wikipedia)

An airport is a system-of-systems. Shown above is the Dulles International Airport in Virginia, near Washington, D.C. The individual systems include aircraft like the Boeing 777 discussed earlier, radio systems, multiple radar systems, baggage handling systems, food service, public address systems, and even individual ground transportation vehicles. Each of these systems performs an independent and specialized function. When these individual systems are integrated into a larger system-of-systems, they become interdependent and perform unique interactions that provide a distinctive capability. (Courtesy of Wikipedia)

SoS is a rapidly changing area for product development and T&E. The SoS concept should include the system in the broadest sense, from performance of specific components to the network-centric interaction of an entire SoS. The exchange of data in a SoS can result in unexpected interactions and unintended consequences. T&E of a SoS will have unique requirements due to the complexity of the communication and data exchange within the individual systems. T&E of SoS must not only assess the desired performance capability of individual systems, but must also assess the additional capabilities or limitations due to these unexpected interactions. Early testing, both technical and user testing, is especially important in a SoS to help to mitigate the risk of "late-cycle churn" (defined in the lesson "What Is Test and Evaluation?"). Testing in a distributed LVC environment can more easily, and with fewer resources, replicate the intended SoS and its operating environment. LVC simulations, Hardware-in-the-Loop (HWIL), or a Systems Integration Laboratory (SIL) provide a unique capability to conduct testing of SoS and is a practical and efficient way to build a test environment for complex SoS. In fact, using distributed LVC simulations may be the only practical way to discover the unexpected interactions and untended consequences common to a SoS.

What Is Network-Centric?

Network-centric, or "net-centric," refers to participating as a part of a continuously evolving, complex community of people, devices, information, and services interconnected by a communications network to achieve optimization of resources and better synchronization of events and their consequences. Many experts believe the terms "information-centric" or "knowledge-centric" would capture the concepts more aptly because the objective is to find and exploit information. The network itself is only one of several enabling factors along with sensors, data processing and storage, expert analysis systems and intelligent agents, and information distribution. The shift away from point-to-point system interfaces to network-centric interfaces brings significant implications for the T&E community.4

The net-centric environment is a framework for full human and technical connectivity. It allows product users to share the information they need, when they need it, and in a form they can understand and act on with confidence.5 It should also protect information from those who should not have it. Net-centric systems for T&E enable users to handle information only once, post data before processing it, access data when it is needed, collaborate to make sense of data, and diversify network paths to provide reliable and secure network capabilities. This allows for improved resource management and provides better information on events and conditions needed by the developer and test planners.

The challenge to the T&E community is to represent the integrated architecture in the intended operating environment for the product, systems, and system-of-systems — that is, in a net-centric environment. It is imperative that the T&E community develop net-centric test planning that reflects the intended operational environment and interfaces within which the intended products and capabilities must be tested. This would include characteristics of individual systems and network characteristics and factors such as network loading, bandwidth, throughput, packet loss, latency, and jitter.6 While these terms of network performance are beyond the scope of this lesson, the student should understand that each of these characteristics form only a part of a very complicated process that will need to be addressed when conducting T&E in a network-centric environment.

The continental U.S. power transmission grid is a good example of a net-centric system-of-systems. It consists of about 300,000 km of lines operated by approximately 500 companies. Transmission networks are complex, with redundant pathways and a wide-area synchronous grid, or "interconnection." This allows transmission of power throughout the area, connecting a large number of electricity generators and consumers and potentially enabling more efficient electricity markets and redundant generation. Electricity generation and consumption must be balanced across the entire grid because energy is consumed almost immediately after it is produced. Remember that the exchange of data in a system-of-systems can result in unexpected interactions and unintended consequences. Likewise, a large failure in one part of the power grid — unless quickly compensated for — can cause current to re-route itself to flow from the remaining generators to consumers over transmission lines of insufficient capacity, causing further failures. One downside of a net-centric system-of-systems, such as a widely connected power grid, is the potential of cascading failure and widespread power outage if the grid is not managed properly. The challenge for T&E planners is developing the capability to test products in this type of distributed net-centric environment.7

What Is Interoperability?

Interoperability is defined as the ability of systems or components to provide data, information, and services to, and accept the same from, other systems or components and then to use the data, information, and services to enable them to operate effectively together. Interoperability is absolutely required of systems and SoS operating in a net-centric environment. It is this interoperability that enables systems that have been connected to act as a system-of-systems.

Assessing interoperability performance in a net-centric environment for system-of-systems is a key element of using distributed LVC simulations for T&E.

The complexity and expense of today’s net-centric systems clearly demonstrate the need to test early and often throughout the development and fielding process of a product. Early testing of a system’s capability to operate in its intended environment will allow designers and system engineers to identify and correct fundamental issues with performance and interoperability before they become operational specifications. As the transition to net-centric SoS is accelerated, the requirement to successfully demonstrate systems interoperability will increase. Thus, the need to use distributed testing with LVC simulations becomes the only feasible method to efficiently confirm system maturity, effectiveness, and performance.

Lesson Summary

  • A system-of-systems can be defined as the configuration or arrangement that results when independent and specific systems are integrated into a larger system, thus delivering unique capabilities and interactions.
  • Network-centric, or "net-centric," refers to participating as a part of a continuously evolving, complex community of people, devices, information, and services interconnected by a communications network.
  • Interoperability is the ability of systems or components to provide and accept data, information, and services that enable them to operate effectively together.

Review Questions

1. Describe the differences between a system and a system-of-systems.

2. List the two significant features of a system-of-systems.

3. How does a network-centric capability affect distributed T&E?

4. Why is interoperability between systems important?

Further Reading/Supplemental Links

http://www.youtube.com/watch?v=ZMDROyNXrvo Go to this link to view how a car company used distributed capabilities to design a new car. Note the globalization where design team is one place and engineering team is another place.

Points to Consider

For this lesson, points to consider are provided as an aid to the instructor to stimulate critical thinking among the students. These questions have no right or wrong answers, but may help further the student’s understanding of the material. Suggested responses are provided to help the instructor guide the discussion.

  • We now have all the "parts" necessary to understand T&E using distributed live, virtual, and constructive simulations.
    • What practical applications do you see? (Suggested response: When properly understood and planned for, distributed LVC methodologies will apply to the T&E of most products and systems that must interact or exchange data with other products or systems. The most dramatic applications will be in large system-of-systems testing, due in part to the difficulty and expense in developing live-only testing scenarios.
    • What benefits do you envision? (Suggested response: When live systems do not need to be co-located to complete a test event, there will be immediate cost and time savings to the T&E program, simply because of the expensive and time-consuming nature of collecting these systems in one place. That in turn will have multiple affects in how realistic that testing can be, how early the product or system can be tested, and eventually reduce the cost of making corrections to the design and development of the product. That will ultimately result in a better, quicker, and cheaper product to the customer.
    • What are some potential drawbacks? (Suggested response: Distributed LVC testing will not be cost-free. There will be a lot of "up front" costs, and it can be difficult to provide the network and environment needed. Distributed testing will not be appropriate for every T&E situation. Also, it is a relatively new concept, and in some cases it can appear to be "too hard" to do. These drawbacks will be discussed in the next lesson, but it should be stressed that, for many if not most cases, distributed LVC testing will save the developer time and money.
  • How does network-centric affect T&E planners? (Suggested response: Net-centric systems-of-systems will present T&E planners a very complicated problem in designing accurate environments for test events that will replicate the "real world" in which the product or products will operate. They will have to anticipate and account for unexpected and unintended interactions of the various systems. This is important, as the developer will need to know what deficiencies belong to his product(s) and what problems are due to other system(s) or even the environment.


1 DoD 5000.59-M “DoD Modeling and Simulation Glossary”

2 Defense Acquisition Guidebook, July 29, 2010, Chapter 9, Test and Evaluation

3 Director, Systems and Software Engineering, Deputy Under Secretary of Defense (Acquisition and Technology), Office of the Under Secretary of Defense, (Acquisition, Technology and Logistics), Aug 2008, http://www.acq.osd.mil/se/docs/SE-Guide-for-SoS.pdf

4 Webster’s Dictionary (online), http://www.websters-online-dictionary.org/definitions/Net-Centric

5 Net-Centric Environment Joint Functional Concept, Version 1.0, Apr 2005, http://www.carlisle.army.mil/dime/getDoc.cfm?fileID=30

6 Encyclopedia of Networking and Telecommunications, McGraw Hill, Thomas Sheldon, 2001

7 Wikipedia article entitled “Electrical Grid,” http://en.wikipedia.org/wiki/Electrical_grid

Notes/Highlights Having trouble? Report an issue.

Color Highlighted Text Notes
Show More

Image Attributions

Show Hide Details
Date Created:
Aug 06, 2012
Last Modified:
Jan 30, 2016
Save or share your relevant files like activites, homework and worksheet.
To add resources, you must be the owner of the section. Click Customize to make your own copy.
Please wait...
Please wait...
Image Detail
Sizes: Medium | Original