Safe Testing of Autonomous Systems Performance

The role of unmanned platforms is rapidly expanding across a wide range of defense and homeland security missions. Currently operational unmanned vehicles are “tele-operated”, using a command and control link to a remotely located pilot. However, operational complexity, operational pace, and a need to function in communication denied environments necessitate a trend toward autonomous unmanned vehicles. Autonomous systems that make independent decisions in complex engagements, such as the Navy’s Autonomous Aerial Cargo Unmanned System, are currently under development and will require development and operational testing within the next 3-5 years. Testing of autonomous systems presents some unique and vexing challenges. For instance, the infinite number of variations of test conditions that can exist to stimulate autonomous behaviors and the complexity of the interactions that can occur among multiple autonomous systems combine to make comparative measurement of autonomous system performance extremely difficult. Also, the inherent unpredictability of decision making by autonomous systems may result in decisions that are considered unsafe by managers of live test ranges. Advanced test and evaluation techniques that focus on the unique challenges of autonomy represent a clear and increasing need within the DoD. The Safe Testing of Autonomy in Complex, Interactive Environments (TACE) Program is a research initiative to develop an advanced test infrastructure that can measure the performance of autonomous systems operating in complex Live-Virtual-Constructive (LVC) environments while ensuring that the autonomous system does not violate range safety policy. This paper will provide an overview of the TACE hardware and software architecture and will highlight the LVC testing that has been performed at the Aberdeen Test Center to validate TACE capabilities. A discussion of anticipated transition activities with DoD partner programs will also be provided.