Investigating the influence of autonomy controllability and observability on performance, trust, and risk perception

Abstract Given the rise in autonomous vehicles in domains like space and underwater exploration, military applications, and surface transportation, human supervisors must often remotely assess risk in path navigation tasks under uncertainty. However, human risk assessment often diverges from an autonomous planner's quantitative calculation of risk, which can lead to degraded system performance. To further investigate this, an experiment was conducted to investigate the impact of controllability and observability on participants’ performance, self-confidence, trust, and risk assessments in uncertain environments. Observability was expressed through a risk budget representation and controllability allowed participants to directly control the time horizon of autonomy-generated paths through different path leg lengths. Results demonstrated that observability of the risk budget did not impact performance but reduced self-confidence and trust in the autonomy when map complexity (i.e., number of obstacles) was high. Controllability of path planning algorithm leg lengths helped participants respect algorithm soft constraints, but they tended to take more risk as they approached the goal. This effort demonstrates that while risk-aware autonomy can generate optimized paths and quantify corresponding risks, designing interfaces to close the risk perception gap and enhance operator performance while promoting appropriate trust may not always have the intended effect.