Explainable AI for Autonomous Decision-Making in Unmanned Systems: A Systematic Review

Abstract

The increasing deployment of AI-driven autonomous unmanned systems in safety-critical applications demands transparency in decision-making processes to enable human oversight, facilitate debugging, ensure regulatory compliance, and build operator trust. This systematic review comprehensively examines explainable AI (XAI) methods for autonomous decision-making across perception, planning, and control functions in unmanned systems. We categorize XAI approaches into three paradigms: post-hoc explanation methods that interpret black-box models through saliency maps, attention visualization, and counterfactual analysis; intrinsically interpretable models including decision trees, rule-based systems, and linear models with inherent transparency; and hybrid architectures combining neural network performance with symbolic reasoning for explainability. The review analyzes explanation modalities spanning visual explanations highlighting image regions influencing perception decisions, textual explanations providing natural language justifications for actions, and interactive explanations enabling operator queries about system reasoning. Particular emphasis is placed on domain-specific requirements: real-time explanation generation for time-critical decisions, explanation fidelity ensuring accurate representation of actual decision processes, and explanation utility measured by operator comprehension and trust calibration. We examine evaluation methodologies for XAI including human subject studies assessing explanation quality, computational metrics measuring explanation consistency, and task-based evaluations quantifying performance improvements from explanations. Application-specific implementations are analyzed for autonomous vehicles explaining trajectory decisions, UAVs justifying target classifications, and multi-agent systems clarifying coordination strategies. The review identifies critical research gaps including lack of standardized evaluation frameworks, limited investigation of explanation effects on long-term operator performance, and insufficient attention to explanation security against adversarial manipulation.