Multi-Agent Simulation for Coordinated Aerial Surveillance using Digital Twin and SITL Framework

Abstract

The increasing complexity of aerial surveillance tasks in urban and industrial environments demands scalable, coordinated, and resilient multi-UAV systems. This paper presents a Digital Twin–Software-in-the-Loop (DT–SITL) framework for learning-based cooperative aerial surveillance, enabling systematic development and validation of multi-drone tracking behaviors prior to physical deployment. Within the proposed framework, each UAV is represented by a synchronized Digital Twin instance and controlled by a shared Proximal Policy Optimization (PPO) policy, allowing decentralized execution while preserving coordinated system-level behavior. The framework integrates UAV dynamics, target motion modeling, structured observation construction, and reward-driven multi-agent learning into a unified DT–SITL execution loop. Simulation results involving three UAVs tracking a dynamically maneuvering target demonstrate stable policy convergence, bounded tracking error, and emergent cooperative behaviors such as spatial distribution and persistent engagement. System-level analysis shows that the DT–SITL approach enables safe experimentation, repeatable evaluation, and scalable coordination analysis, establishing a practical foundation for future autonomous multi-UAV surveillance systems and their transition toward real-world deployment.