Autopilot-Preserving Residual Q-Learning with HJB-Inspired Finite-Action Risk Filtering for Fixed-Wing UAV Command Supervision

This paper introduces a novel concept for the command supervision of fixed-wing UAVs that addresses the issues of maintaining airspeed, altitude, and heading references while adjusting for external factors like wind, gusts, and turbulence. Classical autopilots are great for stabilizing the airframe, but they cannot adjust to changing situations like aggressive maneuvers with crosswinds. On the other hand, direct reinforcement learning approaches at the actuator level run the risk of excessive exploration. This paper aims to place a learned supervisor above an existing autopilot (as opposed to below it) to use a residual Q-learning approach to determine which of the actions residuals (from an a priori chosen, finite, and bounded

set of actions) will adjust the commanded reference airspeed, altitude, and heading. These adjusted references are then clipped to an admissible command envelope and sent to the autopilot, meaning that the autopilot will continue to be the only controller facing the actuators. The most important contribution consists of the ‘residual selection’ method, which combines a Hamilton-Jacobi-Bellman (HJB) risk- filtering approach for finite actions to select candidates for the control barrier/ control Lyapunov/ finite actions control shield. The method described here is meant to enhance UAV vehicle control systems’ adaptive potential and reduce the exploration issues.

Message: Hi FE, I need help with “Autopilot-Preserving Residual Q-Learning with HJB-Inspired Finite-Action Risk Filtering for Fixed-Wing UAV Command Supervision” in “Aerospace Engineering”