Goal-Oriented Bundle Management and Flow Control for Deep-Space Communications via Reinforcement Learning

This project tackles three big problems in deep‑space communications: connections that come and go, long signal delays, and limited storage on relay nodes. It uses the Bundle Protocol, which lets a relay store data, carry it, and forward it when a link becomes available. Because each relay has only a small buffer and only brief contact windows, it must decide carefully what to send and when. Simple scheduling rules such as First‑In‑First‑Out (FIFO) don’t work well. They ignore the overall mission goals and can treat data from different planets unfairly. To fix this, we built a closed‑loop model that learns how to manage bundles and data flows on the relay

side. The model treats the relay as a shared buffer that forwards data from the Moon and Mars back to Earth. We measure how fresh the information is at the destination with an “excess Age of Information” (AoI) metric. This metric adds a baseline delay that depends on the source and applies nonlinear penalties for older data. We formulate the relay’s control problem as a constrained semi‑Markov decision process (CSMDP). To solve it, we use a masked Lagrangian Proximal Policy Optimization (PPO) algorithm. The masking step blocks any transmission that falls outside an active contact window, so the solution always respects physical limits. Overall, the framework gives a clear way

to simulate and test goal‑driven scheduling policies. It provides a solid method for improving data usefulness and making the best use of limited resources in future space‑exploration networks.

Message: Hi FE, I need help with “Goal-Oriented Bundle Management and Flow Control for Deep-Space Communications via Reinforcement Learning” in “Telecommunications and Space Systems”