Reinforcement Learning for Congestion Control in networked environments

<p dir="ltr">Congestion Control (CC) is an important computer network management task that ensures network paths are utilised effectively, without unnecessary delays or data loss. Reinforcement Learning (RL) is a branch of AI that has emerged as a powerful tool for optimising sequential decision-making processes. Recent years have seen growth in interest in applying RL to managing Congestion Control (CC) in computer networks to handle diverse kinds of traffic that may be challenging for human-designed algorithms. This thesis investigates the design of the learning environment of RL, proposes a hybrid approach to training, and introduces an approach to safe RL training in CC. Designing a learning environment is a complex challenge in RL, consisting of numerous design choices that are not well understood. This thesis evaluates different design strategies for designing the state space, action space, and reward function, and examines how they contribute to learning. The metrics used in state space development significantly affect the RL agent training process. This thesis also proposes a hybrid approach for RL in CC that mitigates the limitations of random exploration by incorporating a rule-based guide to aid training. This hybrid approach, known as Jumpstart Reinforcement Learning (JSRL), utilises expert knowledge to guide RL training and outperforms the classical RL approach. JSRL is evaluated across six different network environment settings and shows greater utility and lower loss than classical RL. Finally, the important problem of RL safety is explored using a cost constraint to guide training and obtain an agent that takes safer actions in the CC context.</p>