Statistical methods for task detection in lifelong reinforcement learning

Lifelong reinforcement learning is a growing field where artificial intelligence agents are expected to learn multiple tasks over a lifetime. Great strides have been made in the field recently, with agents being able to adapt to multiple tasks, and even taking advantage of experience from previous tasks when new tasks are encountered. However, the majority of those agents rely on being told when the task changes or an explicit task label. Labelling these task changes in the lifelong reinforcement learning case is non-trivial as reinforcement learning environments have temporal dynamics and sometimes large state spaces.

In this thesis I introduce a variety of statistical methods for detecting and labelling tasks, in order to empower existing lifelong learning agents to work in environments where task labels are not provided. Specifically, two algorithms are introduced: Adaptive Model Detection (AMD), for the discrete Partially Observable Markov Decision Process (POMDP) setting; and Sliced Wasserstein Online Kolmogorov-Smirnov (SWOKS), for the deep lifelong reinforcement learning setting. Comparisons are made in the latter case between different goodness-of-fit statistical tests.

The methods introduced in this thesis are assessed in lifelong learning environments with task interference. Lifelong learning agents equipped with the novel task labelling modules are shown to achieve a higher overall reward compared to agents with no task labels provided. With artificial intelligence being used in an expanding range of real-world problems, statistical methods of detecting changes in the environment may be necessary for the long-term deployment of systems in non-stationary applications.