Loughborough University
803_lifelong_reinforcement_learnin (3).pdf (4.58 MB)

Lifelong reinforcement learning with modulating masks

Download (4.58 MB)
journal contribution
posted on 2023-08-18, 14:40 authored by Eseoghene Ben-IwhiwhuEseoghene Ben-Iwhiwhu, Saptarshi NathSaptarshi Nath, Praveen Pilly, Soheil Kolouri, Andrea SoltoggioAndrea Soltoggio

Lifelong learning aims to create AI systems that continuously and incrementally learn during a lifetime, similar to biological learning. Attempts so far have met problems, including catastrophic forgetting, interference among tasks, and the inability to exploit previous knowledge. While considerable research has focused on learning multiple supervised classification tasks that involve changes in the input distribution, lifelong reinforcement learning (LRL) must deal with variations in the state and transition distributions, and in the reward functions. Modulating masks with a fixed backbone network, recently developed for classification, are particularly suitable to deal with such a large spectrum of task variations. In this paper, we adapted modulating masks to work with deep LRL, specifically PPO and IMPALA agents. The comparison with LRL baselines in both discrete and continuous RL tasks shows superior performance. We further investigated the use of a linear combination of previously learned masks to exploit previous knowledge when learning new tasks: not only is learning faster, the algorithm solves tasks that we could not otherwise solve from scratch due to extremely sparse rewards. The results suggest that RL with modulating masks is a promising approach to lifelong learning, to the composition of knowledge to learn increasingly complex tasks, and to knowledge reuse for efficient and faster learning.


United States Air Force Research Laboratory (AFRL) and Defense Advanced Research Projects Agency (DARPA) under Contract No. FA8750-18-C-0103 (Lifelong Learning Machines) and Contract No. HR00112190132 (Shared Experience Lifelong Learning).



  • Science


  • Computer Science

Published in

Transactions on Machine Learning Research


1 - 23


Transactions on Machine Learning Research (TMLR)


  • VoR (Version of Record)

Publisher statement

This is an Open Access Article. It is published by Transactions on Machine Learning Research (TMLR) under the Creative Commons Attribution 4.0 International Licence (CC BY). Full details of this licence are available at: https://creativecommons.org/licenses/by/4.0/

Acceptance date


Publication date


Copyright date




Publisher version


  • en


Dr Andrea Soltoggio. Deposit date: 1 July 2023

Usage metrics

    Loughborough Publications


    No categories selected