Loughborough University
Browse
2004.12846.pdf (4.14 MB)

Evolving inborn knowledge for fast adaptation in dynamic POMDP problems

Download (4.14 MB)
conference contribution
posted on 2020-05-11, 13:38 authored by Eseoghene Ben-Iwhiwhu, Pawel Ladosz, Jeff DickJeff Dick, Wen-Hua ChenWen-Hua Chen, Praveen Pilly, Andrea SoltoggioAndrea Soltoggio
Rapid online adaptation to changing tasks is an important problem in machine learning and, recently, a focus of meta-reinforcement learning. However, reinforcement learning (RL) algorithms struggle in POMDP environments because the state of the system, essential in a RL framework, is not always visible. Additionally, hand-designed meta-RL architectures may not include suitable computational structures for specific learning problems. The evolution of online learning mechanisms, on the contrary, has the ability to incorporate learning strategies into an agent that can (i) evolve memory when required and (ii) optimize adaptation speed to specific online learning problems. In this paper, we exploit the highly adaptive nature of neuromodulated neural networks to evolve a controller that uses the latent space of an autoencoder in a POMDP. The analysis of the evolved networks reveals the ability of the proposed algorithm to acquire inborn knowledge in a variety of aspects such as the detection of cues that reveal implicit rewards, and the ability to evolve location neurons that help with navigation. The integration of inborn knowledge and online plasticity enabled fast adaptation and better performance in comparison to some non-evolutionary meta-reinforcement learning algorithms. The algorithm proved also to succeed in the 3D gaming environment Malmo Minecraft.

Funding

United States Air Force Research Laboratory (AFRL) and Defense Advanced Research Projects Agency (DARPA) under Contract No. FA8750-18-C0103

History

School

  • Science
  • Aeronautical, Automotive, Chemical and Materials Engineering

Department

  • Aeronautical and Automotive Engineering
  • Computer Science

Published in

GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference

Pages

280 - 288

Source

Genetic and Evolutionary Computation Conference (GECCO 2020)

Publisher

Association of Computing Machinery (ACM)

Version

  • AM (Accepted Manuscript)

Rights holder

© ACM

Publisher statement

© ACM 2020. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in GECCO '20: Proceedings of the 2020 Genetic and Evolutionary Computation Conference, http://dx.doi.org/10.1145/3377930.3390214

Publication date

2020-06-25

Copyright date

2020

Notes

Accepted as a full paper in the Genetic and Evolutionary Computation Conference (GECCO 2020)

ISBN

9781450371285

Language

  • en

Location

Electronic-only conference

Event dates

July 8th-12th 2020

Depositor

Dr Andrea Soltoggio. Deposit date: 9 May 2020

Usage metrics

    Loughborough Publications

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC