Meng_1-s2.0-S0921889021001767-main.pdf (36.75 MB)

Hybrid autonomous controller for bipedal robot balance with deep reinforcement learning and pattern generators

Download (36.75 MB)
journal contribution
posted on 20.09.2021, 13:08 by Christos Kouppas, Mohamad SaadaMohamad Saada, Qinggang MengQinggang Meng, Mark KingMark King, Dennis Majoe
Recovering after an abrupt push is essential for bipedal robots in real-world applications within environments where humans must collaborate closely with robots. There are several balancing algorithms for bipedal robots in the literature, however most of them either rely on hard coding or power-hungry algorithms. We propose a hybrid autonomous controller that hierarchically combines two separate, efficient systems, to address this problem. The lower-level system is a reliable, high-speed, full state controller that was hardcoded on a microcontroller to be power efficient. The higher-level system is a low-speed reinforcement learning controller implemented on a low-power onboard computer. While one controller offers speed, the other provides trainability and adaptability. An efficient control is then formed without sacrificing adaptability to new dynamic environments. Additionally, as the higher-level system is trained via deep reinforcement learning, the robot could learn after deployment, which is ideal for real-world applications. The system’s performance is validated with a real robot recovering after a random push in less than 5 seconds, with minimal steps from its initial position. The training was conducted using simulated data.


EPSRC Centre for Doctoral Training in Embedded Intelligence

Engineering and Physical Sciences Research Council

Find out more...



  • Science
  • Sport, Exercise and Health Sciences


  • Computer Science

Published in

Robotics and Autonomous Systems




AM (Accepted Manuscript)

Rights holder

© The authors

Publisher statement

This is an Open Access Article. It is published by Elsevier under the Creative Commons Attribution 4.0 Unported Licence (CC BY). Full details of this licence are available at:

Acceptance date


Publication date







Prof Qinggang Meng. Deposit date: 6 September 2021