Robot Policy Learning from Demonstration with Advantage Weighting and Early Termination

• Abdalkarim Mohtasib, Heriberto Cuayahuitl

  Lincoln Centre for Autonomous Systems

  University of Lincoln

  Lincoln, UK

  
  

  Paper | Code | Data

Abstract

Learning robotic tasks in the real world is still highly challenging and effective practical solutions remain to be found. Traditional methods used in this area are imitation learning and reinforcement learning, but they both have limitations when applied to real robots. Combining reinforcement learning with pre-collected demonstrations is a promising approach that can help in learning control policies to solve robotic tasks. In this paper, we propose an algorithm that uses novel techniques to leverage offline expert data using offline and online training to speed up policy convergence and improved performance. The proposed algorithm (AWET) weights the critic losses with a novel agent advantage weight to improve over the expert data. In addition, AWET makes use of an automatic early termination technique to stop and discard policy rollouts that are not similar to expert trajectories---to prevent drifting far from the expert data. In an ablation study, AWET showed improved and promising performance when compared to state-of-the-art baselines on four standard robotic tasks.

Research Method

More is coming soon!!!

Contact

For comments/questions, contact Abdalkarim Mohtasib