Publication
Dynamic Experience Replay
This paper presents a novel technique in reinforcement learning that largely improves the training efficiency, especially in contact-rich robotic assembly tasks.
Download publicationAbstract
Dynamic Experience Replay
Jieliang Luo, Hui Li
Conference on Robot Learning 2019
We present a novel technique called Dynamic Experience Replay (DER) that allows Reinforcement Learning (RL) algorithms to use experience replay samples not only from human demonstrations but also successful transitions generated by RL agents during training and therefore improve training efficiency. It can be combined with an arbitrary off-policy RL algorithm, such as DDPG or DQN, and their distributed versions. We build upon Ape-X DDPG and demonstrate our approach on robotic tight-fitting joint assembly tasks, based on force/torque and Cartesian pose observations. In particular, we run experiments on two different tasks: peg-in-hole and lap-joint. In each case, we compare different replay buffer structures and how DER affects them. Our ablation studies show that Dynamic Experience Replay is a crucial ingredient that either largely shortens the training time in these challenging environments or solves the tasks that the vanilla Ape-X DDPG cannot solve. We also show that our policies learned purely in simulation can be deployed successfully on the real robot.
Associated Autodesk Researchers
Jieliang (Rodger) Luo
Sr. Principal AI Research Scientist
Related Resources
2023
A Hybrid Intelligence Approach to Training Generative Design Assistants: Partnership Between Human Experts and AI Enhanced Co-Creative ToolsThe research presents a framework for designing and evaluating…
2023
CAD-LLM: Large Language Model for CAD GenerationThis research presents generating Computer Aided Designs (CAD) using…
2022
A Discretization-free Metric For Assessing Quality Diversity AlgorithmsA multi-scale generative design model that adapts the Wave Function…
2022
T-Domino: Exploring Multiple Criteria with Quality-Diversity and the Tournament Dominance ObjectiveA new ranking system for Multi-Criteria Exploration (MCX) that uses…
Get in touch
Something pique your interest? Get in touch if you’d like to learn more about Autodesk Research, our projects, people, and potential collaboration opportunities.
Contact us