Recurrent Off-Policy Deep Reinforcement Learning Doesn't Have to be Slow
By: Tyler Clark, Christine Evers, Jonathon Hare
Recurrent off-policy deep reinforcement learning models achieve state-of-the-art performance but are often sidelined due to their high computational demands. In response, we introduce RISE (Recurrent Integration via Simplified Encodings), a novel approach that can leverage recurrent networks in any image-based off-policy RL setting without significant computational overheads via using both learnable and non-learnable encoder layers. When integrating RISE into leading non-recurrent off-policy RL algorithms, we observe a 35.6% human-normalized interquartile mean (IQM) performance improvement across the Atari benchmark. We analyze various implementation strategies to highlight the versatility and potential of our proposed framework.
Similar Papers
PipelineRL: Faster On-policy Reinforcement Learning for Long Sequence Generatio
Machine Learning (CS)
Trains AI faster and smarter using new methods.
Going Beyond Expert Performance via Deep Implicit Imitation Reinforcement Learning
Machine Learning (CS)
Teaches robots to learn from watching, not just doing.
Periodic Asynchrony: An Effective Method for Accelerating On-Policy Reinforcement Learning
Machine Learning (CS)
Makes computer learning much faster and cheaper.