A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.




Paper Repro: Deep Neuroevolution


In this post, I reproduce the recent Uber paper “Deep Neuroevolution: Genetic Algorithms are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning”

View here


Estimating Q(s,s’) with Deep Deterministic Dynamics Gradients

Ashley D. Edwards, Himanshu Sahni, Rosanne Liu, Jane Hung, Ankit Jain, Rui Wang, Adrien Ecoffet, Thomas Miconi, Charles Isbell, Jason Yosinski

ICML, 2020

In this paper, we introduce a novel form of value function, Q(s, s′). We highlight the benefits of this approach in terms of value function transfer, learning within redundant action spaces, and learning off-policy from state observations generated by sub-optimal or completely random policies.

View here

Exploration Based Language Learning for Text-Based Games

Andrea Madotto, Mahdi Namazifar, Joost Huizinga, Piero Molino, Adrien Ecoffet, Huaixiu Zheng, Alexandros Papangelis, Dian Yu, Chandra Khatri, Gokhan Tur

IJCAI, 2020

This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games using the Go-Explore exploration algorithm.

View here

First return, then explore

Adrien Ecoffet, Joost Huizinga, Joel Lehman, Kenneth O. Stanley, Jeff Clune

Nature, 2021

The definite version of the Go-Explore algorithm. On top of the results from the original pre-print, it introduces a dynamic representation that supports all Atari games, a variant in which the exploration phase can be performed in stochastic environments, and demonstrates Go-Explore working in a robotics environment.

View here

Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft

Ingmar Kanitscheider, Joost Huizinga, David Farhi, William Hebgen Guss, Brandon Houghton, Raul Sampedro, Peter Zhokhov, Bowen Baker, Adrien Ecoffet, Jie Tang, Oleg Klimov, Jeff Clune

arXiv, 2021

An important challenge in reinforcement learning is training agents that can solve a wide variety of tasks. If tasks depend on each other (e.g. needing to learn to walk before learning to run), curriculum learning can speed up learning by focusing on the next best task to learn. We explore curriculum learning in a complex, visual domain with many hard exploration challenges: Minecraft.

View here

Reinforcement Learning Under Moral Uncertainty

Adrien Ecoffet, Joel Lehman

ICML 2021, 2021

An ambitious goal for artificial intelligence is to create agents that behave ethically. Unfortunately, there is widespread disagreement about which ethical theory an agent should follow. This paper translates philosophical work on moral uncertainty into an RL algorithm and investigates how moral uncertainty can avoid extreme behavior compared to single theories.

View here

Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos

Bowen Baker, Ilge Akkaya, Peter Zhokhov, Joost Huizinga, Jie Tang, Adrien Ecoffet, Brandon Houghton, Raul Sampedro, Jeff Clune

arXiv, 2022

We show that with a small amount of labeled data we can train an inverse dynamics model accurate enough to label a huge unlabeled source of online data – here, online videos of people playing Minecraft – from which we can then train a general behavioral prior.

View here