Sitemap
A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.
Pages
Posts
blog
Beat Atari with Deep Reinforcement Learning! (Part 0: Intro to RL)
Published:
The first post in my tutorial series on deep reinforcement learning, which introduces RL in general.
View here
Beat Atari with Deep Reinforcement Learning! (Part 1: DQN)
Published:
A tutorial on deep reinforcement learning, reproducing ‘Playing Atari with Deep Reinforcement Learning’, which introduces the notion of a Deep Q-Network.
View here
Beat Atari with Deep Reinforcement Learning! (Part 2: DQN improvements)
Published:
A tutorial on deep reinforcement learning, reproducing ‘Human-level control through deep reinforcement learning’
View here
Paper repro: “Learning to Learn by Gradient Descent by Gradient Descent”
Published:
This is a reproduction of the metalearning paper “Learning to Learn by Gradient Descent by Gradient Descent”
View here
Investigating Focal and Dice Loss for the Kaggle 2018 Data Science Bowl
Published:
This post details my experiments and implementations with three important loss functions for the Kaggle 2018 data science bowl, and compares their effects on a simplified implementation of U-Net.
View here
Deep Learning Book Notes, Chapter 1
Published:
View here
Paper repro: “Self-Normalizing Neural Networks”
Published:
This post details my experience reproducing “Self-Normalizing Neural Networks” as part of the Nurture.AI NIPS Challenge 2017, as well as my experience participating in the challenge.
View here
Deep Learning Book Notes, Chapter 2: Linear Algebra for Deep Learning
Published:
These are my notes for chapter 2 of the Deep Learning book. They can also serve as a quick intro to linear algebra for deep learning.
View here
Paper Repro: Deep Neuroevolution
Published:
In this post, I reproduce the recent Uber paper “Deep Neuroevolution: Genetic Algorithms are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning”
View here
Paper repro: Deep Metalearning using “MAML” and “Reptile”
Published:
In this post I reproduce two papers in the field of metalearning: MAML and the similar Reptile.
View here
Deep Learning Book Notes, Chapter 3 (part 1): Introduction to Probability
Published:
These are the first part of my notes for chapter 3 of the Deep Learning book. They can also serve as a quick intro to probability.
View here
An Intuitive Explanation of Policy Gradient
Published:
In this post, I attempt to provide a clear explanation of the intuition behind policy gradient algorithms.
View here
Quantum Circuit Decomposition
Published:
An introduction to quantum circuit decomposition, a technique in quantum computing. Master’s project.
View here
publications
Go-explore: a new approach for hard-exploration problems
arXiv, 2019
Introduces Go-Explore, an exploration algorithm capable of solving the grand-challenge hard-exploration Atari games Montezuma′s Revenge and Pitfall.
View here
Open Questions in Creating Safe Open-ended AI: Tensions Between Control and Creativity
ALIFE, 2020
Open-ended search algorithms are relatively unstudied in the field of AI safety and yet are starting to show increasing promise as a path to producing advanced AI. This paper discusses the safety implications of open-ended search algorithms in AI.
View here
Estimating Q(s,s’) with Deep Deterministic Dynamics Gradients
ICML, 2020
In this paper, we introduce a novel form of value function, Q(s, s′). We highlight the benefits of this approach in terms of value function transfer, learning within redundant action spaces, and learning off-policy from state observations generated by sub-optimal or completely random policies.
View here
Exploration Based Language Learning for Text-Based Games
IJCAI, 2020
This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games using the Go-Explore exploration algorithm.
View here
First return, then explore
Nature, 2021
The definite version of the Go-Explore algorithm. On top of the results from the original pre-print, it introduces a dynamic representation that supports all Atari games, a variant in which the exploration phase can be performed in stochastic environments, and demonstrates Go-Explore working in a robotics environment.
View here
Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft
arXiv, 2021
An important challenge in reinforcement learning is training agents that can solve a wide variety of tasks. If tasks depend on each other (e.g. needing to learn to walk before learning to run), curriculum learning can speed up learning by focusing on the next best task to learn. We explore curriculum learning in a complex, visual domain with many hard exploration challenges: Minecraft.
View here
Reinforcement Learning Under Moral Uncertainty
ICML 2021, 2021
An ambitious goal for artificial intelligence is to create agents that behave ethically. Unfortunately, there is widespread disagreement about which ethical theory an agent should follow. This paper translates philosophical work on moral uncertainty into an RL algorithm and investigates how moral uncertainty can avoid extreme behavior compared to single theories.
View here
Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos
arXiv, 2022
We show that with a small amount of labeled data we can train an inverse dynamics model accurate enough to label a huge unlabeled source of online data – here, online videos of people playing Minecraft – from which we can then train a general behavioral prior.
View here