# Sitemap

A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.

## Pages

## Posts

## blog

## Beat Atari with Deep Reinforcement Learning! (Part 0: Intro to RL)

** Published:**

The first post in my tutorial series on deep reinforcement learning, which introduces RL in general.

View here

## Beat Atari with Deep Reinforcement Learning! (Part 1: DQN)

** Published:**

A tutorial on deep reinforcement learning, reproducing ‘Playing Atari with Deep Reinforcement Learning’, which introduces the notion of a Deep Q-Network.

View here

## Beat Atari with Deep Reinforcement Learning! (Part 2: DQN improvements)

** Published:**

A tutorial on deep reinforcement learning, reproducing ‘Human-level control through deep reinforcement learning’

View here

## Paper repro: “Learning to Learn by Gradient Descent by Gradient Descent”

** Published:**

This is a reproduction of the metalearning paper “Learning to Learn by Gradient Descent by Gradient Descent”

View here

## Investigating Focal and Dice Loss for the Kaggle 2018 Data Science Bowl

** Published:**

This post details my experiments and implementations with three important loss functions for the Kaggle 2018 data science bowl, and compares their effects on a simplified implementation of U-Net.

View here

## Deep Learning Book Notes, Chapter 1

** Published:**

View here

## Paper repro: “Self-Normalizing Neural Networks”

** Published:**

This post details my experience reproducing “Self-Normalizing Neural Networks” as part of the Nurture.AI NIPS Challenge 2017, as well as my experience participating in the challenge.

View here

## Deep Learning Book Notes, Chapter 2: Linear Algebra for Deep Learning

** Published:**

These are my notes for chapter 2 of the Deep Learning book. They can also serve as a quick intro to linear algebra for deep learning.

View here

## Paper Repro: Deep Neuroevolution

** Published:**

In this post, I reproduce the recent Uber paper “Deep Neuroevolution: Genetic Algorithms are a Competitive Alternative for Training Deep Neural Networks for Reinforcement Learning”

View here

## Paper repro: Deep Metalearning using “MAML” and “Reptile”

** Published:**

In this post I reproduce two papers in the field of metalearning: MAML and the similar Reptile.

View here

## Deep Learning Book Notes, Chapter 3 (part 1): Introduction to Probability

** Published:**

These are the first part of my notes for chapter 3 of the Deep Learning book. They can also serve as a quick intro to probability.

View here

## An Intuitive Explanation of Policy Gradient

** Published:**

In this post, I attempt to provide a clear explanation of the intuition behind policy gradient algorithms.

View here

## Quantum Circuit Decomposition

** Published:**

An introduction to quantum circuit decomposition, a technique in quantum computing. Master’s project.

View here

## publications

## Go-explore: a new approach for hard-exploration problems

arXiv, 2019

Introduces Go-Explore, an exploration algorithm capable of solving the grand-challenge hard-exploration Atari games Montezuma′s Revenge and Pitfall.

View here

## Open Questions in Creating Safe Open-ended AI: Tensions Between Control and Creativity

ALIFE, 2020

Open-ended search algorithms are relatively unstudied in the field of AI safety and yet are starting to show increasing promise as a path to producing advanced AI. This paper discusses the safety implications of open-ended search algorithms in AI.

View here

## Estimating Q(s,s’) with Deep Deterministic Dynamics Gradients

ICML, 2020

In this paper, we introduce a novel form of value function, Q(s, s′). We highlight the benefits of this approach in terms of value function transfer, learning within redundant action spaces, and learning off-policy from state observations generated by sub-optimal or completely random policies.

View here

## Exploration Based Language Learning for Text-Based Games

IJCAI, 2020

This work presents an exploration and imitation-learning-based agent capable of state-of-the-art performance in playing text-based computer games using the Go-Explore exploration algorithm.

View here

## First return, then explore

Nature, 2021

The definite version of the Go-Explore algorithm. On top of the results from the original pre-print, it introduces a dynamic representation that supports all Atari games, a variant in which the exploration phase can be performed in stochastic environments, and demonstrates Go-Explore working in a robotics environment.

View here

## Multi-task curriculum learning in a complex, visual, hard-exploration domain: Minecraft

arXiv, 2021

An important challenge in reinforcement learning is training agents that can solve a wide variety of tasks. If tasks depend on each other (e.g. needing to learn to walk before learning to run), curriculum learning can speed up learning by focusing on the next best task to learn. We explore curriculum learning in a complex, visual domain with many hard exploration challenges: Minecraft.

View here

## Reinforcement Learning Under Moral Uncertainty

ICML 2021, 2021

An ambitious goal for artificial intelligence is to create agents that behave ethically. Unfortunately, there is widespread disagreement about which ethical theory an agent should follow. This paper translates philosophical work on moral uncertainty into an RL algorithm and investigates how moral uncertainty can avoid extreme behavior compared to single theories.

View here

## Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos

arXiv, 2022

We show that with a small amount of labeled data we can train an inverse dynamics model accurate enough to label a huge unlabeled source of online data – here, online videos of people playing Minecraft – from which we can then train a general behavioral prior.

View here