I am currently a researcher at Anthropic, where I’m working on aligning large language models; some of my interests include scalable oversight and developing better written specifications of model behavior (like OpenAI’s Model Spec, Constitutional AI).

Motion Planning with Sequential Convex Optimization and Convex Collision Checking John Schulman, Yan Duan, Jonathan Ho, Alex Lee, Ibrahim Awwal, Henry Bradlow, Jia Pan, Sachin Patil, Ken Goldberg, Pieter Abbeel.

Code GitHub profile Highlighted projects developed by my collaborators and me: Procgen Benchmark (2019): GitHub / blog post. Gym Retro (2018): GitHub / blog post on dataset / contest OpenAI Baselines (2016): GitHub / original post (DQN) / ACKTR + A2C / PPO OpenAI Gym (2016): homepage / GitHub / blog post / article on NVIDIA blog Computation Graph Toolkit (2015): GitHub / docs. Computation ...

2024 Talk about OpenAI Model Spec at Scale conference 2023 ICML talk on proxy objectives 2023 Berkeley talk on truthfulness Older slides and video presentations:

O P T I M I Z I N G E X P E C TAT I O N S : F R O M D E E P R E I N F O R C E M E N T L E A R N I N G T O S T O C H A S T I C C O M P U TAT I O N G R A P H S john schulman

Jan 24, 2020 · I originally wrote this guide in back in December 2017 for the OpenAI Fellows program In this essay, I provide some advice to up-and-coming researchers in machine learning (ML), based on my experience doing research and advising others. The advice covers how to choose problems and organize your time.

Jan 24, 2020 · Sending Samples Without Bits-Back (2020/03/08) Approximating KL Divergence (2020/03/07) An Opinionated Guide to ML Research (2020/01/24)

The Nuts and Bolts of Deep RL Research John Schulman

• Future: more research and contests Thanks John Schulman, OpenAI thanks to many colleagues from OpenAI and Berkeley who collaborated on the work I presented

[2018] MIT Technology Review's 35 Innovators Under 35. [2016] C.V. Ramamoorthy Distinguished Research Award [2013] Best Vision Paper, awarded at ICRA 2013