Language models that write code
In 2020 OpenAI has presented GPT-3 to the world: a transformer model that operated natural language and showcased high level of general skills and emergent meta-learning capabilities. Shortly thereafter, a research project got started that was designed to use the same techniques to train the best model to write computer programs – which concluded with the release of Codex models this year. In my talk, I’ll try to briefly recap how that research journey looked from the inside, how we evaluate programming capabilities of our models and where are their strengths and weaknesses. I’ll conclude my talk with a broad and slightly speculative analysis of future directions for AI programming technology and broader impacts of it on economics and security perspectives.
Bio
Jarosław (Jerry) Tworek – Jerry Tworek is a research scientist at OpenAI. His current focus is on teaching artificial programmers to work hand in hand with humans and increase their productivity. Graduated with MSc in Mathematics from University of Warsaw and spent first five years of his career in the Hedge Fund industry, which finally led him to study deep reinforcement learning. Jerry took part in the OpenAI robotics project „Solving Rubik’s Cube with a Robot Hand” which he later presented at NeurIPS 2019 DeepRL workshop. Currently holds a lead research role in OpenAI program synthesis efforts.