[Article Voiceover] Reverse engineering OpenAI's o1
What productionizing test-time compute shows us about the future of AI. Exploration has landed in language model training.This is AI generated audio with Python and 11Labs.Source code: https://github.com/natolambert/interconnects-toolsOriginal post: https://www.interconnects.ai/p/reverse-engineering-openai-o100:00 Reverse engineering OpenAI's o101:52 From Q-star to Strawberry to o105:13 Training o1 with reinforcement learning09:24 What is o1 doing when given a prompt?11:49 Questions to consider to understand o1's structure11:56 1. How does an RL-trained language model act?12:38 2. Is it an online / test-time search?14:20 3. Is it one model at inference?15:29 Open-source o1, the future of o1, and the future of AIFig 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_014.pngFig 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_016.pngFig 3: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_018.pngFig 4: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_020.pngFig 5: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_024.pngFig 6: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_026.pngFig 7: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_034.pngFig 8: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/o1/img_048.png Get full access to Interconnects at www.interconnects.ai/subscribe