#42 - Anthony Aguirre and Malo Bourgon - Why Superintelligent AI Should Not Be Built
Humanity is in a race to build superintelligent AI gods. What happens if we create beings smarter than us—and can no longer steer them?In this special Win-Win IRL episode, Liv Boeree sits down with Anthony Aguirre (CEO of Future of Life Institute) and Malo Bourgon (CEO of MIRI) for a no-holds-barred conversation on finding alignment, avoiding power concentration and surveillance states, and the geopolitical implications of the race to AGI.Questions include: Why can’t the market be trusted to self-regulate AI? Why does power concentration in AI pose such a massive threat to democracy? And what happens when even well-meaning developers lose control of the systems they’ve built? We also explore win-win solutions—from compute governance and open-source safety measures, to collaborative international treaties and new models for responsible innovation. Also keep an eye out So buckle up for a provocative episode that challenges assumptions and asks what it will really take to keep the future human.Chapters:0:00 - Intro4:08 - Different Categories of Risks10:51 - Why Can't the Market Manage AI Risks Sufficiently?15:33 - The Dangers of Power Concentration22:52 - What Solution Space Exists?32:55 - Should Compute be Regulated?38:38 - Taylor Capito - Are we at an Evolutionary Nexus42:07 - JD Ross - Decentralization vs Centralization46:46 - Isabelle Boemeke - Nuclear Power & AI51:06 - Peter Wang - Wrong Stories Attract the Wrong People57:39 - Aurora Quinn-Elmore & Eliezer Yudkowsky - Coherent Extrapolated Volition1:05:59 - Can Win-Win be embedded into AI? Links: ♾️ Keep the Future Human Paper: https://keepthefuturehuman.ai ♾️ Future of Life Institute: https://futureoflife.org ♾️ MIRI: https://intelligence.org ♾️ Superintelligence Book by Nick Bostrom - https://www.amazon.com/Superintelligence-Dangers-Strategies-Nick-Bostrom/dp/0198739834Credits:♾️ Hosted by Liv Boeree♾️ Produced by Luca de VicoThe Win-Win Podcast:Poker champion Liv Boeree takes to the interview chair to tease apart the complexities of one of the most fundamental parts of human nature: competition. Liv is joined by top philosophers, gamers, artists, technologists, CEOs, scientists, athletes and more to understand how competition manifests in their world, and how to change seemingly win-lose games into Win-Wins.Podcast links:♾️ Youtube: https://www.youtube.com/playlist?list=PLWgq0OZMtwtOIyMsVM_vksqdfWcM-b68S♾️ Spotify: https://open.spotify.com/show/03bGVUaFZmJUmEvSHNDPdI?si=64379cc23696454f ♾️ Apple Podcasts: https://podcasts.apple.com/us/podcast/win-win-with-liv-boeree/id1724791350 ♾️ Pocketcast: https://play.pocketcasts.com/podcasts/7f708340-d17c-013b-f46e-0acc26574db2 #AGI #podcast #aialignment