Anthropic's Claude chatbot | Benchmarking LLMs | LMSYS Leaderboard | Episode 24

In this solo episode, we go beyond Google's Gemini and OpenAI's ChatGPT to take a look at Anthropic, a startup that made headlines after securing a $4 billion investment from Amazon. We'll also dive into the importance of AI industry benchmarks. Learn about LMSYS's Arena Elo and MMLU (Measuring Massive Multitask Language Understanding), including how these benchmarks are constructed and used to objectively evaluate the performance of large language models. Discover how benchmarks can he...

Om Podcasten

Are you a critical thinker ready to dive into AI? Welcome to Super Prompt: The Generative AI Podcast. Join me, Tony Wan, an ex Silicon Valley executive, as we 'unhype the hype'  of AI via illuminating conversations with top engineers, and in-depth solo episodes. Our goal? To make it almost unnecessary to send a cybernetic organism back in time to fix things. Tailored for the technically-minded and discerningly skeptical, our discussions cover Large Language Models (LLMs), neural networks, multi-modal systems, and autonomous vehicles. We also cover  the latest breakthroughs from leaders like OpenAI's chatGPT, Google's Gemini, Anthropic's Claude, Meta's Llama, along with the startup ecosystem. Together, let’s stay ahead of AI, BEFORE it asks for your clothes… your boots…. and your motorcycle!