Improving AI Evaluation with Data-Driven Approach | Yi Zhang, Relari
When getting into AI application development, it's important not to overlook the evaluation process. Many companies concentrate on building and training AI models, but struggle with systematically assessing the performance of these models. This emphasizes the necessity of using robust evaluation tools to accurately gauge the effectiveness of AI systems, stressing the importance of quantitatively measuring AI performance to identify strengths and weaknesses. Today, we sit with Yi Zhang, co-founder of Relari AI, to learn about their journey from a mix of finance and technology backgrounds into the field of artificial intelligence. We discussed Yi Zhang's early career in computer science, a role in investment banking, and then a transition to AI with a significant position at Pony AI. Our conversation focused on the evolution of the AI industry, specifically the transformative impact of LLMs (Large Language Models) like ChatGPT. We also examined Relari AI's mission to tackle the challenges of operationalizing AI applications, emphasizing their unique approach using synthetic data for rigorous evaluation. Additionally, we covered the importance of cross-industry AI applications, the critical need for reliability in AI outputs, and the company's ongoing recruitment efforts to grow their innovative team. If your company is looking to scale its AI initiatives, head over to Tesoro AI (www.tesoroai.com). We are experts in AI strategy, staff augmentation, and AI product development. Founder Bio: Yi is the co-founder & CEO of Relari AI. Relari is the testing and simulation platform for Generative AI systems. AI teams use Relari to generate large scale synthetic data to systematically test and improve their LLM-powered applications. Relari is backed by top investors like Y Combinator, Soma Capital and General Catalyst. Prior to founding Relari, Yi was a product leader at two AI unicorns, Pony.ai - an autonomous vehicle startup and Dexterity.ai - an AI robotics startup. Yi holds a Bachelor's Degree in Computer Science from Northwestern University and an MBA from Harvard Business School. Time Stamps: 01:40 Yi’s background and journey into AI 05:41 Founding Relari: The genesis and vision 07:46 Challenges in moving LLMs from demos to production 11:57 Relari's approach to AI evaluation and synthetic data 19:19 Synthetic data and its importance 21:36 Using synthetic data to improve AI chatbot accuracy 26:38 Optimizing AI systems through evaluation and improvement strategies 32:08 The evolution of AI adoption in enterprises 35:33 Building the Relari team 40:17 Closing remarks and how to get in contact with the Relari team Resources Company website: https://www.relari.ai/ LinkedIn: https://www.linkedin.com/company/relari/ Twitter: https://x.com/RelariAI