AI, Liability, and Hallucinations in a Changing Tech and Law Environment

Since ChatGPT came on the scene, numerous incidents have surfaced involving attorneys submitting court filings riddled with AI-generated hallucinations—plausible-sounding case citations that purport to support key legal propositions but are, in fact, entirely fictitious. As sanctions against attorneys mount, it seems clear there are a few kinks in the tech. Even AI tools designed specifically for lawyers can be prone to hallucinations. In this episode, we look at the potential and risks of AI-assisted tech in law and policy with two Stanford Law researchers at the forefront of this issue: RegLab Director Professor Daniel Ho and JD/PhD student and computer science researcher Mirac Suzgun. Together with several co-authors, they examine the emerging risks in two recent papers, “Profiling Legal Hallucinations in Large Language Models” (Oxford Journal of Legal Analysis, 2024) and the forthcoming “Hallucination-Free?” in the Journal of Empirical Legal Studies. Ho and Suzgun offer new insights into how legal AI is working, where it’s failing, and what’s at stake.Links:Daniel Ho  >>> Stanford Law pageStanford Institute for Human-Centered Artificial Intelligence (HAI) >>> Stanford University pageRegulation, Evaluation, and Governance Lab (RegLab) >>> Stanford University pageConnect:Episode Transcripts >>> Stanford Legal Podcast WebsiteStanford Legal Podcast >>> LinkedIn PageRich Ford >>>  Twitter/XPam Karlan >>> Stanford Law School PageStanford Law School >>> Twitter/XStanford Lawyer Magazine >>> Twitter/X (00:00:00) Introduction to AI in Legal Education (00:05:01) AI Tools in Legal Research and Writing(00:12:01) Challenges of AI-Generated Content (00:20:0) Reinforcement Learning with Human Feedback(00:30:01) Audience Q&A

Om Podcasten

Law touches most aspects of life. Here to help make sense of it is the Stanford Legal podcast, where we look at the cases, questions, conflicts, and legal stories that affect us all every day. Stanford Legal launched in 2017 as a radio show on Sirius XM. We’re now a standalone podcast and we’re back after taking some time away, so don’t forget to subscribe or follow this feed. That way you’ll have access to new episodes as soon as they’re available. We know that the law can be complicated. In past episodes we discussed a broad range of topics from the legal rights of someone in a conservatorship like Britney Spears to the Supreme Court’s abortion decision to how American law firms had to untangle their Russian businesses after the invasion of Ukraine. Past episodes are still available in our back catalog of episodes. In future shows, we’ll bring on experts to help make sense of things like machine learning and developments in the regulation of artificial intelligence, how the states draw voting maps, and ways that the Supreme Court’s affirmative action ruling will change college admissions. Our co-hosts know a bit about these topics because it’s their life’s work. Pam Karlan studies and teaches what is known as the “law of democracy,”—the law that regulates voting, elections, and the political process. She served as a commissioner on the California Fair Political Practices Commission, an assistant counsel and cooperating attorney for the NAACP Legal Defense Fund, and (twice) as a Deputy Assistant Attorney General in the Civil Rights Division of the U.S. Department of Justice. She also co-directs Stanford’s Supreme Court Litigation Clinic, which represents real clients before the highest court in the country, working on important cases including representing Edith Windsor in the landmark marriage equality win and David Riley in a case where the Supreme Court held that the police generally can’t search digital information on a cell phone seized from an individual who has been arrested unless they first get a warrant. She has argued before the Court nine times. And Rich Ford’s teaching and writing looks at the relationship between law and equality, cities and urban development, popular culture and everyday life. He teaches local government law, employment discrimination, and the often-misunderstood critical race theory. He studied with and advised governments around the world on questions of equality law, lectured at places like the Sorbonne in Paris on the relationship of law and popular culture, served as a commissioner for the San Francisco Housing Commission, and worked with cities on how to manage neighborhood change and volatile real estate markets. He writes about law and popular culture for lawyers, academics, and popular audiences. His latest book is Dress Codes: How the Laws of Fashion Made History, a legal history of the rules and laws that influence what we wear. The law is personal for all of us—and pivotal. The landmark civil rights laws of the 1960s have made discrimination illegal but the consequences of the Jim Crow laws imposed after the civil war are still with us, reflected in racially segregated schools and neighborhoods and racial imbalances in our prisons and conflict between minority communities and police. Unequal gender roles and stereotypes still keep women from achieving equality in professional status and income. Laws barring gay people from marrying meant that millions lived lives of secrecy and shame. New technologies present new legal questions: should AI decide who gets hired or how long convicted criminals go to prison? What can we do about social media’s influence on our elections? Can Chat GPT get copyright in a novel? Law matters. We hope you’ll listen to new episodes that will drop on Thursdays every two weeks. To learn more, go to https://law.stanford.edu/stanford-legal-podcast/.