Evaluations: Trust, performance, and price (bonus, announcing RewardBench)
Evaluation is not only getting harder with modern LLMs, it's getting harder because it means something different.This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen.Source code: https://github.com/natolambert/interconnects-toolsOriginal post: https://www.interconnects.ai/p/evaluations-trust-performance-and-price00:00 Evaluations: Trust, performance, and price (bonus, announcing RewardBench)03:14 The rising price of evaluation05:40 Announcing RewardBench: The First reward model evaluation tool08:37 Updates to RLHF evaluation toolsYouTube code intro: https://youtu.be/CAaHAfCqrBAFigure 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/evals/img_026.pngFigure 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/evals/img_030.pngFigure 3: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/evals/img_034.pngFigure 4: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/evals/img_040.png Get full access to Interconnects at www.interconnects.ai/subscribe