Multimodal LM roundup: Unified IO 2, inputs and outputs, Gemini, LLaVA-RLHF, and RLHF questions
A sampling of recent happenings in the multimodal space. Be sure to expect more this year.This is AI generated audio with Python and 11LabsSource code: https://github.com/natolambert/interconnects-toolsOriginal post: https://www.interconnects.ai/p/multimodal-rlhf00:00 Multimodal LM roundup: Unified IO 2, inputs and outputs, Gemini, LLaVA-RLHF, and RLHF questions02:46 Unified IO 2: Scaling multi-input, multi-output model pretraining07:47 Collecting preference data for images09:31 LLaVA-RLHF: The first experiments in multimodal RLHF fine-tuning13:20 Multimodal RLHF questions, ideas, and resources Get full access to Interconnects at www.interconnects.ai/subscribe