A16z Podcast

General Partner Anjney Midha explores the cutting-edge world of text-to-video AI with AI researchers Andreas Blattmann and Robin Rombach. Released in November, Stable Video Diffusion is their latest open-source generative video model, overcoming challenges in size and dynamic representation.In this episode Robin and Andreas share why translating text to video is complex, the key role of datasets, current applications, and the future of video editing.Topics Covered: 00:00 - Text to Video: The Next Leap in AI Generation02:41 - The Stable Diffusion backstory04:25 - Diffusion vs autoregressive models06:09 - The benefits of single step sampling09:15 - Why generative video?11:19 - Understanding physics through AI video12:20 - The challenge of creating generative video15:36 - Data set selection and training17:50 - Structural consistency and 3D objects19:50 - Incorporating LoRAs21:24 - How should creators think about these tools?23:46 - Open challenges in video generation 25:42 - Infrastructure challenges and future