Video Diffusion
A diffusion model extended to generate motion across a sequence of frames rather than a single image.
Video diffusion extends the same denoising process behind image diffusion models across a sequence of frames, so the output holds together as motion rather than a slideshow of unrelated stills. It's the technique underneath most current text-to-video and image-to-video tools.
The extra dimension — time — is exactly what makes temporal consistency hard: the model has to keep a subject coherent not just within one frame, but across dozens of them in a row.
See also: Temporal Consistency, Diffusion Model.
Put these ideas into motion.
DownloadFrequently asked questions
Video diffusion extends the image-generating diffusion process across a sequence of frames, denoising a whole short clip at once (or frame by frame with extra machinery) so it holds together as motion rather than a slideshow of unrelated images.
Video diffusion is the technique; text-to-video is the application. Most modern text-to-video and image-to-video tools are built on some form of video diffusion.