The Diffusion Model's Unsung Sidekick: A Science of Solving (Almost) Any Problem using Probability
Audience: high-schoolundergraduategraduate
Tags: generative-aiimage-generationartificial-intelligencedeep-learning
Diffusion models, which are the best class of AI image generators known today, are typically portrayed as models that learn to denoise a corrupted image. This way, they can generate new images by gradually removing noise from a sample of pure noise. This video explains diffusion models through an alternative perspective that is more intuitive and practical; in particular, I show how this perspective results in some interesting realizations, that most other resources seem to miss: 1. Image generation is the same thing as rolling a dice; they can be reduced to the same basic computational recipe 2. That diffusion models elegantly extend the success of gradient descent, the technique used to train all neural networks today, from train time to test time 3. How diffusion models separate the "creative" and "logical" capabilities of image generation into two different actors/players 4. And how this might lead us to a general recipe of solving (almost) any hard problem Given their broad application also to video generation, robotics, drug discovery, music generation, and more, a better understanding of diffusion models is needed, which this video aims to address. The intended audience is broad: 1. AI researchers and engineers looking to get a better understanding of a key technological development in generative modeling 2. People interested in computer graphics, image generation, AI art, and artificial intelligence more broadly 3. General math enthusiasts
Analytics
Comments
It took a long time to introduce itself and get through things that could have been simplified. It didn’t really tell me anything I didn’t know about AI image generation (I don’t know who ever conceptually thought of it as denoising a corrupted image in the first place), and while Langevin sampling doubtless contained information I don’t know, it clearly is a deep subject that would take a lot of concentration to understand, and which there wasn’t a clear motivation for - the example given was of a dice roll which everyone clearly already understands without the need to go into such complex math to figure it out.
This video is very good at making it clear what exactly a random error term is doing in an LLM. My main gripe with this video is the pacing is a little slow and the visuals started to get repetitive. Great video, and I would recommend this to CS students who are interested in what LLMs are doing under the hood
I think this is a great video for anyone studying machine learning! As someone who has worked with diffusion models I especially found your explanation of noise escaping local optima insightful. I’ve never really fully understood the math behind diffusion and was a bit lost on the technical details and the part about Langevin sampling but I can definitely seeing this being useful to others far more experienced then me.
I really liked the presentation of the video, the animation was helpful and the voice was very clear. The beginning had a hook that made me interested to learn more about the topic. I personally would have made the video a bit shorter if possible. After about the 15-20 minute mark when we have explored the idea of images being samples from a probability space I lost the motivation of “what is the missing piece the author refers to?”. While there is still plenty of worthwhile information in the rest of the video a beginner might abandon the video after a while.
Very clear motivation and explanation! Also nice pacing. I love all the scaffolding techniques you used here, like prompting questions, analogies, visualizations, etc…You made a dry topic very approachable.
I learned a lot about diffusion sampling via Langevin by watching your video. I know I’ll need to rewatch a few more times over to understand it more. I appreciated how you expounded upon how vast the current landscape is for image gen in AI, and simple and somtimes unseen techniques that are fitting because it’s how deep neural networks have been constructed. Your Ranking score was averaged based on these individual scores: Motivation: 9 Clarity: 9 Novelty: 5 Memorability: 9
This is very valuable in-depth knowledge!
Very good explanation of Langevin dynamics. However, I am not entirely convinced about the main takeaway and the interpretation of the noiseless sampling experiment. In particular, there are deterministic generative models based on repeated denoising, such as Basal et al., Cold Diffusion: Inverting Arbitrary Image Transforms Without Noise. These are not state-of-the-art, but their mere existence suggests to me that adding noise during the process is crucial only if the neural network was trained under such conditions. In any case, the editorializing about the importance of Langevin dynamics was redundant after a while, it could be cut a bit.
A minor point: in some sections, the same irrelevant static image was shown for too long.
The SoME rules state that “Each entry should be self-contained, not part of a series, playlist, or larger project: something one can dive into without needing extra context.” This video is definitely part of a series, but it’s quite self-contained, so I don’t think it’s really an issue.