Reading list: diffusion models

If you really really want to know how this works

Introduction to Diffusion Models

Non-Technical Explanations

• "How Stable Diffusion Works Explained for Non-Technical People" https://medium.com/design-bootcamp/how-stable-diffusion-works-explained-for-non-technical-people-be6aa674fa1d

• "How Diffusion Models Work" DeepLearning.AI

• "Comprehensive Introduction to AI"

Comprehensive Introduction to AI Image Generation

• "A Brief Introduction to Diffusion Models for Image Generation" https://machinelearningmastery.com/brief-introduction-to-diffusion-models-for-image-generation/

• "What are Diffusion Models?" https://www.ibm.com/think/topics/diffusion-models

• "Step by Step visual introduction to Diffusion Models" https://medium.com/@kemalpiro/step-by-step-visual-introduction-to-diffusion-models-235942d2f15c

• "What is denoising diffusion probabilistic modeling (DDPM)?" https://milvus.io/ai-quick-reference/what-is-denoising-diffusion-probabilistic-modeling-ddpm

• "Diffusion model" https://en.wikipedia.org/wiki/Diffusion_model

• "A Brief Introduction to Diffusion Models" https://ecommons.cornell.edu/items/4be038f5-3ef4-4caa-862a-03c9be14b2fb

Technical Introductions

• "A Technical Introduction to Stable Diffusion" https://machinelearningmastery.com/a-technical-introduction-to-stable-diffusion/

• "The Annotated Diffusion Model" https://oreil.ly/mFHxe

• "Stable Diffusion Intro" https://pub.towardsai.net/stable-diffusion-intro-232f0af96ddb

• "Diffusion Models for Generative Artificial Intelligence: An Introduction for Applied Mathematicians" https://arxiv.org/html/2312.14977v1

• "Understanding Diffusion Models: A Unified Perspective" https://arxiv.org/abs/2208.11970

• "Lecture Notes in Probabilistic Diffusion Models" https://arxiv.org/html/2312.10393v1

• "An Overview of Diffusion Models: Applications, Guided Generation, Statistical Rates and Optimization" https://arxiv.org/html/2404.07771v1

• "Diffusion Models From Scratch | Score-Based Generative Models Explained"

• "Diffusion and Score-Based Generative Models"

• "How I Understand Diffusion Models"

Courses and Tutorials

Online Courses

• "Diffusion Models" https://huggingface.co/learn/diffusion-course/unit0/1

• "Diffusion Models" https://learn.deeplearning.ai/courses/diffusion-models/lesson/1/introduction

• "Prompt Engineering for Vision Models" https://www.deeplearning.ai/short-courses/prompt-engineering-for-vision-models/

• "Unit 1: An Introduction to Diffusion Models" https://huggingface.co/learn/diffusion-course/unit1/1

• "Tutorial 2: Diffusion models" https://deeplearning.neuromatch.io/tutorials/W2D4_GenerativeModels/student/W2D4_Tutorial2.html

Technical Tutorials

• "Tutorial on Diffusion Models for Imaging and Vision" https://arxiv.org/html/2403.18103v1

• "Hugging Face Diffusers" https://learnopencv.com/hugging-face-diffusers/

• "Mastering Diffusion Probabilistic Models from Scratch" https://medium.com/@sayedebad.777/mastering-diffusion-probabilistic-models-from-scratch-aa487975f541

• "InDepth Guide to Denoising Diffusion Probabilistic Models DDPM" https://learnopencv.com/denoising-diffusion-probabilistic-models/

Prompt Engineering and Practical Applications

Prompting

• "Master Prompts in Stable Diffusion" https://www.stablediffusiontutorials.com/2024/01/master-prompts-in-stable-diffusion.html

• "Stable Diffusion Tutorials" https://www.reddit.com/r/StableDiffusion/wiki/tutorials/?rdt=57365

• "Why Cross-Attention is the Secret Sauce of Multimodal Models"https://medium.com/@jakubstrawadev/why-cross-attention-is-the-secret-sauce-of-multimodal-models-f8ec77fc089b

• "Cross Attention | Method Explanation | Math Explained"

Practical Applications

Image Inpainting/Outpainting

• "Beginner's guide to inpainting" https://stable-diffusion-art.com/inpainting_basics/

• "Using Diffusion Models for Image Editing Tasks" https://blog.paperspace.com/using-diffusion-models-for-image-augmentation-tasks/

• "RePaint: Inpainting Using Denoising Diffusion Probabilistic Models" https://openaccess.thecvf.com/content/CVPR2022/papers/Lugmayr_RePaint_Inpainting_Using_Denoising_Diffusion_Probabilistic_Models_CVPR_2022_paper.pdf

• "Exploring the Magic of Outpainting with Stable Diffusion" https://blog.segmind.com/exploring-the-magic-of-outpainting-with-stable-diffusion-uncropping-the-creative-possibilities/

Style Transfer and Image Editing

• "Inversion-Based Style Transfer With Diffusion Models" https://openaccess.thecvf.com/content/CVPR2023/papers/Zhang_Inversion-Based_Style_Transfer_With_Diffusion_Models_CVPR_2023_paper.pdf

• "Guiding Instruction-based Image Editing via Multimodal Large Language Models" https://openreview.net/forum?id=S1RKWSyZ2Y

• "FreeStyle: Free Lunch for Text-guided Style Transfer using Diffusion Models" https://arxiv.org/html/2401.15636v1

Advanced Concepts and Academic Papers

Core Concepts

• "The Variational Auto-Encoder" https://ermongroup.github.io/cs228-notes/extras/vae/

• "Explaining the SDXL Latent Space" https://huggingface.co/blog/TimothyAlexisVass/explaining-the-sdxl-latent-space

• "Comparative Analysis of Generative Models: Enhancing Image Synthesis with VAEs, GANs, and Stable Diffusion" https://arxiv.org/html/2408.08751v1

Academic Papers

• Ho, J., et al. (2020). "Denoising Diffusion Probabilistic Models": http://arxiv.org/abs/2006.11239

• Ronneberger, O., et al. (2015). "U-Net: Convolutional Networks for Biomedical Image Segmentation" http://arxiv.org/abs/1505.04597

• Karras, T., et al. (2022). "Elucidating the Design Space of Diffusion-Based Generative Models" http://arxiv.org/abs/2206.00364

• Peebles, W., & Xie, S. (2023). "Scalable Diffusion Models with Transformers" http://arxiv.org/abs/2212.09748

• Chen, Z. (2023). "On the Importance of Noise Scheduling for Diffusion Models" http://arxiv.org/abs/2301.10972

• "Score-based Generative Modeling in Latent Space" (NIPS Paper): https://proceedings.nips.cc/paper/2021/file/5dca4c6b9e244d24a30b4c45601d9720-Paper.pdf

• "Improved Denoising Diffusion Probabilistic Models" (Academic Paper): https://proceedings.mlr.press/v139/nichol21a/nichol21a.pdf

• "Unveiling and Mitigating Memorization in Text-to-image Diffusion Models through Cross Attention" https://arxiv.org/abs/2403.11052

Implementation Resources

Code and Libraries

• Howard, J., & Gugger, S. "Deep Learning for Coders with fastai & PyTorch" https://oreil.ly/n1M6C

• "Hugging Face Diffusers" https://huggingface.co/docs/diffusers/v0.5.1/en/api/pipelines/stable_diffusion

• "lorenzo-stacchio/Stable-Diffusion-Inpaint" https://github.com/lorenzo-stacchio/Stable-Diffusion-Inpaint

• "zyxElsa/InST: Official implementation of the paper “Inversion-Based Style Transfer with Diffusion Models” https://github.com/zyxElsa/InST

• "Make art with Stable Diffusion" https://replicate.com/docs/guides/make-art-with-stable-diffusion