Deep Generative Diffusion Models

Registration is closed (you can still apply for the waiting list).

For any issues regarding the seminar, please e-mail Gerrit Großmann and have [DeepDiffusion] in the subject line.

Organizers

Prof. Dr. Verena Wolf
Gerrit Großmann, MSc
Dr. Michael Backenköhler

Organization

  • The kick-off meeting takes place on Monday, January 9, 14:15 in room 328 (E 1.3). Send three topics preferences (unordered) by Friday 23:59 to Gerrit.
  • This seminar takes place on the March, 6 and 7, 2023 in room 1.06 (E1.1) (we start at 9:30!).
  • The seminar will take place in-person.
  • The seminar language is English.
  • The seminar earns you 7 ECTS.
  • The seminar is eligible for bachelor/master/graduate students of computer science and related courses.
  • Depending on the study regulations, (un)registration in HISPOS/LSF is due by January 30, 2023.

Topic Overview

Diffusion models are a powerful machine learning tool and a thriving area of research. They enable the generation of new samples with extraordinary and unprecedented quality. This seminar explores the theoretical foundations rooted in the theory of stochastic processes and explores their potential in various domains.

Requirements

General background knowledge in deep learning is recommended.

Grade

To pass the seminar, you have to:

  • attend both sessions;
  • give a presentation (with a passing grade);
  • give an ungraded mock presentation;
  • participate in discussions;
  • submit reports (with a passing grade);
  • read the papers and watch the videos on the Compulsory Reading List;
  • pass the practical project.

The final grade consists of: your presentation (50%), reports (35%), and participation during the seminar discussions (15%). The practical project can be failed or passed. Extronary submission will earn you a bonus on your final grade.

Presentation (50%)

Identify the key ideas and concepts and give a self-consistent presentation explaining these concepts to your fellow students.

The presentation should be about 20 minutes long (no longer than 25 minutes!).

Here are some suggestions for a good presentation (we will use this as a basis for grading the presentations):

  • The goal is to tell a (self-consistent and entertaining) story - not to convince us that you understand the paper.
  • Put time and effort into creating visualizations and preparing (running) examples.
  • Prioritize concreteness, simplicity, and clarity.
  • Focus on intuition, high-level understanding, and contextualization, not on technical details.
  • Don’t overcrowd your slides. Try to avoid full sentences and be cautious with bullet points.
  • Explore and include supplementary material where it seems useful (literature, youtube, GitHub, medium articles, OpenReview, etc.).
  • Use equations only when necessary; use color-coded equations to improve their readability (example).
  • Be critical of the authors’ claims, don’t fall for overselling.
  • Use slide numbers.

Send the first draft of your slides as a .pdf file to your supervisor 7 days before the presentation.

You will also be required to give an ungraded mock presentation in your group (see topic list) by March 5. You can meet in our seminar room or online (Zoom/Teams).  For more information, please refer to the email (Seminar Information VI).

Reports (35%)

Every participant has to write three short reports (two to four pages each). You can choose freely from the twelve topics (you may also choose your own topic). The report should contain a short summary of (what you consider to be) the main contribution or most intriguing idea of the paper. Otherwise, you can freely express your own thoughts on the topic. For instance: What did you like/dislike about the paper (both methodically and didactically)? What are connections to other seminar papers? Can you suggest improvements? What do you think is missing?

The report deadline is 14 days after the last presentation (March 21, 23:59).

You can use the Neurips (activate the preprint flag) or any other reasonable format (don’t write an abstract). Please use a spell+grammar checker like languagetool or grammarly before submitting.

Please email the report to your advisor.

Discussions (15%)

Read each presented paper in advance (of course, we do not expect you to understand them entirely) and actively participate in the discussion.

Practical Project (fail/pass/bonus)

You can work on the project alone or in groups of two.  The deadline is 28 days after the last presentation (April 11, 23:59).

You may find the specifications here.

Topics

Compulsory Reading List


Monday (March 6, 2023)

Student (Group, Superviser) Topic Additional Material
Maryam Meghdadi Esfahani (1, GG) Generative Modeling by Estimating Gradients of the Data Distribution (2019) [1,2,3]
Mohammad Sadegh Akhondzadeh (2, MB) Permutation Invariant Graph Generation via Score-Based Generative Modeling (2020) [1,2,7,8]
Jorge Augusto Calvimontes Robles (3, GG) Denoising Diffusion Probabilistic Models (2020) [4,5,6]
Devikalyan Das (1 , MB) Score-Based Generative Modeling through Stochastic Differential Equations (2021) [1,2,7,F]
Lisa Dargasz (3, GG) High-resolution image synthesis with latent diffusion models (2022) [10,11,12,F]
Simone Antonelli (4, MB) Equivariant diffusion for molecule generation in 3D (2022) [17,26,27]


Tuesday (March 7, 2023)

Student (Group, Superviser) Topic Additional Material
Advait Ganesh Athreya (2, GG) DiGress: Discrete Denoising diffusion for graph generation (2022) [7,13,14]
Simon Graf (2, MB) Structure-based Drug Design with Equivariant Diffusion Models (2022) [15,16,17,18]
Tim Valentin Kruse (4, GG) 3D Shape Generation and Completion through Point-Voxel Diffusion (2022) [19,20]
Vikram Singh (4, MB) Diffusion probabilistic modeling of protein backbones in 3D for the motif-scaffolding problem (2022) [17,21,22,23]
Shreyash Arya (3, GG) Make-A-Video: Text-to-Video Generation without Text-Video Data (2022) [10,12,24,25]
Hamayoon Behmanush (1 , MB) Continuous diffusion for categorical data (2022) [2,27,28]


The group is only relevant for the mock presentation. Your supervisors are either Gerrit Großmann (GG) or Michael Backenköhler (MB).

Supplementary Material

A non-exhaustive list of supplemental materials that we think might be helpful to you is provided here. We encourage you to seek and consult additional resources.

  1. A Pedagogical Introduction to Score Models
  2. Generative Modeling by Estimating Gradients of the Data Distribution
  3. Improved Techniques for Training Score-Based Generative Models
  4. Improved Denoising Diffusion Probabilistic Models
  5. Diffusion Models Beat GANs on Image Synthesis
  6. Ultimate Guide to Diffusion Models - ML Coding Series
  7. A Gentle Introduction to Graph Neural Networks
  8. Score-based Generative Modeling of Graphs via the System of Stochastic Differential Equations
  9. Monte Carlo Methods (Chapter 4, Random Process Generation)
  10. U-Net: Convolutional Networks for Biomedical Image Segmentation
  11. Stable Diffusion - What, Why, How?
  12. Learning Transferable Visual Models From Natural Language Supervision
  13. DiGress: Discrete Denoising Diffusion for Graph Generation - Clément Vignac
  14. A Generalization of Transformer Networks to Graphs
  15. T015 - Protein ligand docking
  16. Exploiting Symmetries in Inference and Learning
  17. An introduction to E(3)-invariant graph neural networks (this is WIP)
  18. Pocket2Mol: Efficient Molecular Sampling Based on 3D Protein Pockets
  19. Project Website
  20. Point-Voxel CNN for Efficient 3D Deep Learning
  21. Protein Structure and Folding
  22. Diffusion probabilistic modelling of protein backbones in 3D - Jason Yim & Brian Trippe
  23. Robust deep learning based protein sequence design using ProteinMPNN
  24. Movie Diffusion explained - Make-a-Video from MetaAI and Imagen Video from Google Brain
  25. Video Diffusion Models
  26. Talkturial: Molecular representations (this is WIP)
  27. Torsional Diffusion for Molecular Conformer Generation - Gabriele Corso & Bowen Jing
  28. Attention Is All You Need
  29. BERT Neural Network - EXPLAINED!
  30. Elucidating the Design Space of Diffusion-Based Generative Models

The header image was created using DALL-E with the prompt a robot painting a picture, abstract oil painting.