Session 4: Paradigms and applications of Generative AI¶

Date: 2025-10-23

We continue with the learning paradigms of AI and understand the learning process. We then get an overview of generative AI models and applications.

Plan¶

Homework¶

We compare work on paradigms and the code examples.

Homework results¶

Chantal, Alyssa: Geige lernen, Link zum Dokument tbd
Isabell, Athanasios: Spanisch lernen, Link zum Dokument tbd
Leon, Flavio: Schwimmen lernen, Link zum Dokument tbd
Ali: Reiten lernen, Link zum Dokument tbd

Attention mechanisms and transformers¶

One key architectural concept behind large language models and other generative AI models are attention mechanisms and transformers. We discuss these concepts using the following video resources:

Transformers: The best idea in AI - a snippet from the podcast discussion between Andrej Karpathy and Lex Fridman
3Blue1Brown: Transformers, the tech behind LLMs | Deep Learning Chapter 5
3Blue1Brown: Attention in transformers, step-by-step | Deep Learning Chapter 6
Eventually, we also look at 3Blue1Brown: How might LLMs store facts | Deep Learning Chapter 7

Notes about those videos¶

The sentence "a fluffy blue creature roamed the verdant forest" translates to German as "ein flauschiges blaues Wesen durchstreifte den grünen Wald".

Additional resources (not shown in class)¶

The Illustrated Transformer by Jay Alammar is a highly recommended blog post explaining transformers with great illustrations. There is also a book and a course that contains an updated version and some more context.
It is highly recommended to watch the first 15 minutes of the lecture "MIT 6.S087: Foundation Models and Generative AI. ECOSYSTEM" by Rickard Brüel Gabrielsson, also available on YouTube: Lecture 5: ECOSYSTEM. It summarizes the main idea behind foundation models.

Remainder of the term¶

Next step is theory: each participant researches one application area and presents it to the group.
Then we do a small project with a chosen tool.
Finally, we reflect on the implications of generative AI.

Choice of theory topics¶

The students will choose between the following topics:

(Athanasios) Text generation with LLMs (focus on natural language, as well as applications in chatbots, etc.)
(Alyssa) Text based generation of other modalities (focus on text to image, text to video, text to audio and how text prompts are used to control these models)
(Flavio) Image generation with Foundation Models (Diffusion, GAN or others) (focus on how the models generate individual images and graphics)
(Leon) Video generation with Foundation Models (focus on short video clips)
(Isabell) 3D model generation with Foundation Models (focus on generation of 3D objects and scenes)
(Chantal) Audio generation with Foundation Models (focus on music, speech, sound effects)
(Ali) Code generation with LLMs (focus on program code, software development)

The entry point for the research is the curated list of resources on generative AI: awesome-generative-ai. If further resources are needed, students are encouraged to search for additional materials and get inspired by the awesome-generative-ai-guide by Aishwarya Naresh Reganti. There are some more specific resources listed in the theory section as well. You can also use this generated taxonomy to get a better impression of foundation models.

The goal is the get a deep understanding of one application area of generative AI, including relevant tools and frameworks. Students will present their findings in the upcoming sessions so that everyone gets a good overview of the current state of the art in generative AI.

Plans for the upcoming sessions¶

Session 5 on 2025-10-30: Personal review of collected knowledge, book your slot in FELIX (HFU internal).
Session 6 on 2025-11-06: Free for exploration of the Hochschulkontaktbörse. Ask the companies about the following:
- How they use AI in the company.
- Which jobs will change due to AI.
- Whether they are interested in an intern that is familiar with generative AI tools.
Session 7 on 2025-11-13: Presentation of theory topics.
Session 8 on 2025-11-20: Presentation of remaining theory topics.
Session 9 on 2025-11-27: Choice of tool and first status presentations.
Session 10 on 2025-12-04: Personal reflection and status presentations of projects, book your slot in FELIX (HFU internal).
Session 11 on 2025-12-11: Final presentation of projects.
Session 12 on 2025-12-18: Remaining final presentation of projects.
Reflection sessions in January 2026

Materials¶

Code examples¶

See task description for details and links to the code examples.

Taxonomy of generative AI models and applications¶

You can find a generated taxonomy of generative AI models and applications here.

Results¶

tbd