Special Topics in Deep Reinforcement Learning

Team

Prof. Dr. Verena Wolf, Timo P. Gros (lecturers)
Magnus Cunow (organization)
Julius Gabelmann, Joshua Meyer, Nicola J. Müller (assistants)

Content

Reinforcement Learning (RL) is a popular sub-discipline of Machine Learning for problems that require strategies to solve complex tasks such as board games, scheduling problems, or other discrete optimization problems. Deep Reinforcement Learning (DRL) is the combination of Reinforcement Learning with Neural Networks.

In this seminar, we first covered the most important algorithms of Deep Reinforcement Learning. Afterwards, we looked at special topics that our group is conducting research on: exploration techniques, agentic RL, and AI planning.

Format

After a kick-off in April, each participant was assigned a paper and presented it in one of two seminar blocks in June, wrote a summary paper about it, and worked on a practical project in a team. The seminar ended with a closing session on the project results, with prizes, and short presentations of the group’s current research for students interested in a Master’s thesis or a HiWi position.

Topics

Deep Reinforcement Learning

1. Value-based DRL Evolution (core: Rainbow)
2. Actor-Critic: Structure & Estimation (core: A2C/A3C)
3. Policy Optimization Stability (core: TRPO & PPO)
4. Continuous Control in DRL (core: DDPG & SAC)

Planning & RL

5. FOL for goal-conditioned RL
6. Sketch Decomposition via DRL
7. Model Aware Policy Transfer using Q-Learning

Exploration

8. NovelD
9. DEIR
10. Episodic Novelty Through Temporal Distance
11. Cell-Free Latent Go-Explore

Agentic RL

12. Group Relative Policy Optimization Algorithm
13. Tool-Integrated RL
14. Reflexion: Verbal RL
15. Multi-Agent RL

Organization

All information and materials are available in the CMS.