Neuro-symbolic General Policy Learning

Tue, 01 Oct 2024 00:00:00 +0000

Supervisors

Timo P. Gros, Jörg Hoffmann, Daniel Höller, Nicola J. Müller and Prof. Dr. Verena Wolf

Content

Symbolic AI relies on explicit, human-readable symbols and rules to represent knowledge and solve problems. This approach contrasts with modern, data-driven methods like machine learning, which rely on statistical patterns rather than explicit rules. Recently, both approaches have been combined, leading to the development of neuro-symbolic methods.

For instance, heuristic search, as a very successful part of symbolic AI, must be applied individually to each instance of a domain or problem. In contrast, neuro-symbolic general policy learning focuses on neural network-based policies that can solve all instances within a specific domain, combining the benefits of symbolic and data-driven AI.

In this seminar, we explored the emerging trend of learning neuro-symbolic general policies where the underlying architecture is based on Graph Neural Networks or Transformers.

Format

A seminar with 7 graded ECTS points, run in block format: an initial meeting in November and a block of student presentations in March. Each participant read and understood a piece of research, wrote a summary paper in their own words, gave a presentation, and gave detailed feedback on the paper and presentation of a fellow student.

Topics

Area 1: Analysis of General Policies (supervisor: Timo P. Gros)

Simon Ståhlberg, Blai Bonet, and Hector Geffner. Learning general optimal policies with graph neural networks: Expressive power, transparency, and limits.
Simon Ståhlberg, Blai Bonet, and Hector Geffner. Learning generalized policies without supervision using GNNs.
Rostislav Horčík and Gustav Šír. Expressiveness of graph neural networks in planning domains.
Dillon Z. Chen, Sylvie Thiébaux, and Felipe Trevizan. Learning domain independent heuristics for grounded and lifted planning.

Area 2: Training of General Policies (supervisor: Daniel Höller)

Sam Toyer, Felipe Trevizan, Sylvie Thiébaux, and Lexing Xie. Action schema networks: Generalised policies with deep learning.
Or Rivlin, Tamir Hazan, and Erez Karpas. Generalized planning with deep reinforcement learning.
Simon Ståhlberg, Blai Bonet, and Hector Geffner. Learning general policies with policy gradient methods.

Area 3: General Policies using Transformers (supervisor: Nicola J. Müller)

Nicola J. Müller, Pablo Sánchez, Jörg Hoffmann, Verena Wolf, and Timo P. Gros. Comparing state-of-the-art graph neural networks and transformers for general policy learning.
Nicholas Rossetti, Massimiliano Tummolo, Alfonso Emilio Gerevini, Luca Putelli, Ivan Serina, Mattia Chiari, and Matteo Olivato. Learning general policies for planning through GPT models.
Lucas Lehnert, Sainbayar Sukhbaatar, Paul Mcvay, Michael Rabbat, and Yuandong Tian. Beyond A*: Better planning with transformers via search dynamics bootstrapping.
Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling.

Organization