Reinforcement Learning for
Automotive Production Control

MoDigPro — Modernisierung und Digitalisierung der Produktionssteuerung des Ford Werkes in Saarlouis zur störungsfreien kontinuierlichen Produktion durch multidimensionale Algorithmen und künstliche Intelligenz — was a cooperation with the Ford Motor Company and the Max Planck Institute for Informatics.

In a car manufacturing process, the order of production is crucial for every production step; especially for the final assembly, where all pre-produced parts of the car are put together.

While more recent factories have a more modern structure, such that there is a generous number of options to optimize this production order, in the Ford factory in Saarlouis, the options to optimize the production order are limited by the line buffer, which can store and take out cars from one of 27 different lines. Currently, the optimization in Saarlouis is done manually by following some rules of thumbs.

In this project, we compensate for this structural disadvantage with intelligent control of the line buffer. We use deep reinforcement learning to train neural networks by self-play. These neural networks are supposed to take over control of the line buffer.

Since deep reinforcement learning is known to be data hungry, training the neural networks while they are deployed in the factory is not an option. Instead, we use discrete event simulation to construct a digital twin of the line buffer and the final assembly. We can use this simulation to train the agents for optimizing car production in Saarlouis.