Completion of the PBL Project
“Application of machine learning in the search for the optimal sequence of movements during room sweeping”.
with students of the 3rd Secondary School in Katowice
A project implemented since 04.2023 with secondary school students under the Excellence Initiative – Research University Programme was completed in our Department on 30.09.2023. The project team consisted of students from the 3rd A. Mickiewicz High School in Katowice, currently 4th grade students: Wojciech Siostrzonek and Filip Morawiec. The supervisor of the project on the part of the Silesian University of Technology was prof. Witold Beluch.
The aim of the project was to develop a methodology for cleaning optimisation using machine learning algorithms for given assumptions. The project analysed and tested various algorithms and selected the one best suited to solve the problem posed. Convolutional neural network training was carried out and preliminary results were obtained.
The initial assumptions of the project are shown schematically in Figure 1:
A computer programme consisting of 2 modules was developed (Fig. 2):
- An environment that, among other things, based on the start and end coordinates of the broom movement the next state and the amount of dirt that has fallen into the mound.
- A cleaning agent that learns how to clean optimally by acting on the environment.
An example of the environment is represented by Fig. 3, which shows randomly placed mounds in purple and walls in blue.
The following project implementation methods were used:
- Python implementation;
- use of a machine learning library: Tensorflow/Keras;
- use of the Matplotlib library to visualise the operation of the algorithms and room simulation.
Tried and tested the performance of the following neural network learning algorithms: A2C, DQL and DDPG of which the latter (Deep Deterministic Policy Gradient, Figure 4) proved to be the most effective.
The actor and critic network architectures are shown in Figure 5:
E.g. an actor making an observation-based decision takes the environment as a 3x35x35 tensor – these are the 3 layers of a square room, of which: the first corresponds to the layout of the walls, the second corresponds to the layout of the mounds and the third corresponds to the density of the dirt.
A visualisation of the training is shown in Figure 6:
The config.cfg configuration file sets the more important simulation parameters, such as the width of the broom or the number of mounds.
