HSE AI Research Centre Simplifies Particle Physics Experiments

© iStock

Scientists at the HSE AI Research Centre have developed a novel approach to determining robustness in deep learning models. Their method works eight times faster than an exhaustive model search and significantly reduces the need for manual verification. It can be applied to particle physics problems using neural networks of various architectures. The study has been published in IEEE Access.

Machine learning (ML) and artificial intelligence (AI) are increasingly used in particle physics to make the analysis of experimental data easier and faster. Neural networks, for example, help process instrument signals and reconstruct missing information about particle properties. Since such predictions influence subsequent analysis, it is essential to understand how robust they are. However, in practice, model accuracy is often the only metric evaluated, and little attention is paid to how much the results may vary across different training instances. This issue is particularly pronounced for deep neural networks: their behaviour is hard to interpret, and repeated training can produce noticeably different outcomes. As a result, despite their potential advantages, many physicists remain wary of using neural networks.

Scientists at the HSE AI Centre have proposed a novel solution. They developed a method that automatically compares dozens of neural network variants and identifies the most reliable and stable ones. The idea is as follows: if a model is trained repeatedly on slightly modified data with different initial weights, the distribution of errors reveals how robust the model is to small changes in conditions. A robust model will produce nearly identical results across these tests.

The researchers tested their method on a task in which, based on an image formed by the cells of an electromagnetic calorimeter, one must determine the energy of a particle and the point at which it struck the detector. An electromagnetic calorimeter is a device made up of many cells that measure the amount of energy deposited in each cell when a particle passes through.

Fedor Ratnikov

'For the analysis, we generated half a million virtual signals simulating the detector’s operation and repeatedly fed them into different models, each time changing the training and test samples. We then used this method to identify the most reliable models and examine their properties. In doing so, we determined the minimum amount of training data required for a model to become robust—that is, to perform consistently across different training runs,' comments Fedor Ratnikov, Leading Research Fellow at the Laboratory of Methods for Big Data Analysis of the HSE AI and Digital Science Institute.

A key element of the approach is a special selection algorithm. For each model variant, the researchers collected a set of errors accumulated over dozens of independent runs and used this distribution to estimate how predictably the model behaves. This makes it possible to automatically filter out models that performed well only by chance and to identify those that remain stable under any reasonable changes in conditions.

Alexey Boldyrev

'We repeatedly trained all the models on half a million calorimeter simulation events, each time splitting the data into new training and test sets and initialising the weights randomly. This allowed us not only to measure how often each model made mistakes but also to track how its learning behaviour changed from one run to the next,’ comments Alexey Boldyrev, Research Fellow at the Laboratory of Methods for Big Data Analysis of the AI and Digital Science Institute.

The study also showed that models supplied not only with raw signals but also with simple, pre-known physical values require less data and reach stable results more quickly. The authors estimated the minimum amount of data needed for such models to maintain consistent performance across runs and identified two architectures that were reliably accurate and robust.

Andrey Shevelev

'The new method allows for much faster selection of robust AI models for certain particle physics problems—achieving results eight times more quickly than the traditional exhaustive search of all model variants,' comments Andrey Shevelev, Research Assistant at the Laboratory of Methods for Big Data Analysis of the AI and Digital Science Institute.

The researchers emphasise that the algorithm is fully automated and does not require manual tuning. As a result, it can serve as the foundation for self-learning systems that operate robustly, regardless of fluctuations in the training data or inherent model limitations.