Optimization of neural networks to predict results of mechanical models

Project reference: 2208

Neural networks have shown remarkable performance in lots of domains to solve problems of high complexity. They can, in principle, be used to learn any type of model as they are universal approximators. Their use would be of a great benefit as they can be intended to automate major tasks within an engineering project (such as structural sizing, dimensioning certification, criteria verification, …). However, it is not yet usual to use these technics; may be for lack of competitiveness against feedforward calculations. One major issue is the robustness and the difficulty to ensure high precisions for deterministic predictions.
In this project, we would like to investigate the ability of neural networks to approximate mechanical models and their performance in terms of precision. Increasing accuracy requires understanding how these models work in a deeper way.
Datasets from several models of different complexities will be provided to train neural networks and study their performance and robustness. The type of problems we are dealing with use, conveniently, metrics in terms of relative error. Such metric may yield poor accuracy, especially for target values of low amplitudes.
As a first approach to improve a neural network performance, all needed resources can be used to evaluate the maximum achievable precision. A theoretical reflexion can then be done to understand the unsurmountable limiting factors (numerical precision, model architecture, type of the model, nonlinear behaviour, size of the dataset, …).
After the justification of the limitations, a second step would be to optimize the capacity of the used models in a way to reduce the computational costs whilst maintaining the same level of performance.

Influence of the architecture and nonlinear features on the performance of a neural network.

Project Mentor: Yassine EL ASSAMI

Project Co-mentor: Benoit Gely

Site Co-ordinator: Karim Hasnaoui

Learning Outcomes:
This project will allow the students to understand the functioning of neural networks and practice with high performance computational resources.

Student Prerequisites (compulsory):
This work is relevant for a student with a scientific profile with competence in computational science and machine learning.

Student Prerequisites (desirable):
A knowledge of the mechanical engineering domain is a plus. It is also desirable to have some ease with theoretical mathematics.

Training Materials:
Any course in the field of machine learning could be interesting.
We advise “Fidle project” (for French speakers).
Sites like Coursera, Kaggle and HuggingFace are also good references.

Workplan:
In this project, several data and scripts may be provided for starting.

• 1 week is expected to make some bibliography search about the subject.
• 2 to 3 weeks are needed to study precision enhancement: test different architectures, optimize hyperparameters, make cluster computations for fine tuning.
• 1 week would be needed for theoretical reflexions.
• 1 week to try model optimization to reduce computational cost.

Final Product Description:
The students will implement calculations that should be well documented and conveniently structured to meet scientific required quality. Also, a scientific report about this work is to be provided.

Adapting the Project: Increasing the Difficulty:
We intend to work mostly with dense layers. But other types of architectures may be used if justified in terms of precision (like model stacking or even adversarial networks).

Adapting the Project: Decreasing the Difficulty:
Some provided data is based on very simplified problems that would allow a better understanding of the weights. It is also possible to make studies for specific parameters and study how they affect performance.

Resources:
The students would mostly work with machine learning libraries (Sklearn, Tensorflow) on Python. The use of Git is advised to exchange scripts with the mentors. The resources of the Paris-Saclay mesocentre can be allocated for the duration of the internship [http://mesocentre.centralesupelec.fr/]

Organisation:
IDRIS, Capgemini Engineering – Technology & Engineering Center (TEC)