Goodbye CINECA! Goodbye Li!

Goodbye CINECA! Goodbye Li!

Last week concluded my (almost) two month stay at CINECA in Bologna. I worked on an interesting project and got to experience the life in Bologna.

Project

The first big part of my project was data preparation. From raw data, stored in a database, data had to be transformed into form suitable for machine learning. Due to the sheer volume of the data, this transformation had to be done in parallel and in batches.

Once the data transformation was completed, I focused on a subgroup of the dataset – data about high-level failure reporting. My task was to try to understand how faults relate to system availability (which faults cause system failures). The nature of this task requires an explainable model – a model that provides reasoning behind predictions.

The first step in creating such a model is to compare explainable models (decision trees) to black box models (neural nets, SVMs, ensemble methods). It turned out that the decision trees achive accuracy that is as high as the black box models (10 fold cross validation with preprocessing included in pipeline – see previous post). This suggests that the problem is

• Suitable for construction of explainable model
• Possesses informative features (attributes)

The problem, however, with decision trees is that they are unstable. The structure of decision trees changes with (relatively) small changes of the dataset. As such they do not provide the most reliable explanation of the model.

The goal for continuation of this work (outside the scope of HPC) is to adopt consensus tree classifier (explainable ensemble method) to work efficiently (in parallel) on HPC infrastructure on big quantities of data.

Additional details about my work are included in my presentation:

Life in Bologna

Work at CINECA and University of Bologna was a great experience. My colleges – my site supervisor dr. Massimiliano Guarrasi (CINECA), mentor: dr. Andrea Bartolini, and co-mentors dr. Andrea Borghesi and Francesco Beneventi (university of Bologna) – were extremely helpful and they created a pleasant working environment.

Also Li was a great roommate.

All in all I am grateful for the opportunity to be a part of summer of HPC 2019