Importance of a Sensei and HPC kung fu!

Didn’t understand?, well that served it’s purpose then!

This time I would simply like to boast the awesomeness of my Sensei, here at Cineca, who has been indirectly, gelled with some fascinating and intriguing conversations, teaching me HPC kung fu! Well, it is me grasping more than him actually teaching, but you get my point!  His official designation for my project is of Site-Coordinator, but Sensei sounds much more awesome, just like he actually is!

Hello again, this is my 2nd blog for my Journey as a PRAACE Summer of HPC 2018 participant. This time I’ll walk you through the some of the sneak peaks of my work so far, as I have developed, debugged and tested the brand new software called Catalyst, which is a subset of another visualization software called Paraview

What does this do?, well it shows images (results), right off the bat, while you’re simulation is running (in my case a CFD simulation). That means with this bad boy, you could have your initial results just as quickly as possible without having to wait untill the last step of the run.

Just think of this software as another game changer for CFD research and workflows! Before Catalyst, this was not possible. Which is why I get to be the lucky one to test it with a full scale – use as many cores feasible – SuperComputer level- HPC test case. So that researchers, academics , or just another curious nerdy kid, could later easily post process their setups, making their life easier. Amazing, right!

As to what exactly I am running. That is an injector which is slightly different than the ones used in car, but it is a model after all! And also the damBreak Case of OpenFOAM which comes in default with the software. Both of them have two phases of fluid involved. So we basically at the end of our test, observe to see one fluid morph into other as time proceeds.

Easier said than done, there are the following elements involved, if I need to test this right.

1) The CAD aspect, (the geometry and mesh should be correct, as is required for any CFD setup)

2) The OpenFOAM aspect, this sets up the geometry or rather the mesh, to be specific. The solver, think of it as a fancy calculator that solves big equations, and output details  i.e to say how often do you want your results to be stored and so on. All of which, of course should be set correctly if one wants to avoid the “Garbage in, Garbage out” results!

3) The HPC aspects would involve:
a) The supercomputer, which is the giant tool that actually does the operations and calculations mentioned above, in parrallel.
b)Its access, so that you don’t have to wait in line for your calculation.
c)Your partition, or the place where you run your job on a supercomputer, must have enough resources, like the processing power and so on.
d)The installed and supported packages (so that apples are calculated with respect to apples and oranges with respect to oranges).

The last aspect is especially important because that’s what I am here for. When you have an OpenFOAM job to be submitted to a cluster, the software that works with OpenFOAM , namely Paraview, has been widely tested and appreciated. What has not been tested is the Catalyst side of it. A fairly recent development of sorts, and hence the necessity for its research.

So it is obvious to run into an error every now and then , which may come from any of the above mentioned aspects. And because such a testing is being done for the first time, the number, instances, flavour and variety of errors encountered, increase even more. Mind well, these aspects, in actuality, have much complex dependencies with one another as well as among themselves than what is mentioned.

So the other day, I was couple of hours in, fixing this error I’ve got. “Attribute error: Name not defined”. To give you a context, if one wants to find the vorticities of a flow in this software, a filter would be required, which filters out only the required calculation needed to compute vorticies. And it didn’t seem to work, no matter what I tried.

So, my Sensei comes in and asks for the updates, I brief him about the developments uptill now, and he smiles and tells me that this error was not up for me to fix!!. Turns out, the feature didn’t exist in Catalyst’s current edition (as of 10.08.2018) , and had to be edited in the current edition with a patch file, then recompile it again for the cluster, and then install afresh for this feature to work.

This is when I see the HPC kung fu of my Sensei. He very calmly does the above one by one, explaining to me what’s going on along the way. Few steps extremely intricate because the supercomputer already has many versions of some packages/softwares already installed, which he must navigate through and adjust accordingly, mentally keeping a tab on where he needs to change a setting if required and simultaneously foreseeing any dependencies that might affect at a later stage of compilation.

And then after 2-2.5 hours of tweaking, (remember compiling the whole software again is a lengthy process), the thing is installed (well the workaround at least). Oh and did I mention, it worked in the first attempt.

Just check out the preliminary results!!

I may be stating the obvious here, but to just see him do his jam, was very thought provoking. I may have just scratched the surface of how much consideration goes into software development, especially when you do it on a  HPC scale, just by watching him. I’ve never had a mentor before, let alone a one on one mentor. And I must admit, to be striving toward a goal with a mentor is a whole different ball game. The one which you have already won, because you have learned so many things.
Which is why I appreciate so much, that I got this wonderful opportunity to work at Cineca on this project. Those simple conversations that are the by product of his experience, and his wisdom are simply amazing. I at least for one, always look forward to strike up a conversation with my Sensei, just to see what new thing I might acquire in the process!

I perhaps wouldn’t have been able to acquire and learn so many things, if it were not for him. The least I could say is
Arigatto Sensei!!