Python, MPI, CFD and ARCHER – what is a bigger challenge?
Hi Guys!
In this blog post, I would like to introduce you my project “Parallelising Scientific Python Applications” which I’m working on at EPCC under Dr. Neelofer Banglawala. Wait! Sit down, don’t worry and get a cup of excellent British tea. I promise I’ll try to explain it easily and amusingly.
To be original, I decided to draw all illustrations by my hand, in spite of not being a very good painter, actually. I could use some professional graphic tools, of course, but this is handmade! And gluten free and diary free … and it’s simply cool! Pure bio quality and no animal came to harm while creating this article.
Let’s start with what I actually do. I was given a serial CFD code (skip “serial” if it’s difficult) and I’m trying (really hard) to make it better, faster and more efficient – parallelise the problem (simply, save time and money for you). What does CFD mean? CFD stands for computational fluid dynamics. Fluid dynamics studies the mechanics of fluid flow (liquids and gases in motion).For example, you can imagine a car in the wind tunnel (F1 car aerodynamics), blood flow in vessels (possibility of aneurysm rupture), combustion processes (Jumbo jet engine), traffic modeling, etc.
Why do I need a supercomputer? BTW what is a supercomputer? A Supercomputer is a highly powerful machine. Simply but not precisely, you can image it as a hangar filled with extremely good personal or gaming computers, connected to huge data storage via a fast communication network. I’m using the ARCHER supercomputer for my project. ARCHER belongs among 50 fastest supercomputers in the world this year. And no, I’m not using a supercomputer just for fun, although it could be great as well. The CFD problem is counted among biggest scientific challenges, is very demanding for memory capacity and computational performance. A realistic computation wouldn’t be realizable on a usual laptop, even 100x faster than Apple MacBook.
Fig. 1: Fast prototyping with Python.
Why am I using Python? Imagine, there are two islands. On the island S (like Scientist), there are the top scientists. On the island C (like Computer geeks), there are very good programmers. (Have you noticed the hidden connection between S and C? Super Computing?). The problem is that there is a deep ocean between the islands. So, the aim is to find such a language (precisely a guy who masters it) which connects those distant islands with a bridge. The solution can be just Python (ehm, Python Ninja) because Python is easy to learn and fast to prototype (as you can see in fig. 1).
Fig. 2: Python connecting a scientific field with computation.
Great, now you know what CFD is, why I need a supercomputer and why I use Python. That’s nice but I need an appropriate way to force my code running on a supercomputer efficiently. When you run a code on a supercomputer, it doesn’t mean that it will be automatically super fast, efficient and pretty awesome! You need to write a perfect code which knows how to run on such a machine. That’s why I’m using MPI (Message Passing Interface).
Fig. 3: A CFD formula.
Fig. 4: A grid of processors – each working on it’s own tile.
What does that mean for me? Let’s have a look at the picture with a formula. In figure 3, you can see the air blowing around a formula and a driver inside a wind tunnel. Using MPI, you can divide this wind tunnel into many tiles and let each tile to be computed by a single processor (e.g. create a grid of processors as it’s shown in figure 4) – this process is called decomposition. For example, imagine that one processor is computing air flow around the front right wheel, another processor is computing the area surrounding the driver’s helmet, and so on. Usually, it is necessary for the processors to exchange data on the borders with their neighbours (in this case the air is blowing over multiple tiles). To do so, you have to make each tile a little bit bigger so the neighbouring tiles are overlapping. This overlaps are known as halo zones.
More precisely, as you may know, the air is made of millions of molecules. The movement of these molecules causes the air flows. The molecules enter the wind tunnel, hit the front spoiler, then flow over the driver’s helmet, meet the rear spoiler and fly away. The halo zones ensure these molecules can travel between tiles (not get trapped in the front spoiler’s tile). This molecule exchange is orchestrated by MPI. The principle is depicted in figure 5.
Fig. 5: Principle of halo zones swapping. The colour hatched stripes are the shared borders.
Enjoy!
Marta
(You can also have a look to some of my previous blog posts:
Leave a Reply