Project 2106 – Working on the BSC Interpolation Tool!

Hi again! It’s been a while now, and I have finally gotten to begin working at the Barcelona Supercomputing Centre, helping improve their interpolation tool for atmospheric models! I have gotten to meet the team at the CES department (Computational Earth Sciences), and it has been a blast so far. The interpolation tool is used for regridding atmospheric models in post-processing, outputting the model data in various different ways depending on how it’s needed. This way, we can provide excellent flexibility with the tool. What I am doing now and will continue to do in the next few weeks is trying to improve the performance and scalability of the interpolation tool using HPC techniques and any other innovations that I can come up with.

What is the interpolation tool?

Interpolation is a very powerful mathematical method where we can take a datapoint or datapoints in a time-space domain with some measured variable. Then, we use a combination of these datapoints’ measured variables and positional values in space and time to infer the variable at some nearby, unmeasured point. This enables us to flexibly reconfigure datasets and potentially even increase their resolution! Naturally, one is dealing with large, multi-dimensional arrays when dealing with these types of problems which is where HPC can become incredibly powerful.

Currently, the interpolation tool at BSC interpolates atmospheric variables both vertically and horizontally, vertically using the spline interpolation method, and horizontally using the interpolation package provided by pyproj in Python. The vertical interpolation can be easily parallelised because of its separate layers using MPI, however this is not so simple for horizontal interpolation, which is currently only parallelised using one node.

So what am I doing?

Right now, the horizontal interpolation is carried out before vertical interpolation. What I plan to do is reverse this order. By doing so the horizontal interpolation processes may be able to take advantage of the parallelised vertical processes, and by doing so essentially become parallelise itself and much less computationally demanding. I have been working on this with my mentors and colleagues at BSC, and soon hope to be able to implement my changes into their system. Afterwards I plan to move on to a second task, improving the scalability and performance of the tool’s horizontal interpolation. Currently, this is done using a technique known as pickling, python objects are serialised and stored for later reopening. By doing this, the local memory is freed up for other processes, greatly improving performance. I hope to further optimise this process and see if there are any additional improvements or ideas I can introduce into to tool. I’m super excited to see what I can come up with!