Interactive weather forecasting on supercomputers as a tool for education

Project reference: 1708

A common and very important application of HPC is that of weather forecasting. In conjunction with the UK Met Office we have developed a new, state of the art, atmospheric model which the scientific community use to simulate clouds. During the 2016 SoHPCprogramme a student developed an interactive outreach tool for setting up initial conditions such as temperature, pressure and the time of the year, then simulating this via our model running on Wee Archie (a mini supercomputer we have built out of Raspberry PIs) and visualising the development of the weather in real time.

Whilst this current demo (written in Python) is really successful for public engagement, we also believe it could be very useful for atmospheric, computational and HPC education targeted at undergraduates. The visualisation is currently split in two and the user can see both the actual weather progression and also aspects of parallelisation & performance such as the trade-off between communication and computation for each core along with the number of simulation seconds per real time second the code is achieving.

There are many choices that a user of our weather model has about their simulation, from what scientific method should be used (for instance for modelling flows in the atmosphere) to what accuracy certain calculations need to be solved to (for instance when solving pressure terms) and it can be very difficult for novice scientists to appreciate and understand their decisions and the impacts that they will have. Atool where the impact of these different choices is illustrated upon their simulation in real time, both in terms of the actual progression of the weather but also the performance & scalability would be hugely useful.For instance, there are many decisions that present an accuracy vs performance trade-off, being able to actually see and experiment with these to understand what is “good enough” will be very powerful and might even be useful for experienced scientists.

Additionally, some areas of atmospheric study require very careful set up and take a while to occur in the simulation (called the model spin up.) An example of this isfog which can take many simulation hours to build up.Understanding exactly how to achieve different conditions as effectively as possible would also be another useful impact of this work.

The simulation code itself is designed for very large core counts on modern supercomputers, such as ARCHER, the UK national supercomputer. For outreach purposes we often run the model on Wee Archie, our mini supercomputer made out of Raspberry PIs, so that people can actually see where their weather will be simulated. With a slightly different focus towards education, this project will target the model running on a variety of different machines, from large supercomputers to clusters of Raspberry PIs that institutions can easily put together.

The first photo illustrates a top down view of cumulous clouds in the current visualisation Python code which have formed during a simulation due to evaporation from the sea. The white grid illustrates the decomposition of these across the different processor cores.

The second photo again illustrates work done during the 2016 SoHPC programme and a side on view of the simulation can be seen, it is raining from the cloud both into the sea and onto the land (with some green crops growing on the land.) The bottom bars illustrate the percentage of time spent in communication (red) and computation (green) for each core. The number on the top left (7.7 sps) is the number of simulation seconds the model is achieving every real time second.

Project Mentor: Nick Brown

Site Co-ordinator: Anne Whiting

Learning Outcomes:

This project involves working with an existing model used for real world atmospheric science. You will therefore gain an understanding of what makes up a computational model, experience with using real world supercomputers, technical skills writing Python codes (specifically for visualisation) and general software developmentpractice.

Student Prerequisites (compulsory): 

A strong programming background, with an interest in both HPC and the visualisation aspects of the field as well as a flair for design.

Some experience with using Python and Linux.

Student Prerequisites (desirable): 

More substantial experience with Python and some use of Fortran. Some exposure toVTK which we use as the visualisation framework.

Training Materials:

Tomislav’s project video (2016 SoHPC student who developed the current outreach tool) https://www.youtube.com/watch?v=yM6LM6jsYvo

We use VTK for the visualisation framework, there are plenty of tutorials & documentation on the website http://www.vtk.org/

Python is used as the implementation language https://docs.python.org/3/tutorial/

Workplan:

• Work package 1: (1 week) – SoHPC training week
• Work package 2: (2 weeks) –Gain experience with the existing code base, both the current visualisation tool and simulation code.Agree on exact scope of work and start preliminary implementation and experimentation towards this. Submit workplan.
• Work package 3: (3 weeks) – Main development phase of the project where the current visualisation and interaction toolis matured and the focus brought towards education.
• Work package 4: (2 weeks) – Final tidy up of code and interface. Produce a video demonstration of the work that has been done. Submit final report

Final Product Description: 

The final product will be an interactive visualisation tool which can be used for education in weather prediction, illustrating the impact that different scientific choices can have upon the fidelity of results and performance of the models. This will be useful to further illustrate to undergraduate students the importance of HPC & computational science and a video will be produced as part of the project which summaries the work done and will help with its dissemination. We believe that the results of this work will be of substantial interest to the UK weather and climate communities.

Adapting the Project: Increasing the Difficulty:

There are many options, for instance the user could take different “views” of the weather system (such as the temperature, pressure etc.)

Resources:

Nothing specific will be needed, the student will be given access to our supercomputers and also other machines such as Wee Archie for this work as well as access to the source code repositories.

Organisation:
Edinburgh Parallel Computing Centre (University of Edinburgh)