Turbulence Simulations with Accelerators

Project reference: 2210

Direct numerical simulations of turbulence flows are still challenging and need the performance of the fastest super-computers in the world. As the latter are nowadays of heterogeneous architecture, i.e. mixing CPUs and accelerators such as GPUs, algorithms updated. In this project we will explore the benefits of the new programming languages (SYCL and/or Kokkos) for turbulence simulations. SYCL is a high-level programming standard for heterogeneous computing.  It is a single code language (one file for CPU and GPUs) and closely tied to modern C++. Kokkos is library with a usage similar to SYCL. More precisely, this project aims at porting an existing high-order Discontinuous Galerkin (DG) C++ code to CPU-GPU computers in order to challenge well establish CPU codes. The DG code solves partial differential equations, such as shallow water or Navier-Stokes, from fluid mechanics in Cartesian coordinates as well as on the sphere. The latter coordinates use an interesting map called “cubed sphere” borrowed from differential geometry.

Deformational flow on the two-dimensional sphere surface

Project Mentor: Holger Homann

Project Co-mentor: /

Site Co-ordinator: Karim Hasnaoui

Learning Outcomes:
Students will be in touch with modern fluid dynamical algorithms (Discontinuous Galerkin), modern C++ (SYCL and/or Kokkos), and advanced mathematics (partial differential equations and differential geometry)

Student Prerequisites (compulsory):

– Excellent C++ programming skills
– Knowledge on numerical methods for fluid dynamics

Student Prerequisites (desirable):

– Knowledge on CUDA or OPENCL or MPI or OPENMP

Training Materials:

– Excellent tutorial: “Emerging numerical methods for atmospheric modeling” by Nair, R. D., Levy, M. N., & Lauritzen, P. H. (2010) https://opensky.ucar.edu/islandora/object/books%3A214
– Tutorial on SYCL with online coding and testing: https://www.codingame.com/playgrounds/48226/introduction-to-sycl/introduction-to-sycl-2
– Video + hands on Kokkos https://www.youtube.com/watch?v=PjWwith05oA

Workplan:

1-2 week: training and tests of porting a simple 1D Discontinuous Galerkin code with SYCL
3-7 week: Porting of the scientific production code to SYCL/Kokkos

SYCL and Kokkos being a programming languages under heavy development, finding and adapting the best data layout is not trivial. The existing DG code uses modern C++ abstractions. Porting these to SYCL and/or Kokkos is a challenging task that surely benefits form discussion among 2 or 3 students. Various tests of simplified codelets will certainly be usefull.

Final Product Description:
Performance comparison of pure C++ and SYCL/Kokkos implementations of a discontinuous Galerkin solver.

Adapting the Project: Increasing the Difficulty:
The code is made up of different unities that need, such as the spherical geometry in terms of the “cubed sphere” map. Starting from the basics routines, students can study more complex parts of the code.

Adapting the Project: Decreasing the Difficulty:
Only example parts of the code can be tested with SYCL and/or Kokkos and benchmarked to get an idea of the performance of the whole code.

Resources:

– Recent SYCL compiler – such as Intel’s dpcpp
– Kokkos environment
– Modern GPGPU server

Test systems are available in the host institute. A test account on the Idris machine Jean Zay would be beneficial.

Organisation:
IDRIS, OCA-Laboratoire Lagrange UMR7293, Observatoire de la Côte d’Azur