Timeline profiling and visualisation in Python
Location: Edinburgh Parallel Computing Centre (EPCC), Edinburgh, Scotland, UK
Project Abstract:
The use of Python in HPC is increasing as many programmers start to realise that they need not write their entire software stack in a traditional high performance language like C or Fortran. Instead, the high-level control constructs are written in Python and the computationally intensive parts of the code are pushed down into a compiled language.
Unfortunately, it is current not possible to use the profiling and visualisation features provided by tools such as Scalasca (http://www.scalasca.org/) and Vampir (http://www.vampir.eu) with such codes.
This project aims to address this by providing Python bindings for the Score-P tracing library (http://www.vi-hps.org/projects/score-p/). This will allow programmers to either automatically annotate and produce event traces for their Python codes, or manually annotate interesting regions.
The project will then use this new interface to profile a number of Python-based HPC applications, predominantly addressing finite element problems.
Distribution of individual process computation times in PFLOTRAN stepperrun (‘flow’+‘transport’) with 65,536 processes on Jaguar Cray XT5 presented by Scalasca.
(From Brian J. N. Wylie, Markus Geimer: Large-scale performance analysis of PFLOTRAN with Scalasca. In Proc. of the 53rd Cray User Group meeting, Fairbanks, AK, USA, Cray User Group Inc., May 2011.)
Project Mentor:Lawrence Mitchell
Site Co-ordinator: Irina Nazarova
Learning Outcomes:
The student will gain experience in:
- Python programming, and interfacing with C libraries
- performance analysis through profiling/tracing and their visualisations
Student Prerequisites (Compulsory):
Student should be comfortable programming in:
- Python and C
Student Prerequisites (Desirable):
- Experience of writing shared libraries. Familiarity with parallel programming (MPI).
- Passing familiarity with one of Swig, Cython or Python ctypes (see training materials)
Training Materials:
Wrapping C libraries for Python can be carried out by one of:
- Swig (www.swig.org)
- Cython (www.cython.org)
- Python ctypes (http://docs.python.org/2/library/ctypes.html)
Project Application Reference: UK – EPCC – Timeline Profiling
Applications are now closed
