Timeline Profiling & Visualisation in Python : The first introduction

Project Description

About a week ago, I was asked to write a few words about my project on Summer of HPC. The abstract on that report was this:

This project’s objective is to make Python interaction with Score-P possible. The use of Python in HPC is increasing, therefore a module that will allow programmers to either automatically annotate and produce event traces for their Python codes, or manually annotate interesting regions seems a great idea.To build the module we will make use of several technologies; not only Python programming and interfacing with C libraries but wrapping C libraries for Python as carried out by Cython. After constructing bindings to the Score-P tracing library from Python, the module will be used to instrument and trace example HPC programs.The application will be used to profile a Python-based HPC application

So now,let me explain all these in simple words.First of all,we have Score-P.Score-P is a scalable performance measurement infrastructure for Parallel Codes. This tool allows you to get several information about your code;from the total time that elapsed to the percentage of the thread that a single function used. However,at this time , this only possible for C,C++ or Fortran codes. So, the aim of this project is to make this tool usable for Python as well.

Output example from my project.

But how?Well,here comes Cython. Cython is a painless way of embedding compiled, optimized bits of code in a Python program. First,it translates Python code into equivalent C code and then it uses a C compiler to create a shared library that can be loaded as a Python module. That means that we combine C with Python so that we can actually connect Python with Score-P.

At the end,what I expect as a result, is a module that will be able to provide to Python codes as much of Score-P functionality as possible.There will also be an effort to add OpenMP support as this is present in Score-P.Finally, we intent to ask Score-P developers to consider adding this code to their package.

In the posts to come I will explain step by step all the aspects of the project and I will try to give some rough information about the technologies used.