Online visualisation of current and historic supercomputer usage
Project reference: 1709
We house and run the UK national supercomputing service, ARCHER, which has an active user base of around 3000 users. Many different people rely on this machine as a crucial instrument in their science and there are a very wide variety of tools, technologies and communities involved with a modern HPC machine.
Not only is it desirable to be able to analyse this data and provide some visualisation to help track what is working well and what needs attention, but policy makers (and ultimately the funders) find this sort of information crucial when deciding what the future of HPC should look like.
We have large amounts of present day data, but we also have data going back around 10 years which we can take advantage of to track trends & changes in the community as time has progressed.
We have built a small prototype (http://www.archer.ac.uk/status/codes/) which is fairly basic and only takes advantage of a very small amount of the overall data, but this in itself has been of significant interest to people and will be a starting point for this project to build upon.
The interests of the student will drive the exact project and there are plenty of challenges which we could tackle, from the deep technical side of developing an advanced online visualisation tool where one can explore the data in detail, to questions around exactly how we can best illustrate the data in a way that is accessible and easy to understand.
Variety of data visualisations possible using the D3.js tool (taken from https://d3js.org/).
Project Mentor: Andy Turner
Site Co-ordinator: Anne Whiting
Learning Outcomes:
Real world data analytics and visualisation, exposure to software development techniques and web development technologies (HTML5, Javascript and common frameworks.)
Student Prerequisites (compulsory):
Some programming background, experience of developing using HTML and javascript. A desire to pick up and learn new technologies.
Student Prerequisites (desirable):
Existing data analytics and visualisation experience. Some experience with the D3.js javascript library.
Training Materials:
Workplan:
- WP1 (1 week) – SoHPC training week
- WP2 (2 weeks) – Gain experience with the existing prototype, understand the data and the potential for analytics & visualisation
- WP3 (3 weeks) – Main development phase, extend the prototype to provide an advanced visualisation & exploration capacity for current and historical data
- WP3 (2 weeks) – Final tidy of the code, produce a video describing the project & achievements. Write and submit final report.
Final Product Description:
An online tool will be developed that visualises, and potentially allows one to explore, the usage data of modern HPC machines. This could then be used by other PRACE partners to help explore their own usage data. In terms of outreach, the general public find it very interesting to see what sciences take advantage of HPC machines and the tools/technologies that HPC developers routinely use.
Adapting the Project: Increasing the Difficulty:
A more advanced online visualisation where people can interactively drill down into the data and explore trends, anomalies and other factors.
Resources:
The current and historical usage data, we have this stored in house and it is managed by Andy Turner (the project mentor) who will make it available. None of the data is sensitive and it is all anonymised so there are no privacy concerns.
Organisation:
Edinburgh Parallel Computing Centre (University of Edinburgh)
Leave a Reply