Making Waves

This week, I show how I created animations for collective communications and transferred them to Wee Archie (with videos) and introduce the wave simulator. I’ll also talk about issues with my system on Wee Archie and how this will affect my goals going forwards. Make sure not to miss the cute animal pictures at the end!

Animating Collectives

Since my last post, I’ve created a prototype of every type of MPI communication needed for an outreach demonstration. You can see the collective communications in the video below, run on a Wee Archlet. For details on how I made these animations, and what a Wee Archlet is, see the previous post. These aren’t quite the final animations, but are a good proof of concept which I’ll polish as I create the final interface for the demos.

For full context, a collective communication is a one-to-many or many-to-many communication, vs. the simpler one-to-one communications of last post. Some of the most commonly used collectives are shown above. In one-to-many, a master Pi is needed, one which controls the flow, gives out all the data, or receives all the data. Here, this is always the bottom Pi.

Gather will collect data from every computer in your group and drop it all on one. Reduce will do the same, but apply an operation as it goes to the data – here it sums it up. Broadcast will send data from the root to every computer in the group. Scatter will do the same, but rather then send them all everything, they each get slices of the data.

Animating for Production

Once I had finished prototyping my animation server on the Wee Archlet, and created animations for many simple operations, it was time to get it working on Wee Archie. Transferring my animations to render on Wee Archie was easy in theory… It ended up taking less time than I thought it was going to, but there were still many complications that need to be ironed out! Below you can see one of my first demonstrations on Wee Archie.

How would you broadcast a message from one computer to every other one on a network? Would you just get it to send it to them all, one by one, or try and send them all at once? Neither is perfect – what if the message is large, or the network is slow to start a connection on? A different approach to sending these messages is shown below, which is the best way to make full use of network where all the computers are connected to each other, and the actual processing of the message will take a while compared to the sending.

Here, you can see the broadcast as shown earlier, but broken down so you see how it happens inside MPI (a simplified version, anyway). Here, one of the central Pis will send a message to every other one. There are 16 of them, so with the implementation I show, you can do it in 4 steps! Assuming all of your Pis are connected to each other, and all connections are as fast as each other, this is a very fast way of getting a message across.

Initially, the central Pi will send the message to a neighbour. This means the neighbour also has the message, so they can both pass it on! When they do this to their neighbours, now 4 have the message. Then they will all pass on the message to another neighbour, and then we have 8. If there were more, this would continue on, but we’re done after the next step, when the 8 send on their message, and all 16 Pis have it!

This demonstration will be the final one in a series of tutorials building up to it, illustrating the basics of parallel computing on Wee Archie. They include all the previous demos I’ve shown in some form, with explanations and motivation as you go. They’re mostly written now, and when the first version is complete, they’ll all be posted on the Wee Archie Github repository.

The main issue with the current system is lack of synchronisation. Because I use HTTP requests to queue all my animations, they often end up out of sync – starting a connection takes time. A potential fix for this is setting up a network of pipes using WebSockets, which I’ll investigate soon. The other main issue is playback speed – I need a global control for frame rate on the various animations, as well as the ability to change this as the circumstance needs it. Hopefully making this part of the server doesn’t prove too difficult, as it would improve the feel of the animations a lot!

Introducing the Sea

So, the famed wave simulator, or coastline defence simulation – what is it?

Initially developed by my mentor Gordon Gibb, it’s the most recent demonstration for Wee Archie. It asks the user to place a series of coastal defence barriers – represented here by gray blocks. You have a budget, and each block has a fixed cost. The coastline has several different locations pictured. They are more or less susceptible to being damaged, and damage costs various amounts.

For example, the beachfront is cheap and easy to damage. The cliff-top housing is hard to damage and expensive. The library, expensive and easy to damage. The shopping market is somewhere in-between them all.

When you place the barriers and set it running, it will run a fluid simulation, and show the waves hitting the shore, calculating the damage they do, and how effective your barriers are! At the end of it, the amount of money saved/spent is your score. It’s quite simple to use, and very popular at outreach venues such as Edinburgh Science Festival.

All the current demonstrations use a framework where Wee Archie runs a server on the access Pi – the left of the two on top in the video. This server will run the simulations when called, and give the data back to the client, a graphical Python application running on the connected laptop.

Upcoming Goals

So this week I’ll be working on getting the wave demo working using my version of MPI, showing the communication as it happens. I won’t be able to show all of them, as to render the simulation, there are hundreds of operations per second, but I’ll display the main logic of the program once or twice out of those hundreds.

The nice thing about visualising communication is that unlike the data in the program, the way the computers communicate stays the same throughout the execution. It also doesn’t need advanced mathematics to understand. This makes it a good thing to sample and explain in detail!

The next goal after this will be to change the graphical interface from Python to running in the browser. Once I set this up, I’ll be able to host a website on Wee Archie, and all the client computer will have to do will be connect! The website will be able to host all the tutorial I write and the demonstrations, and easily include extra details, explanations and resources.

If it goes well and porting over the old demos is fast enough, I anticipate it being the future for Wee Archie. If I have all that working by the end of the project I’ll be very pleased with the progress made, as it will have improved the experience of using Wee Archie enormously. Rather than a collection of independent demos, it can be a cohesive whole which lets students explore at their own pace and learn as they go!

Sightseeing Segment

It’s been a couple of weeks, but it’s been very busy. I haven’t had time to see too much, though last weekend I visited the beautiful Pentland Hills regional park. The swan featured in this post is from the reservoir there in Glencorse. It’s a beautiful park, with many options for different visitors, though I stuck to some trekking through the fields – getting a bit sunburned in the process. I hope to go swimming in the reservoir with the others if the weather holds!

• 

• 

• 

Sadly, I am not much better at ping pong then I was previously. I don’t think I was cut out for this line of rapid reflex wrist-flicking work. I’ve managed to take one game off Benjamin, but it was in the face of many! I hear the Fringe has some public tables which are better than the one in our accommodation, so perhaps that will help?