Passing a library-defined matrix over MPI
The communication of data between processes is a fundamental concept of MPI. In algorithm development one usually makes use of external dedicated linear algebra libraries (I choose Armadillo for my summer project) mainly because they are well tested, have good performance and optimized structures. Initially one may not think of a straightforward way to pass a library-defined matrix over MPI, but soon you find out it is not complicated at all. From now on I will deal with Armadillo C++ library.
The MPI buffer is a pointer to a block of memory and MPI expects the elements you want to communicate to be in contiguous memory space. If you use a high quality linear algebra library, this is already true (Armadillo, Eigen and Blitz++ are just an example). In this case you need only pass the MPI function a pointer to the base of the object. This can be achieved with an iterator referring to the first element ( .begin() in Armadillo ). For instance, to broadcast an Armadillo matrix, one would call MPI_Bcast the following way:
Now what if, for whatever reason, you do not have a matrix object stored contiguously in the memory? A solution might be writing the matrix to a one-dimensional array and have that sent over. However I would like to address a solution involving a 2D array, which shows how to allocate a multidimensional array on the heap. What we need is an array of pointers to arrays: basically one new for an array of doubles and another new for an array of pointers to doubles. Then each pointer is set to point to the first item in the row. A function implementing an MPI broadcast is shown below (rows 5 to 9 are of interest here).