Often, a parallel algorithm will require moving data between the engines. Oneway of accomplishing this is by doing a pull and then a push using themultiengine client. However, this will be slow as all the data has to gothrough the controller to the client and then back through the controller, toits final destination.

Following is a matrix multiplication code written in MPI (Message Passing Interface) which could be run on CPU cluster for parallel processing. This has been successfully tested with two square matrices, each of the size 1500.1500. Changes in this release: See this page if you are upgrading from a prior major release series of Open MPI. It shows the Big Changes for which end users need to be aware. See the NEWS file for a more fine-grained listing of changes between each release and sub-release of the Open MPI v4.0 series. The MPI C class library must provide all functionality deﬁned by MPI-1. T o the greatest extent possible, this functionality should be provided through member functions of objects, although some. MPI, the Message Passing Interface, is a standard API for communicating data via messages between distributed processes that is commonly used in HPC to build applications that can scale to multi-node computer clusters. As such, MPI is fully compatible with CUDA, which is designed for parallel computing on a single computer or node.

Since, currently, the MPI for C is developing stronger than C (see References), I am wrapping MPI-C commands in C classes here. If we imagine the data transfer between two nodes is like a ping-pong game, the data is the ball. The ball can be an array of MPI types. Here for simplicity, is the integer.

A much better way of moving data between engines is to use a message passinglibrary, such as the Message Passing Interface (MPI) [MPI]. IPython’sparallel computing architecture has been designed from the ground up tointegrate with MPI. This document describes how to use MPI with IPython.

Additional installation requirements¶

If you want to use MPI with IPython, you will need to install:

A standard MPI implementation such as OpenMPI [OpenMPI] or MPICH.
The mpi4py [mpi4py] package.

Note

The mpi4py package is not a strict requirement. However, you need tohave some way of calling MPI from Python. You also need some way ofmaking sure that MPI_Init() is called when the IPython engines startup. There are a number of ways of doing this and a good number ofassociated subtleties. We highly recommend just using mpi4py as ittakes care of most of these problems. If you want to do somethingdifferent, let us know and we can help you get started.

Starting the engines with MPI enabled¶

To use code that calls MPI, there are typically two things that MPI requires.

The process that wants to call MPI must be started usingmpiexec or a batch system (like PBS) that has MPI support.
Once the process starts, it must call MPI_Init().

There are a couple of ways that you can start the IPython engines and getthese things to happen.

Automatic starting using mpiexec and ipcluster¶

The easiest approach is to use the MPI Launchers in ipcluster,which will first start a controller and then a set of engines usingmpiexec:

Mpi In Dev C 2b 2b Test

This approach is best as interrupting ipcluster will automaticallystop and clean up the controller and engines.

Manual starting using mpiexec¶

If you want to start the IPython engines using the mpiexec, justdo:

This requires that you already have a controller running and that the FURLfiles for the engines are in place. We also have built in support forPyTrilinos [PyTrilinos], which can be used (assuming is installed) bystarting the engines with:

Automatic starting using PBS and ipcluster¶

The ipcluster command also has built-in integration with PBS. Formore information on this approach, see our documentation on ipcluster.

Actually using MPI¶

Once the engines are running with MPI enabled, you are ready to go. You cannow call any code that uses MPI in the IPython engines. And, all of this canbe done interactively. Here we show a simple example that uses mpi4py[mpi4py] version 1.1.0 or later.

Mpi In Dev C 2b 2b 1b

First, lets define a simply function that uses MPI to calculate the sum of adistributed array. Save the following text in a file called psum.py:

Now, start an IPython cluster:

Note

Mpi In Dev C 2b 2b 1

It is assumed here that the mpi profile has been set up, as described here.

Finally, connect to the cluster and use this function interactively. In thiscase, we create a distributed array and sum up all its elements in a distributedmanner using our psum() function:

Any Python code that makes calls to MPI can be used in this manner, includingcompiled C, C++ and Fortran libraries that have been exposed to Python.

Mpi In Dev C 2b 2b 2c

[MPI]

Message Passing Interface. http://www-unix.mcs.anl.gov/mpi/

[mpi4py]

(1, 2) MPI for Python. mpi4py: http://mpi4py.scipy.org/

[OpenMPI]

Open MPI. http://www.open-mpi.org/

[PyTrilinos]

PyTrilinos. http://trilinos.sandia.gov/packages/pytrilinos/

Mpi In Dev C%2b%2b