MPI: Message Passing Interface Multiprocessing API for distributed memory systems. What is Message Passing? Message passing is a computational model in which processes have only local memory (they lack a common shared memory), but are able to communicate with other processes by sending and receiving messages. Where can we use Message Passing? Message passing can be used where-ever it is possible for processes to exchange messages: • Distributed memory systems • Networks of Workstations • Even, shared memory systems. MPI The Message Passing Interface • MPI is an API for parallel programming on distributed memory systems. Specifies a set of operations, but says nothing about the implementation. • Popular because it many vendors support it, therefore is very portable. • Most common implementation: MPICH. MPI MPI provides specifications for: • • • • Process creation Process termination Process ordering Interprocess Communication MPI Operations: Basic Set. MPI can be thought of as a small specification, because any complete implementation need only provide the following operations: • • • • • • MPI_INIT MPI_COMM_SIZE MPI_COMM_RANK MPI_SEND MPI_RECV MPI_FINALIZE MPI Datatypes. MPI specifies datatypes explicitly in the message. Why? Because we may have a cluster of heterogeneous machines. Word sizes and data formats may differ. • • • • • MPI_INT MPI_FLOAT MPI_BYTE MPI_CHAR etc… Example Programs. • Simple “Hello world” in C. • “Hello World” with process ids. • A simple sum program. MPI Operations: Extended Set. • MPI_BCAST • MPI_REDUCE Buffered and Non-Blocking Communication. • • • • • • MPI_Bsend MPI_Brecv MPI_Isend MPI_Irecv MPI_Ssend MPI_Srecv Groups and Communicators. • A group is a collection of processes. The processes within a group are ordered by their ranks. • The notion of a group and the context are combined in a single object called communicator. Topologies. • Many problems conceptually “fit” best on particular topologies. Ex: the 2D Poisson equation is best solved on a 2D array of processes. • MPI provides a routines to create topologies, particularly Cartesian topologies • MPI_CART_CREATE • MPI_CART_COORDS MPICH tools. • MPICH provides, apart from an implementation of MPI, a set of tools for visualization and debugging. • MPE - Multiprocessing environment: set of tools which provide routines for Graphing, visualization, logging, etc. • Upshot: a viewer for MPE logs. For more information: • Using MPI: Portable Parallel Programming with the Message Passing Interface, Gropp, et. al. • www.mcs.anl.gov/mpi Exercises: 1. Could look at the code for calculating pi in the book and figure it out. 2. Could write a simple sum program which reads numbers from a file and adds them.