Sending and Receiving Messages

The basic functions for sending and receiving messages in MPI are the MPI_Send and MPI_Recv, respectively. The calling sequences of these routines are as follows:

int MPI_Send(void *buf, int count, 
             MPI_Datatype datatype,
             int dest, int tag, 
             MPI_Comm comm)
int MPI_Recv(void *buf, int count, 
             MPI_Datatype datatype,
             int source, int tag, 
             MPI_Comm comm, 
             MPI_Status *status)

1	MPI_Send sends the data stored in the buffer pointed by buf. This buffer consists of consecutive entries of the type specified by the parameter datatype. The number of entries in the buffer is given by the parameter count.

[]1blue!20blue!10

**Table 4.2:** Correspondence between the datatypes supported by MPI and those supported by C.
MPI Datatype	C Datatype
MPI_CHAR	signed char
MPI_SHORT	signed short int
MPI_INT	signed int
MPI_LONG	signed long int
MPI_UNSIGNED_CHAR	unsigned char
MPI_UNSIGNED_SHORT	unsigned short int
MPI_UNSIGNED	unsigned int
MPI_UNSIGNED_LONG	unsigned long int
MPI_FLOAT	float
MPI_DOUBLE	double
MPI_LONG_DOUBLE	long double
MPI_BYTE
MPI_PACKED

Note that for all C datatypes, an equivalent MPI datatype is provided.

MPI allows two additional datatypes that are not part of the C language.
These are MPI_BYTE and MPI_PACKED.
- MPI_BYTE corresponds to a byte (8 bits)
- MPI_PACKED corresponds to a collection of data items that has been created by packing non-contiguous data.
Note that the length of the message in MPI_Send, as well as in other MPI routines, is specified in terms of the number of entries being sent and not in terms of the number of bytes.
The destination of the message sent by MPI_Send is uniquely specified by
- dest argument. This argument is the rank of the destination process in the communication domain specified by the communicator comm.
- comm argument.
Each message has an integer-valued tag associated with it.
This is used to distinguish different types of messages.

MPI_Recv receives a message sent by a process whose rank is given by the source in the communication domain specified by the comm argument. The tag of the sent message must be that specified by the tag argument. If there are many messages with identical tag from the same process, then any one of these messages is received. MPI allows specification of wild card arguments for both source and tag.

If source is set to MPI_ANY_SOURCE, then any process of the communication domain can be the source of the message.
Similarly, if tag is set to MPI_ANY_TAG, then messages with any tag are accepted.

The received message is stored in continuous locations in the buffer pointed to by buf. The count and datatype arguments of MPI_Recv are used to specify the length of the supplied buffer.

The received message should be of length equal to or less than this length.
If the received message is larger than the supplied buffer, then an overflow error will occur, and the routine will return the error MPI_ERR_TRUNCATE.
After a message has been received, the status variable can be used to get information about the MPI_Recv operation.
In C, status is stored using the MPI_Status data-structure.

This is implemented as a structure with three fields, as follows:

typedef struct MPI_Status { 
  int MPI_SOURCE; 
  int MPI_TAG; 
  int MPI_ERROR; 
};

MPI_SOURCE and MPI_TAG store the source and the tag of the received message.

They are particularly useful when MPI_ANY_SOURCE and MPI_ANY_TAG are used for the source and tag arguments.
MPI_ERROR stores the error-code of the received message.
The status argument also returns information about the length of the received message.
This information is not directly accessible from the status variable, but it can be retrieved by calling the MPI_Get_count function.

The calling sequence:

int MPI_Get_count(MPI_Status *status, 
                  MPI_Datatype datatype, 
                  int *count)

MPI_Get_count takes as arguments the status returned by MPI_Recv and the type of the received data in datatype, and returns the number of entries that were actually received in the count variable.

The MPI_Recv returns only after the requested message has been received and copied into the buffer. That is, MPI_Recv is a blocking receive operation. However, MPI allows two different implementations for MPI_Send.

MPI_Send returns only after the corresponding MPI_Recv have been issued and the message has been sent to the receiver.

MPI_Send first copies the message into a buffer and then returns, without waiting for the corresponding MPI_Recv to be executed. MPI programs must be able to run correctly regardless of which of the two methods is used for implementing MPI_Send. Such programs are called safe. In writing safe MPI programs, sometimes it is helpful to forget about the alternate implementation of MPI_Send and just think of it as being a blocking send operation.