Collective Communication and Computation Operations

• MPI provides an extensive set of functions for performing many commonly used collective communication operations. All of the collective communication functions provided by MPI take as an argument a communicator that defines the group of processes that participate in the collective operation.
• All the processes that belong to this communicator participate in the operation, and all of them must call the collective communication function.
• Even though collective communication operations do not act like barriers, these act like a virtual synchronization step in the following sense: the parallel program should be written such that it behaves correctly even if a global synchronization is performed before and after the collective call.
  • Barrier; the barrier synchronization operation is performed in MPI using the MPI_Barrier function.

    int MPI_Barrier(MPI_Comm comm)
    

    The only argument of MPI_Barrier is the communicator that defines the group of processes that are synchronized. The call to MPI_Barrier returns only after all the processes in the group have called this function.
  • Broadcast; the one-to-all broadcast operation is performed in MPI using the MPI_Bcast function.

    int MPI_Bcast(void *buf, int count, MPI_Datatype datatype, 
            int source, MPI_Comm comm)
    

    MPI_Bcast sends the data stored in the buffer buf of process source to all the other processes in the group. The data received by each process is stored in the buffer buf. The data that is broadcast consist of count entries of type datatype.
• Since the operations are virtually synchronous, they do not require tags.
• In some of the collective functions data is required to be sent from a single process (source-process) or to be received by a single process (target-process). In these functions, the source- or target-process is one of the arguments supplied to the routines. All the processes in the group (i.e., communicator) must specify the same source- or target-process.