Principles of Message-Passing Programming

Set of Primitives: Allows processes to communicate with each other.

• A message passing architecture uses a set of primitives that allows processes to communicate with each other.
• i.e., send, receive, broadcast, and barrier.

There are two key attributes that characterize the message -passing programming paradigm.

1. the first is that it assumes a partitioned address space,
2. the second is that it supports only explicit parallelization.

• Each data element must belong to one of the partitions of the space;
  • hence, data must be explicitly partitioned and placed.
  • Adds complexity, encourages data locality.

• All interactions (read-only or read/write) require cooperation of two processes:
  1. the process that has the data,
  2. the process that wants to access the data.
• Primary advantage of explicit two-way interactions is that the programmer is fully aware of all the costs of non-local interactions
• The programmer is responsible for analyzing the underlying serial algorithm/application.
• As a result, programming using the message-passing paradigm tends to be hard and intellectually demanding.
• However, on the other hand, properly written message-passing programs can often achieve very high performance and scale to a very large number of processes.