MIMD Architecture

Figure 1.6: Two MIMD Categories; Shared Memory and Message Passing MIMD Architectures.

• Multiple instruction multiple data streams (MIMD) parallel architectures are made of multiple processors and multiple memory modules connected together via some interconnection network.
• They fall into two broad categories: shared memory or message passing. Figure 1.6 illustrates the general architecture of these two categories.
• Processors exchange information through their central shared memory in shared memory systems, and exchange information through their interconnection network in message passing systems.
  1. A shared memory system typically accomplishes interprocessor coordination through a global memory shared by all processors.
     • The bus/cache architecture facilitates the need for expensive multi-ported memories and interface circuitry as well as the need to adopt a message-passing paradigm when developing application software.
     • Because access to shared memory is balanced, these systems are also called SMP (symmetric multiprocessor) systems.
     • Commercial examples of SMPs are Sequent Computer's Balance and Symmetry, Sun Microsystems multiprocessor servers, and Silicon Graphics Inc. multiprocessor servers.
  2. A message passing system (also referred to as distributed memory) typically combines the local memory and processor at each node of the interconnection network.
     • There is no global memory, so it is necessary to move data from one local memory to another by means of message passing. This is typically done by a Send/Receive pair of commands, which must be written into the application software by a programmer (data copying and dealing with consistency issues).
     • Commercial examples of message passing architectures were the nCUBE, iPSC/2, and various Transputer-based systems. These systems eventually gave way to Internet connected systems whereby the processor/memory nodes were either Internet servers or clients on individuals' desktop.
• It was also apparent that distributed memory is the only way efficiently to increase the number of processors managed by a parallel and distributed system. If scalability to larger and larger systems (as measured by the number of processors) was to continue, systems had to use distributed memory techniques.
• These two forces created a conflict: programming in the shared memory model was easier, and designing systems in the message passing model provided scalability.
• The distributed-shared memory (DSM) architecture began to appear in systems like the SGI Origin2000, and others. In such systems, memory is physically distributed; for example, the hardware architecture follows the message passing school of design, but the programming model follows the shared memory school of thought.
• As far as a programmer is concerned, the architecture looks and behaves like a shared memory machine, but a message passing architecture lives underneath the software. Thus, the DSM machine is a hybrid that takes advantage of both design schools.