Introduction I
Dr. Cem Özdo
˘
gan
LOGIK
Introduction
Four Decades of Computing
Flynn’s Taxonomy of
Computer Architecture
Parallel and Distributed
Computers
2.1
Lecture 2
Introductio n I
View of the Field
IKC-MH.57 Introduction to High Performance and Parallel
Computing at October 20, 2023
Dr. Cem Özdo
˘
gan
Engineering Sciences Department
˙
Izmir Kâtip Çelebi University
Introduction I
Dr. Cem Özdo
˘
gan
LOGIK
Introduction
Four Decades of Computing
Flynn’s Taxonomy of
Computer Architecture
Parallel and Distributed
Computers
2.3
Field I
•
Data-intensive applications;
•
transaction processing,
•
informat ion retrieval,
•
data mining and analysis,
•
multimedia services,
•
computati onal physics/chemistry/biology and
nanotechnology.
•
High performance may come from
•
fast dense cir cuitry,
•
parallelism.
•
Parallel processors are computer systems consisting of
•
multiple processing units
•
connected via some interconnection network
•
plus the software needed to make the processing uni ts work
together.
Introduction I
Dr. Cem Özdo
˘
gan
LOGIK
Introduction
Four Decades of Computing
Flynn’s Taxonomy of
Computer Architecture
Parallel and Distributed
Computers
2.4
Field II
•
Uniprocessor – Single processor supercomputers have
achieved great speeds and have been pushing hardware
technology to the physical limit of chip manufacturing.
•
Physical and architectural bounds (Lithography, µm size,
destructive quant um effects.
•
Proposed solut ions are maskless lithography proce ss and
nanoimprint li thography fo r t he semiconductor).
•
Uniprocessor systems can ach ieve to a limited
computati onal power and not capable of delivering solutions
to some problems in reasonable time.
•
Multiprocessor – Multiple processors cooperate to jointly
execute a single computational task in order to speed up
its execution.
Figure: Abstraction Layers
Introduction I
Dr. Cem Özdo
˘
gan
LOGIK
Introduction
Four Decades of Computing
Flynn’s Taxonomy of
Computer Architecture
Parallel and Distributed
Computers
2.5
Field III
Figure: View of the Fiel d
•
New issues arise;
•
Multipl e threads of control vs. single thread of control
•
Partitioning for concurrent execution
•
Task Scheduling
•
Synchronizat ion
•
Performa nce
Introduction I
Dr. Cem Özdo
˘
gan
LOGIK
Introduction
Four Decades of Computing
Flynn’s Taxonomy of
Computer Architecture
Parallel and Distributed
Computers
2.6
Trends
•
Past Trends in Parallel Architecture (inside the box)
•
Completely custom designed components; processors,
memory, interconnects, I/O.
•
The first three are the major components for the aspects of
the parall el computation.
•
Longer R&D time (2-3 years).
•
Expensive systems.
•
Quickly becoming outdate d .
•
In the form of internally linked processors was the main
form of parallelism.
•
Advances in computer networks ⇒ in the form o f networked
autonomous computers.
•
New Trends in Parallel Architecture (outside the box)
•
Instead of put ting eve rything in a single box and tightly
couple processors to memory, the Internet achieved a kind
of paralle lism by loosely connecting everything outside of
the box.
•
Network of PCs and workstations connected via LAN or
WAN forms a Parallel System.
•
Compete favourably (cost/performance).
•
Utilize unused cycles of systems sitting idle.
Introduction I
Dr. Cem Özdo
˘
gan
LOGIK
Introduction
Four Decades of Computing
Flynn’s Taxonomy of
Computer Architecture
Parallel and Distributed
Computers
2.7
Four Decades of Computing
Most computer scientists agree that there have been four
distinct paradigms or eras of computing. These are: batch,
time-sharing, desktop, and network.
1 Batch Era
2 Time-Sharing Era
3 Desktop Era
4 Network Era. They can generally be classified into two
main categories:
1 shared memory,
2 distributed memory systems.
•
The number of processors in a single machine ranged from
several in a shared memory computer
to hundreds of
thousan ds in a massively parallel system.
•
Examples of parallel computers during this era include
Sequent Symmetry, Intel iPSC, nCUBE, Intel Paragon,
Thinking Machines (CM-2, CM-5), MsPar (MP), Fujitsu
(VPP500) , and oth ers.
5 Current Trends: Clusters, Grids.
Introduction I
Dr. Cem Özdo
˘
gan
LOGIK
Introduction
Four Decades of Computing
Flynn’s Taxonomy of
Computer Architecture
Parallel and Distributed
Computers
2.8
Flynn’s Taxonomy of Computer Architecture I
•
The most popular taxonomy of computer architecture was
defined by Flynn in 1966.
•
Flynn’s classification scheme is based on the notion of a
stream of information
.
•
Two types of information flow into a processor:
1 Instruction. The instruction stream is defined as the
sequence of instructions performed by the processing unit.
2 Data. The data stream is defined as the data traffic
exchanged betwee n the memory and the processing unit.
•
According to Flynn’s classification, either of the instruction
or data streams can be single or multiple.
•
Computer architecture can be classified into the following
four distinct categories:
1 single instruction single data streams (SISD)
2 single instruction multiple data streams (SIMD)
3 multiple instr uction single data streams (MISD)
4 multiple instr uction multiple data streams (MIMD).
Introduction I
Dr. Cem Özdo
˘
gan
LOGIK
Introduction
Four Decades of Computing
Flynn’s Taxonomy of
Computer Architecture
Parallel and Distributed
Computers
2.10
Flynn’s Taxonomy of Computer Architecture III
Parallel computers are either SIMD or MIMD.
•
When there is only one control unit and all processors
execute the same instruction in a synchronized fashion,
the parallel machine is classified as SIMD.
•
In a MIMD machine, each processor has its own control
unit and can execute different instructions on different
data.
•
In the MISD category, the same stream of data flows
through a linear array of processors executing different
instruction streams. In practice, there is no viable MISD
machine; however, some authors have considered
pipelined machines as examples for MISD.
Introduction I
Dr. Cem Özdo
˘
gan
LOGIK
Introduction
Four Decades of Computing
Flynn’s Taxonomy of
Computer Architecture
Parallel and Distributed
Computers
2.11
Parallel and Dist ributed Computers I
•
The processing units can communicate and interact with
each other using either
•
shared memory
•
or message passing methods.
•
The interconnection network for shared memory systems
can be classified as
•
bus-based
•
switch-based.
•
SIMD Computers
•
MIMD Shared Memory, MIMD Distributed Memory
•
Bus based, Switch based
•
CC-NUMA
•
Clusters, Grid Computing
•
Grids are geographically distributed platforms for
computati on.
•
They provide dependable, consistent, general, and
inexpensive access to high end computational capabilities.
Introduction I
Dr. Cem Özdo
˘
gan
LOGIK
Introduction
Four Decades of Computing
Flynn’s Taxonomy of
Computer Architecture
Parallel and Distributed
Computers
2.12
Parallel and Dist ributed Computers II
Figure: (a) MIMD Shared Memory, (b) MIMD Distributed Memo ry.
Figure: (a) SIMD Distributed Computers, (b) Clu sters.
