Next:
Contents
Contents
Ceng 505 Parallel Computing
Cem Özdogan
Date:
27 December 2006
Contents
List of Tables
List of Figures
Introduction
Four Decades of Computing
Flynn's Taxonomy of Computer Architecture
SIMD Architecture
MIMD Architecture
Shared Memory Organization
Message Passing Organization
Interconnection Networks
Introduction to MPI
Parallel Computing
Types of parallel computing
Communicating with other processes
Hardware models
What is MPI?
Who Designed MPI?
MPI Implementations
Is MPI Large or Small?
Where to use MPI?
How To Use MPI?
Getting started
Writing MPI programs
Compiling and linking
Running MPI programs
Finding out about the environment
A simple program
Exercise - Getting Started
Multiprocessors Interconnection Networks
Interconnection Networks Taxonomy
Bus-based Dynamic Interconnection Networks
Single Bus Systems
Multiple Bus Systems
Bus Synchronization
Switch-Based Interconnection Networks
Crossbar Networks
Single-Stage Networks
Multistage Networks
Blockage in Multistage Interconnection Networks
Static Interconnection Networks
Completely Connected Networks
Limited Connection Networks
Analysis and Performance Metrics
Dynamic Networks
Static Networks
Sending and Receiving messages
Current Message-Passing
The Buffer
Sample Program using Library Calls
MPI Basic Send/Receive
Exercise/Example: Translate this Fortran http://siber.cankaya.edu.tr/ParallelComputing/cfiles/sendreceive.fcode to C
Computation of PI as an example
Exercise
MPI Hands-On; Examples
Performance Analysis
Computational Models
Equal Duration Model
Parallel Computation with Serial Sections Model
Skeptic Postulates For Parallel Architectures
Grosch's Law
Amdahl's Law
Gustafson-Barsis's Law
Scalability of Parallel Architectures
Assignment:
Performance tuning MPI 1
Shared Memory Architecture
Classification of Shared Memory Systems
Uniform Memory Access (UMA)
Nonuniform Memory Access (NUMA)
Cache-Only Memory Architecture (COMA)
Bus-based Symmetric Multiprocessors
Basic Cache Coherency Methods
Cache-Memory Coherence
Cache-Cache Coherence
Shared Memory System Coherence
Shared Memory Programming
Task Creation
Communication
Synchronization
Thread Basics
Performance tuning MPI 2
Programming Shared Memory
Why Threads?
Thread Basics: Creation and Termination
Synchronization Primitives in Pthreads
Mutual Exclusion for Shared Variables
Shared Memory I; Processes, Threads
Condition Variables for Synchronization
Controlling Thread and Synchronization Attributes
Attributes Objects for Threads
Thread Cancellation
Composite Synchronization Constructs
Barriers
Tips for Designing Asynchronous Programs
OpenMP: a Standard for Directive Based Parallel Programming
The OpenMP Programming Model
Assignment:
Shared Memory II; Threads, OpenMP Examples
Message Passing Architecture
Introduction to Message Passing
Routing in Message Passing Networks
Routing for Broadcasting and Multicasting
Routing Potential Problems
Switching Mechanisms in Message Passing
Processor Support for Message Passing
Example Message Passing Architectures
Message Passing vs Shared Memory
Parallel Summation
Programming Using the Message-Passing Paradigm
Principles of Message-Passing Programming
Structure of Message-Passing Programs
The Building Blocks: Send and Receive Operations
Blocking Message Passing Operations
Non-Blocking Message Passing Operations
MPI: the Message Passing Interface
Starting and Terminating the MPI Library
Communicators
Getting Information
Sending and Receiving Messages
Examples; Buffer Limit, Barrier, Collective Communications.
Overlapping Communication with Computation
Non-Blocking Communication Operations
Collective Communication and Computation Operations
Reduction
Gather
Scatter
All-to-All
Groups and Communicators
Examples; Non-blocking Communications and Parallel Matrix Multiplication.
Introduction to Parallel Algorithms
Odd-Even Sort
Cannon's Matrix-Matrix Multiplication with MPI's Topologies
One-Dimensional Matrix-Vector Multiplication
Two-Dimensional Matrix-Vector Multiplication
Examples; Parallel Algorithms
Network Computing
Computer Networks Basics
Network Performance
Internet
Other Network Technologies
Client/Server Systems
Sockets
Remote Procedure Call (RPC)
Middleware
A Client Server Framework for Parallel Applications
Clusters
Threads
Interconnection Networks
Ethernet
Switches
Myrinet Clos Network
The Quadrics Network
Cluster Examples
Berkeley Network of Workstations (NOW)
The Beowulf Cluster
FlashMob I
Grid Computing
References:
vita
About this document ...
Cem Ozdogan 2006-12-27
