Next:
Contents
Contents
Ceng 471 Parallel Computing Lecture Notes
Cem Özdogan
Date:
27 December 2010
Contents
List of Tables
List of Figures
First Meeting
Lecture Information
Overview
Text Book
Grading Criteria & Policies
Parallel Computing
MPI Hands-On; Performance Analysis
Analysis of Parallel Summation with Point-to-Point Communications
Introduction
Four Decades of Computing
Flynn's Taxonomy of Computer Architecture
Parallel and Distributed Computers
SIMD Architecture
MIMD Architecture
Shared Memory Organization
Message Passing Organization
MPI Hands-On - Introduction to MPI
Parallel Computing
Communicating with other processes
What is MPI?
MPI Implementations
Is MPI Large or Small?
Where to use MPI?
How To Use MPI? Essential!!
Getting started
Writing MPI programs I
Writing MPI programs II
Writing MPI programs III
Exercise - Getting Started
Performance Analysis
Computational Models
Equal Duration Model
Parallel Computation with Serial Sections Model
Skeptic Postulates For Parallel Architectures
Grosch's Law
Amdahl's Law
Gustafson-Barsis's Law
MPI Hands-On - Sending and Receiving Messages I
Current Message-Passing
The Buffer
MPI Basic Send/Receive
Exercises/Examples
Programming Using the Message-Passing Paradigm
Principles of Message-Passing Programming
Structure of Message-Passing Programs
The Building Blocks: Send and Receive Operations
Blocking Message Passing Operations
Non-Blocking Message Passing Operations
MPI Hands-On; Sending and Receiving Messages II
MPI: the Message Passing Interface
Starting and Terminating the MPI Library
Communicators
Getting Information
Sending and Receiving Messages
Avoiding Deadlocks
Sending and Receiving Messages Simultaneously
MPI Hands-On; Sending and Receiving Messages III
Overlapping Communication with Computation
Non-Blocking Communication Operations
Collective Communication and Computation Operations
Broadcast
Reduction
Gather
Scatter
All-to-All
Groups and Communicators
MPI Hands-On; Collective Communications I
Programming Shared Memory
What is a Thread?
Threads Model
Why Threads?
Thread Basics: Creation and Termination
Thread Creation
Thread Termination
MPI Hands-On; Collective Communications II
Thread Basics: Passing Arguments, Cancellation and Joining
Passing Arguments to Threads
Thread Cancellation
Joining and Detaching Threads
Synchronization Primitives in Pthreads
Mutual Exclusion for Shared Variables
Hands-on; Shared Memory I; Threads
Thread Examples
Computing the value of
Producer-consumer work queues
Condition Variables for Synchronization
Hands-on; Shared Memory II; Synchronization Primitives
Controlling Thread Attributes and Synchronization
Attributes Objects for Threads
Composite Synchronization Constructs
Tips for Designing Asynchronous Programs
OpenMP: a Standard for Directive Based Parallel Programming
The OpenMP Programming Model
Hands-on; Shared Memory III; MPI with Pthread, OpenMP
Network Computing
Computer Networks Basics
Network Performance
Other Network Technologies
Client/Server Systems
Sockets
A Client Server Framework for Parallel Applications
Clusters
Cluster Examples
Hands-on; Miscellaneous I
Interconnection Networks
Ethernet
Switches
Myrinet Clos Network
The Quadrics Network
Grid Computing
Hands-on; Miscellaneous II
References:
vita
About this document ...
Cem Ozdogan 2010-12-27
