Chapter 3 Parallel Algorithm design

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Communication Checklist
Communication operations balanced among tasks Each task communicates with only small Cohesion group of neighbors Tasks can perform communications concurrently Task can perform computations concurrently

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Agglomeration Can Improve Performance
Eliminate communication between primitive tasks agglomerated into consolidated task Combine groups of sending and receiving tasks


The first two steps look for parallelism in the problem. However, the design obtained at this point probably doesn’t map well onto a real machine. If the number of tasks greatly exceed the number of processors, the overhead will be strongly affected by how the tasks are assigned to the processors. Now we have to decide what type of computer we are targeting
Task/Channel Model
Parallel computation = set of tasks Task Program Local memory Collection of I/O ports Tasks interact by sending messages througDividing computation and data into pieces Domain decomposition Divide data into pieces Determine how to associate computations with the data Functional decomposition Divide computation into pieces Determine how to associate data with the computations
Foster’s Design Methodology
Partitioning Communication Agglomeration Mapping

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Scalability issue

Assume we are manipulating a 3D matrix of size 8 x 128 x 256 and
17
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Agglomeration Checklist

Locality of parallel algorithm has increased Replicated computations take less time than communications they replace Data replication doesn’t affect scalability Agglomerated tasks have similar computational and communications costs Number of tasks increases with problem size Number of tasks suitable for likely target systems Tradeoff between agglomeration and code modifications costs is reasonable
Example Functional Decomposition
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Partitioning Checklist
At least 10x more primitive tasks than processors in target computer Minimize redundant computations and redundant data storage Primitive tasks roughly the same size Number of tasks an increasing function of problem size

Our target machine is a centralized multiprocessor with 4 CPUs.


Suppose we agglomerate the 2nd and 3rd dimensions. Can we run on our target machine? Suppose we change to a target machine that is a centralized multiprocessor with 8 CPUs. What if we go to more than 8 CPUs?
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Parallel Programming
in C with MPI and OpenMP Michael J. Quinn
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Agglomeration(凝聚？)
Grouping tasks into larger tasks Goals Improve performance Maintain scalability of program Simplify programming In MPI programming, goal often to create one agglomerated task per processor
Foster’s Methodology
Problem Partitioning Communication
Mapping
Agglomeration
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

What We Have Hopefully at This Point – and What We Don’t Have
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.
Chapter 3
Parallel Algorithm Design
Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.

Is it a centralized multiprocessor or a multicomputer? What communication paths are supported How must we combine tasks in order to map them effectively onto processors?
Outline
Task/channel model Algorithm design methodology Case studies

Copyright © The McGraw-Hill Companies, Inc. Permission required for reproduction or display.