What?
From a programmers perspective, briefly explain the differences between OpenMP and MPI. Explain in your own words why is MPI interesting.
Try to use the following
keywords: Pragma
, Cluster
, Threads
, Fork-Join
.
Suggested Answer
OpenMP uses pragma
annotations to distribute the workload over multiple threads
that communicate via the fork-join model in a shared memory
setting. MPI allows execution on larger cluster consisting
of multiple computers with disjunct memories, that communicate via
messages, but also require a rethinking of the program architecture.
Ranks?
Explain what a Rank
is in the MPI world. Try to relate the
notions of processes, threads and nodes?
Suggested Answer
A rank is a virtual identifier for a group of processes. Processes
may in turn may have multiple threads. Multiple ranks can live on one
or more nodes.
Reduction?
Explain why it is important for all processes to call the same
collective operation (e.g.,
MPI_Reduce). What
happens when one process does something else instead?
Suggested Answer
Collective operations requires input from all ranks to be completed.
If one process doesn't collaborate, the accumulating process
hangs in a futile attempt to await a response.
More Reduction?
Consider the following situation: Multiple ranks have computed some value, say the cost of an optimization problem, and now wish to determine both the minimal value and what node found said value.
Using pseudocode, sketch how you would design a program to solve this
problem. Don't worry about your solution being naive.
We use rank 0 to accumulate values using Bonus
Do worry about your solution being naive. How
can you improve the performance by reducing communication?
Suggested Answer
MPI_Reduce and
the MPI_MINLOC reduction scheme.
All other ranks compute their respective values and send the
result together with an identifier (e.g. the rank index) to rank 0 in
a datatype like MPI_2INT that can bundle the result of
the computation (assuming it is an integer value) and the index.
Primitives?
Show how
using MPI_Recv, MPI_Send, MPI_Comm_size
and MPI_Comm_rank
you implement your own version
of MPI_Scatter
in pseudocode?
Suggested Answer
PROCEDURE MPI_Scatter (sendbuf, sendcount, sendtype, recvbuf, recvcount, recvtype, root, comm)
BEGIN
size := MPI_Comm_Size (comm)
rank := MPI_Comm_Rank (comm)
IF rank = root
THEN
FOR i FROM 1 TO size
DO
IF i ≠ root
THEN
MPI_Send (sendbuf OFFSETBY i * sendcount, sendcount, sendtype, root, 0, comm)
END
END
ELSE
MPI_Recv (recvbuf, recvcount, recvtype, root, 0, comm, MPI STATUS IGNORE)
END
END