Each of these three difficulty level's use cases are defined below.
Use Case A: Distributed Computing and Non-Determinism
Students first gain an understanding of both message passing and communication non-determinism. Specifically, students at a beginner level should be able to meet following goals once they have completed:
Learning Goal A1: Introduce Parallelism using Message Passing Paradigm
In order to satisfy the Goal, The students must understand two key concepts
- Any process in an exectuion can send a message to another process.
- Processes can exchange message using different communication patterns.
Observe the scenarios in each figure
Fig 1. Event graph visualization of AMG 2013 communication pattern on 2 MPI processes. Each row corresponds to a different MPI rank (process). Green circles correspond to the start or end of a process; blue circles correspond to sending a message; and red circles correspond to receiving a message.
Fig 2. Event graph visualization of a message race communication pattern on four MPI processes. Each row corresponds to a different MPI rank (process). Green circles correspond to the start or end of a process; blue circles correspond to sending a message; and red circles correspond to receiving a message.
Questions
- Task 1: Describe the scenarios in each figure.
- Task 2: What strategies would you employ to effectively reproduce the scenarios in the given figures and then broaden the range and quantity of scenarios using ANACIN-X with varying numbers of processes? How would you go about selecting and implementing the diverse communication patterns supported by ANACIN-X in this context?
Solution
A1: Figure 2, has three MPI processes, each sending a message to the fourth process. This is an event graph visualization of a message race communication pattern. Figure 3, on the other hand, has two MPI processes each send a message to the other. Afterward, each process sends a message back asyn- chronously. This is a small-scale replication of the AMG 2013 communication pattern
A2: Solution to TASK A2
Learning Goal A2: Define non-determinism associated to message passing
In order to satisfy the Goal, students must be able to understand that non-determinism is defined by inconsistent type and order of messages exchanged between MPI processes across runs of the same application execution.
Observe the scenarios in each figure
Two event graph visualizations run on the same configuration of the message race communication pattern. Different graphs correspond to two different non-deterministic executions. Each row corresponds to a different MPI rank (process). Green circles correspond to the start or end of a process; blue circles correspond to sending a message; and red circles correspond to receiving a message.
Note
Both Figures a and b are event graphs. They both correspond to the message race communication pattern. Both figures are generated using the same code with the same inputs but are the result of two independent runs.
Questions
- Task 1: Describe the scenarios in each figure.
- Task 2: Using ANACIN-X, generate new scenarios as shown in Figures.
Solution
Task 1: Despite being executed using the same code and the same inputs, the runs from Figure a and Figure b have different communication patterns. Specifically, the messages sent by Process 1 and Process 2 do not arrive at Process 0 in the same order.
Students use this example and other examples generated by ANACIN-X to understand that non-determinism is when multiple executions of the same code, run in the same way, produce different communication patterns. The differences in communication patterns are possible because the ANACIN-X environment is set to mimic delays in individual messages due to network and I/O congestion, or CPU contention, for example. Students can define an average degree of delay by setting the environment’s percentage of non-determinism. Figure a and Figure b were generated by running ANACIN-X with four processes and 100% non-determinism. Tests to reproduce scenarios such as those in Figures a and b should be run across multiple compute nodes to increase the likelihood that runs are non-determinismTask 2: Solution to TASK 2
#
