How to Submit
Week 16-19Establishing Asynchronous Random Communication (ARC)
To solve the types of problems we've been talking about we need a way for every machine in the cluster to communicate with any other machine at any time.
You will write a few small programs, culminating in a medium size program that proves you can robustly handle ARC.
Read: Paul's write-up of the problem so you can get the final goal in your head.
In order to write the final program, you'll probably want to (read: have to) write some small proof-of-concept programs which you can then combine. Here is my suggested list of programs. The first one is required (like, you need to turn it in), the others are not required but sugggested.
ex 1. Proof of Concept: MPI_Test
The purpose of MPI_Test is to allow a program to do some work while waiting for a message to arrive at a random time. Write a program in which two processors communicate with each other at random times.
Each processor should post a recv (iRecv) to the other processor before doing "work."
Each processor's "work" will simply be repeatedly printing a message to the console every second or two. (see: man 3 sleep).
After each step of "work" the proc should do an MPI_Test to see if it the recv it posted has something (check the flag paramter). If so, the processor should immediately post another iRecv to recieve the second chunk of data.
When the recv is complete, the proc should print a message to the console indicating what it received.
At random intervals, (see: man 3 random) the processor should send (iSend) two messages, back-to-back, to the other processor. The first message should be an int containing its proc number. The second should be an int containing the number of times its "work" has looped so far. (i.o.w. just two ints).
Each proc should not loop indefinitely. It should loop a fixed number of times (say 20, for starters). For now, just put an MPI_Barrier after the loop. When all procs reach the barrier the program should finish.
I would expect the output of this program to look something like:
proc 0: 1
proc 1: 1
proc 1: 2
proc 1: 3
proc 0: 2
proc 0: rev'd '3' from proc 1
proc 0: 3
proc 0: 4
proc 1: 4
proc 1: rec'd '4' from proc 0
proc 0: 20
proc 1: 20
proc 1: DONE
proc 0: DONE
Modify ex1 so that the second message that gets sent is a fixed-length string. And have each proc print out the string it recv'd.
Modify ex2 so that the first message sent contains an int that is the length of the string that follows. Allocate a buffer to recv the string of that size and recv it.
You can still send a fixed-length string at this stage, but the code on the recving end will be more dynamic.
Modify ex3 so that you're actually pulling strings to send from a file. Modify the output so that each proc prints the strings one at a time from the file, still sleeping between each print, and have it send random ones to the other proc. Each proc should read a different file.
Files: MacBeth.txt Hamlet.txt
Take out the sleep and see what happens :)
Modify ex5 so that you maintain a list of all the strings recv'd from the other processor (note: hard). Have each processor only print something to the console when it recv's from the other (as a check) and at the end have each proc print out the list of strings it recv'd from the other.
Now instead of sending random strings at random times to the other proc, do this: proc 0 should send any word starting with the letter A/a to proc 1, and proc 1, should send any word starting with B/b to proc 0. The procs should ignore any letters that start with other words.
At the same time, if proc0 encounters a word that starts with B it should store it in the list of B-words that it's recv'ing from proc1. Proc1 should do the same with A-words.
Thus at the end, proc0 will have (and print out) all the words from the file that start with B, and proc1 will have all the words that start with A.
For your reference here are lists of all the A and B words in MacBeth and Hamlet, both separate and combined.
Files: MacBeth_A.txt. MacBeth_B.txt Hamelet_A.txt Hamlet_B.txt MacBeth_Hamlet_A.txt MacBeth_Hamlet_B.txt
HINT: man uniq. The -i and -c flags can be helpful for checking your output.