MPI Testing

Before attempting to run a tomography inversion or a batch picker it’s a good idea to confirm that your MPI setup works as expected.

In the upper-right corner of Phoenix select “MPI” followed by “Test MPI”:

This will open the MPI job preparation dialog. There are a lot of options on this dialog, but once they are set properly you shouldn’t have to change them again.

Compute node selection

The first set of options determine on which compute nodes the job will be run. For now just select the option “Run on local machine”.

Additional library paths

You may need to add paths here if the script fails to run – you’ll see the message “mpirun: error while loading shared libraries” in the printout.

Missing libraries can be found using a linux command similar to “sudo find / -name ‘libname*’”.

The paths should be separated by a “:”.

Phoenix installation path

This is the path that contains the “FRG_MPI” executable and associated libraries. It should be set correctly the first time the dialog is opened.

Shell script file

By default this is placed in the installation path.

Script output file name

Redirects the output of the script to a file.

Run the test

Once the above options are set click “OK”. If the job starts and runs correctly you shoud see the following in the Phoenix dialog.

If this doesn’t appear then check the script output. There are two possibilities – either the job started and ran correctly and the IPC communication method failed, or the job failed to run.

If the job ran correctly then make sure port 45454 is opened. This should permit the host machine to receive UDP communications.

If the job failed view the output script to determine why. The most likely culprit will be missing libraries.

Next test – Use a hostfile

Open the “Test MPI” again, select “Read hosts from file”, and then select the host file:

Note that the contents of the automatically generated script file lists the selected compute nodes:

Click “OK”. You should see a new entry in the Phoenix dialog: