Ordering Output in MPI

by

You guessed right. The MPI standard does not specify how stdout from different nodes should be collected for printing at the originating process. It is often the case that when multiple processes are doing prints the output will get merged in an unspecified way. fflush doesn’t help.

If you want the output ordered in a certain way, the most portable method would be to send the data to the master process for printing.

For example, in pseudocode:

if (rank == 0) {
    print_col(0);
    for (i = 1; i < comm_size; i++) {
        MPI_Recv(buffer, .... i, ...);
        print_col(i);
    }
} else {
    MPI_Send(data, ..., 0, ...);
}

Another method which can sometimes work would be to use barries to lock step processes so that each process prints in turn. This of course depends on the MPI Implementation and how it handles stdout.

for(i = 0; i < comm_size; i++) {
    MPI_Barrier(MPI_COMM_WORLD);
    if (i == rank) {
         printf(...);
    }
}

Of course, in production code where the data is too large to print sensibly anyway, data is eventually combine by having each process writing to a separate file and merged separately, or using MPI I/O (defined in the MPI2 standards) to coordinate parallel writes.

More Related Contents:

  1. How to make output of any shell command unbuffered?
  2. Is stdout line buffered, unbuffered or indeterminate by default?
  3. How to redirect and append both standard output and standard error to a file with Bash
  4. How can I pipe stderr, and not stdout?
  5. multiple prints on the same line in Python
  6. Order of evaluation of arguments using std::cout
  7. Capture stdout from a script?
  8. Capturing stdout from a system() command optimally [duplicate]
  9. log4j redirect stdout to DailyRollingFileAppender
  10. Sending and receiving 2D array over MPI
  11. Multiple inputs and outputs in python subprocess communicate
  12. redirect stdout/stderr to a string
  13. MPI_Bcast a dynamic 2d array
  14. Is there a way to set a variable up to place output to stdout or null?
  15. Running powershell script within python script, how to make python print the powershell output while it is running
  16. What is the simplest way to write to stdout in binary mode?
  17. Get output from a Paramiko SSH exec_command continuously
  18. Creating a counter that stays synchronized across MPI processes
  19. Rewinding std::cout to go back to the beginning of a line
  20. How can a process intercept stdout and stderr of another process on Linux?
  21. “subprocess.Popen” – checking for success and errors
  22. How do I redirect stderr and stdout to file for a Ruby script?
  23. C restore stdout to terminal
  24. Probe seems to consume the CPU
  25. Redirect stdin and stdout in Java
  26. Is there a guarantee of stdout auto-flush before exit? How does it work?
  27. Why do System.err statements get printed first sometimes?
  28. MPI hangs on MPI_Send for large messages
  29. What is the best way to redirect stdout to NSTextView in Cocoa?
  30. Update Command-line Output, i.e. for Progress

Leave a Comment