Thanks for the explanation. I am using GigEth + Open MPI and the
buffered MPI_BSend. I had already noticed that top behaved differently
on another cluster with Infinibandb + MPICH.
So the only option to find out how much time each process is waiting
around seems to be to profile the code. Will gprof show me anything
useful or will I have to use a more sophisticated (any free ones?)