When running my Open MPI application, I'm seeing three processors that
are using five times as much memory as the others when they should all
use the same amount of memory. To start the debugging process, I would
like to know if it's my application or the Open MPI library that's
using the additional memory. Does anyone have any tips on calculating
the amount of memory that Open MPI is using at a given moment in time?
My Open MPI application makes significant use of MPI_Irecv and
MPI_Send. Every process has exactly one MPI_Irecv request active at
any time. When it receives a message, it handles it, possibly
transmits a response packet using MPI_Send, and starts a new MPI_Irecv
request. What happens if one process is slow and lags behind? Will the
messages be buffered at the sender or the receiver? Will the messages
be buffered at the Open MPI level or at the OS level, say in a TCP
transmit buffer or receive buffer?