That would make sense. I able to break OpenMPI by having Node A wait for
messages from Node B. Node B is in fact sleeping while Node C bombards
Node A with a few thousand messages. After a while Node B wakes up and
sends Node A the message it's been waiting on, but Node A has long since
been buried and seg faults. If I decrease the number of messages C is
sending, it works properly. This was on OpenMPI 1.2.4 (using I think the
SM BTL (might have been MX or TCP, but certainly not infiniband. I could
dig up the test and try again if anyone is seriously curious).
Trying the same test on MPICH/MX went very very slow (I don't think they
have any clever buffer management) but it didn't crash.
Sacerdoti, Federico Federico.Sacerdoti-at-deshaw.com
> I am readying an openmpi 1.2.5 software stack for use with a
> many-thousand core cluster. I have a question about sending small
> messages that I hope can be answered on this list.
> I was under the impression that if node A wants to send a small MPI
> message to node B, it must have a credit to do so. The credit assures A
> that B has enough buffer space to accept the message. Credits are
> required by the mpi layer regardless of the BTL transport layer used.
> I have been told by a Voltaire tech that this is not so, the credits are
> used by the infiniband transport layer to reliably send a message, and
> is not an openmpi feature.
> users mailing list
[A dream that comes true can't really be called a dream.]