Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] openmpi credits for eager messages
From: Brightwell, Ronald (rbbrigh_at_[hidden])
Date: 2008-02-04 15:29:50

> > I'm looking at a network where the number of endpoints is large enough that
> > everybody can't have a credit to start with, and the "offender" isn't any
> > single process, but rather a combination of processes doing N-to-1 where N
> > is sufficiently large. I can't just tell one process to slow down. I have
> > to tell them all to slow down and do it quickly...
> When you have N->1 patterns, then the hardware flow-control will
> throttle the senders, or drop packets if there is no hardware
> flow-control. If you don't have HOL blocking but the receiver does not
> consume for any reasons (busy, sleeping, dead, whatever), then you can
> still drop packets on the receiver (NIC, driver, thread) at a last
> resort, this is what TCP does. The key is have exponential backoff (or a
> reasonably large resend timeout) to no continue the hammering.
> It costs nothing in the common case (unlike the credits approach), but
> it does handle corner cases without affecting too much other nodes
> (unlike hardware flow-control).

Right. For a sufficiently large number of endpoints, flow control has to get
pushed out of MPI and down into the network, which is why I don't necesarily
want an MPI that does flow control at the user-level.

> But you know all that. You are just being mean to your users because you
> can :-) The sick part is that I think I envy you...

You know it :)