Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] openmpi credits for eager messages
From: Ron Brightwell (rbbrigh_at_[hidden])
Date: 2008-02-04 12:24:11

> Is what George says accurate? If so, it sounds to me like OpenMPI
> does not comply with the MPI standard on the behavior of eager
> protocol. MPICH is getting dinged in this discussion because they
> have complied with the requirements of the MPI standard. IBM MPI
> also complies with the standard.
> If there is any debate about whether the MPI standard does (or
> should) require the behavior I describe below then we should move
> the discussion to the MPI 2.1 Forum and get a clarification.
> [...]

The MPI Standard also says the following about resource limitations:

  Any pending communication operation consumes system resources that are
  limited. Errors may occur when lack of resources prevent the execution
  of an MPI call. A quality implementation will use a (small) fixed amount
  of resources for each pending send in the ready or synchronous mode and
  for each pending receive. However, buffer space may be consumed to store
  messages sent in standard mode, and must be consumed to store messages
  sent in buffered mode, when no matching receive is available. The amount
  of space available for buffering will be much smaller than program data
  memory on many systems. Then, it will be easy to write programs that
  overrun available buffer space.

Since I work on MPI implementations that are expected to allow applications
to scale to tens of thousands of processes, I don't want the overhead of
a user-level flow control protocol that penalizes scalable applications in
favor of non-scalable ones. I also don't want an MPI implementation that
hides such non-scalable application behavior, but rather exposes it at lower
processor counts -- preferably in a way that makes the application developer
aware of the resources requirements of their code and allows them to make
the appropriate choice regarding the structure of their code, the underlying
protocols, and the amount of buffer resources.

But I work in a place where codes are expected to scale and not just work.
Most of the vendors aren't allowed to have this perspective....