Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Jean-Christophe Hugly (jice_at_[hidden])
Date: 2006-02-09 17:03:09


On Thu, 2006-02-09 at 14:05 -0700, Ron Brightwell wrote:
> > [...]
> >
> > >From an adoption perspective, though, the ability to shine in
> > micro-benchmarks is important, even if it means using an ad-hoc tuning.
> > There is some justification for it after all. There are small clusters
> > out there (many more than big ones, in fact) so taking maximum advantage
> > of a small scale is relevant.
>
> I'm obliged to point out that you jumped to a conclusion -- possibly true
> in some cases, but not always.
>
> You assumed that a performance increase for a two-node micro-benchmark
> would result in an application performance increase for a small cluster.
> Using RDMA for short messages is the default on small clusters *because*
> of the two-node micro-benchmark, not because the cluster is small.

No, I assumed it based on comparisions between doing and not doing small
msg rdma at various scales, from a paper Galen pointed out to me.
http://www.cs.unm.edu/~treport/tr/05-10/Infiniband.pdf

Benchmarks are what they are. In the above paper, the tests place the
cross-over at around 64 nodes and that confirms a number of anecdotal
reports I got. It may well be that in some situations, small-msg rdma is
better only for 2 nodes, but that's note such a likely scenario; reality
is sometimes linear (at least at our scale :-) ) after all.

The scale threshold could be tunable, couldnt it ?

-- 
Jean-Christophe Hugly <jice_at_[hidden]>
PANTA