Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Galen Shipman (gshipman_at_[hidden])
Date: 2006-02-09 17:15:34


On Feb 9, 2006, at 3:03 PM, Jean-Christophe Hugly wrote:

> On Thu, 2006-02-09 at 14:05 -0700, Ron Brightwell wrote:
>>> [...]
>>>
>>>> From an adoption perspective, though, the ability to shine in
>>> micro-benchmarks is important, even if it means using an ad-hoc
>>> tuning.
>>> There is some justification for it after all. There are small
>>> clusters
>>> out there (many more than big ones, in fact) so taking maximum
>>> advantage
>>> of a small scale is relevant.
>>
>> I'm obliged to point out that you jumped to a conclusion -- possibly
>> true
>> in some cases, but not always.
>>
>> You assumed that a performance increase for a two-node micro-benchmark
>> would result in an application performance increase for a small
>> cluster.
>> Using RDMA for short messages is the default on small clusters
>> *because*
>> of the two-node micro-benchmark, not because the cluster is small.
>
> No, I assumed it based on comparisions between doing and not doing
> small
> msg rdma at various scales, from a paper Galen pointed out to me.
> http://www.cs.unm.edu/~treport/tr/05-10/Infiniband.pdf
>
Hmm, this is not what I would conclude from my results, in fact if you
look at the NPB results in my paper you will see that Open MPI
outperforms in the CG and FT benchmarks at both 32 and 64 nodes without
SRQ. The crossover point you are referring to must be the pairwise
ping-pong benchmark. So I would have to conclude that it is totally
application dependent.

- Galen

> Benchmarks are what they are. In the above paper, the tests place the
> cross-over at around 64 nodes and that confirms a number of anecdotal
> reports I got. It may well be that in some situations, small-msg rdma
> is
> better only for 2 nodes, but that's note such a likely scenario;
> reality
> is sometimes linear (at least at our scale :-) ) after all.
>
> The scale threshold could be tunable, couldnt it ?
>
> --
> Jean-Christophe Hugly <jice_at_[hidden]>
> PANTA
>