Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] scaling problem with openmpi
From: Pavel Shamis (Pasha) (pashash_at_[hidden])
Date: 2009-05-20 13:18:50


Tomorrow I will add some printf to collective code and check what really
happens there...

Pasha

Peter Kjellstrom wrote:
> On Wednesday 20 May 2009, Pavel Shamis (Pasha) wrote:
>
>>> Disabling basic_linear seems like a good idea but your config file sets
>>> the cut-off at 128 Bytes for 64-ranks (the field you set to 8192 seems to
>>> result in a message size of that value divided by the number of ranks).
>>>
>>> In my testing bruck seems to win clearly (at least for 64 ranks on my IB)
>>> up to 2048. Hence, the following line may be better:
>>>
>>> 131072 2 0 0 # switch to pair wise for size 128K/nranks
>>>
>>> Disclaimer: I guess this could differ quite a bit for nranks!=64 and
>>> different btls.
>>>
>> Sounds strange for me. From the code is looks that we take the threshold as
>> is without dividing by number of ranks.
>>
>
> Interesting, I may have had to little or too much coffe but the figures in my
> previous e-mail (3rd run, bruckto2k_pair) was run with the above line. And it
> very much looks like it switched at 128K/64=2K, not at 128K (which would have
> been above my largest size of 3000 and as such equiv. to all_bruck).
>
> I also ran tests with:
> 8192 2 0 0 # ...
> And it seemed to switch between 10 Bytes and 500 Bytes (most likely then at
> 8192/64=128).
>
> My testprogram calls MPI_Alltoall like this:
> time1 = MPI_Wtime();
> for (i = 0; i < repetitions; i++) {
> MPI_Alltoall(sbuf, message_size, MPI_CHAR,
> rbuf, message_size, MPI_CHAR, MPI_COMM_WORLD);
> }
> time2 = MPI_Wtime();
>
> /Peter
>
> ------------------------------------------------------------------------
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users