Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Carsten Kutzner (ckutzne_at_[hidden])
Date: 2006-01-04 08:26:00


Hi Graham,

here are the all-to-all test results with the modification to the decision
routine you suggested yesterday. Now the routine behaves nicely for 128
and 256 float messages on 128 CPUs! For the other sizes one probably wants
to keep the original algorithm, since it is faster there. However I have
the feeling that for messages >= 4096 floats there still exists the old
problem since the execution times are so variable there (note that the
standard deviation rises by more than a factor of 10 when going from 2048
to 4096 floats.) If you need additional test results to tune the decision
functions please let me know.

Carsten

OMPI tuned all-to-all with modification:
========================================
       mesg size time in seconds
#CPUs floats average std.dev. min. max.
 128 1 0.001253 0.000074 0.001141 0.001470
 128 2 0.023507 0.000563 0.022562 0.024761
 128 4 0.023435 0.000426 0.022582 0.024166
 128 8 0.023438 0.000359 0.022904 0.024104
 128 16 0.023664 0.000438 0.022844 0.024670
 128 32 0.024136 0.000463 0.023297 0.025117
 128 64 0.024704 0.000535 0.023727 0.026030
 128 128 0.025750 0.000525 0.024592 0.026799 *
 128 256 0.028862 0.000683 0.027389 0.030168 *
 128 512 0.035869 0.001214 0.034067 0.038655
 128 1024 0.046528 0.001722 0.043549 0.050432
 128 2048 0.072388 0.007032 0.066708 0.104358
 128 4096 0.217678 0.097312 0.135113 0.409431
 128 8192 0.378586 0.090267 0.297878 0.577771
 128 16384 0.567473 0.105083 0.483573 0.735509
 128 32768 1.151343 0.146547 0.937150 1.404478
 128 65536 2.298998 0.169669 1.983286 2.572027
 128 131072 4.070989 0.159958 3.691039 4.373587

> > OMPI tuned all-to-all:
> > ======================
> > mesg size time in seconds
> > #CPUs floats average std.dev. min. max.
> > 128 1 0.001288 0.000102 0.001077 0.001512
> > 128 2 0.008391 0.000400 0.007861 0.009958
> > 128 4 0.008403 0.000237 0.008095 0.009018
> > 128 8 0.008228 0.000942 0.003801 0.008810
> > 128 16 0.008503 0.000191 0.008233 0.008839
> > 128 32 0.008656 0.000271 0.008084 0.009177
> > 128 64 0.009085 0.000209 0.008757 0.009603
> > 128 128 0.251414 0.073069 0.011547 0.506703 !
> > 128 256 0.385515 0.127661 0.251431 0.578955 !
> > 128 512 0.035111 0.000872 0.033358 0.036262
> > 128 1024 0.046028 0.002116 0.043381 0.052602
> > 128 2048 0.073392 0.007745 0.066432 0.104531
> > 128 4096 0.165052 0.072889 0.124589 0.404213
> > 128 8192 0.341377 0.041815 0.309457 0.530409
> > 128 16384 0.507200 0.050872 0.492307 0.750956
> > 128 32768 1.050291 0.132867 0.954496 1.344978
> > 128 65536 2.213977 0.154987 1.962907 2.492560
> > 128 131072 4.026107 0.147103 3.800191 4.336205
> >
> > alternative all-to-all:
> > ======================
> > 128 1 0.012584 0.000724 0.011073 0.015331
> > 128 2 0.012506 0.000444 0.011707 0.013461
> > 128 4 0.012412 0.000511 0.011157 0.013413
> > 128 8 0.012488 0.000455 0.011767 0.013746
> > 128 16 0.012664 0.000416 0.011745 0.013362
> > 128 32 0.012878 0.000410 0.012157 0.013609
> > 128 64 0.013138 0.000417 0.012452 0.013826
> > 128 128 0.014016 0.000505 0.013195 0.014942 +
> > 128 256 0.015843 0.000521 0.015107 0.016725 +
> > 128 512 0.052240 0.079323 0.027019 0.320653 !
> > 128 1024 0.123884 0.121560 0.038062 0.308929 !
> > 128 2048 0.176877 0.125229 0.074457 0.387276 !
> > 128 4096 0.305030 0.121716 0.176640 0.496375 !
> > 128 8192 0.546405 0.108007 0.415272 0.899858 !
> > 128 16384 0.604844 0.056576 0.558657 0.843943 !
> > 128 32768 1.235298 0.097969 1.094720 1.451241 !
> > 128 65536 2.926902 0.312733 2.458742 3.895563 !
> > 128 131072 6.208087 0.472115 5.354304 7.317153 !

---------------------------------------------------
Dr. Carsten Kutzner
Max Planck Institute for Biophysical Chemistry
Theoretical and Computational Biophysics Department
Am Fassberg 11
37077 Goettingen, Germany
Tel. +49-551-2012313, Fax: +49-551-2012302
eMail ckutzne_at_[hidden]
http://www.gwdg.de/~ckutzne