Looks to me like things are okay thru 160, and then things fall apart after that point. How many cores are on a node?


On Jun 9, 2013, at 1:59 PM, "Blosch, Edwin L" <edwin.l.blosch@lmco.com> wrote:

Iím having some trouble getting good scaling with OpenMPI 1.6.4 and I donít know where to start looking. This is an Infiniband FDR network with Sandy Bridge nodes.  I am using affinity (--bind-to-core) but no other options. As the number of cores goes up, the message sizes are typically going down. There seem to be lots of options in the FAQ, and I would welcome any advice on where to start.  All these timings are on a completely empty system except for me.
 
Thanks
 
 
    MPI              # cores   Ave. Rate   Std. Dev. %  # timings   Speedup    Efficiency
================================================================================================
MVAPICH            |   16   |    8.6783  |   0.995 % |       2  |   16.000  |  1.0000
MVAPICH            |   48   |    8.7665  |   1.937 % |       3  |   47.517  |  0.9899
MVAPICH            |   80   |    8.8900  |   2.291 % |       3  |   78.095  |  0.9762
MVAPICH            |  160   |    8.9897  |   2.409 % |       3  |  154.457  |  0.9654
MVAPICH            |  320   |    8.9780  |   2.801 % |       3  |  309.317  |  0.9666
MVAPICH            |  480   |    8.9704  |   2.316 % |       3  |  464.366  |  0.9674
MVAPICH            |  640   |    9.0792  |   1.138 % |       3  |  611.739  |  0.9558
MVAPICH            |  720   |    9.1328  |   1.052 % |       3  |  684.162  |  0.9502
MVAPICH            |  800   |    9.1945  |   0.773 % |       3  |  755.079  |  0.9438
OpenMPI            |   16   |    8.6743  |   2.335 % |       2  |   16.000  |  1.0000
OpenMPI            |   48   |    8.7826  |   1.605 % |       2  |   47.408  |  0.9877
OpenMPI            |   80   |    8.8861  |   0.120 % |       2  |   78.093  |  0.9762
OpenMPI            |  160   |    8.9774  |   0.785 % |       2  |  154.598  |  0.9662
OpenMPI            |  320   |   12.0585  |  16.950 % |       2  |  230.191  |  0.7193
OpenMPI            |  480   |   14.8330  |   1.300 % |       2  |  280.701  |  0.5848
OpenMPI            |  640   |   17.1723  |   2.577 % |       3  |  323.283  |  0.5051
OpenMPI            |  720   |   18.2153  |   2.798 % |       3  |  342.868  |  0.4762
OpenMPI            |  800   |   19.3603  |   2.254 % |       3  |  358.434  |  0.4480
_______________________________________________
users mailing list
users@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users