On Fri, Mar 20, 2009 at 2:21 PM, Scott Atchley <atchley@myri.com> wrote:
On Mar 20, 2009, at 5:59 AM, Ricardo Fernández-Perea wrote:

Hello,

I am running DL_POLY in various Xserver 8 processor with a myrinet network.using mx-1.2.7

While I keep in the same node the process scales reasonably well but in the moment I hit the network ...

I will like to try to maximize  the mx network before trying to touch the program code.

Have you run mx_pingpong between the nodes to determine what raw performance you are getting?

For each pair of machines, run:

Host foo:
$ mx_pingpong -s

Host bar:
$ mx_pingpong -d foo -S 0 -E 5000000 -M 2

where "-d foo" means the MX hostname of foo. It could be foo or foo:0 (or foo:1 if you have two NICs, etc.).

See http://www.myri.com/serve/cache/455.html for more details.

Repeat the above after setting MX_RCACHE=1 on both hosts.
This are the results
initially 
Running 1000 iterations.
   Length   Latency(us)    Bandwidth(MB/s)
        0       2.738          0.000
        1       2.718          0.368
        2       2.707          0.739
        4       2.715          1.474
        8       2.719          2.943
       16       2.748          5.822
       32       2.814         11.370
       64       3.333         19.199
      128       3.808         33.609
      256       6.069         42.185
      512       8.101         63.206
     1024      11.841         86.483
     2048      16.100        127.205
     4096      24.942        164.221
     8192      44.332        184.788
    16384      83.784        195.550
    32768     150.186        218.182
    65536     318.591        205.706
   131072     591.277        221.676
   262144    1128.918        232.208
   524288    2215.368        236.660
  1048576    4392.217        238.735
  2097152    8705.028        240.913
  4194304   17359.166        241.619

with  export MX_RCACHE=1

Running 1000 iterations.
   Length   Latency(us)    Bandwidth(MB/s)
        0       2.731          0.000
        1       2.705          0.370
        2       2.719          0.736
        4       2.703          1.480
        8       2.709          2.953
       16       2.743          5.833
       32       2.811         11.386
       64       3.336         19.188
      128       3.815         33.552
      256       6.029         42.465
      512       8.082         63.347
     1024      11.814         86.677
     2048      16.113        127.098
     4096      24.945        164.205
     8192      44.354        184.696
    16384      83.769        195.587
    32768     150.183        218.187
    65536     299.793        218.604
   131072     561.008        233.637
   262144    1093.738        239.677
   524288    2150.151        243.838
  1048576    4265.846        245.807
  2097152    8491.122        246.982
  4194304   16953.997        247.393

Ricardo
 


Scott

Is there some place where I can check what the mca params really means?
Can anyone give me a hint where it would be better to look to optimize OMPI1.3 for macosx?

Thanks

Ricardo
_______________________________________________
users mailing list
users@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users


_______________________________________________
users mailing list
users@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users