thanks for the quick response!
Here some results with enabled process binding. I hope I used the parameters
bind two ranks to one socket:
$ mpirun -np 2 --bind-to-core ./all2all
$ mpirun -np 2 -mca mpi_paffinity_alone 1 ./all2all
bind two ranks to two different sockets:
$ mpirun -np 2 --bind-to-socket ./all2all
All three runs resulted in similar bad latencies (~1.4us).
On Monday 13 February 2012 12:43:22 sylvain.jeaugey_at_[hidden] wrote:
> Hi Matthias,
> You might want to play with process binding to see if your problem is
> related to bad memory affinity.
> Try to launch pingpong on two CPUs of the same socket, then on different
> sockets (i.e. bind each process to a core, and try different
> De : Matthias Jurenz <matthias.jurenz_at_[hidden]>
> A : Open MPI Developers <devel_at_[hidden]>
> Date : 13/02/2012 12:12
> Objet : [OMPI devel] poor btl sm latency
> Envoyé par : devel-bounces_at_[hidden]
> Hello all,
> on our new AMD cluster (AMD Opteron 6274, 2,2GHz) we get very bad
> (~1.5us) when performing 0-byte p2p communication on one single node using
> Open MPI sm BTL. When using Platform MPI we get ~0.5us latencies which is
> pretty good. The bandwidth results are similar for both MPI
> (~3,3GB/s) - this is okay.
> One node has 64 cores and 64Gb RAM where it doesn't matter how many ranks
> allocated by the application. We get similar results with different number
> We are using Open MPI 1.5.4 which is built by gcc 4.3.4 without any
> configure options except the installation prefix and the location of the
> As mentioned at http://www.open-mpi.org/faq/?category=sm we tried to use
> /dev/shm instead of /tmp for the session directory, but it had no effect.
> Furthermore, we tried the current release candidate 1.5.5rc1 of Open MPI
> provides an option to use the SysV shared memory (-mca shmem sysv) - also
> results in similar poor latencies.
> Do you have any idea? Please help!
> devel mailing list