On Oct 18, 2007, at 7:56 AM, Gleb Natapov wrote:
>> Open MPI v1.2.4 (and newer) will get around 1.5us latency with 0 byte
>> ping-pong benchmarks on Mellanox ConnectX HCAs. Prior versions of
>> Open MPI can also achieve this low latency by setting the
>> btl_openib_use_eager_rdma MCA parameter to 1.
>
> Actually setting btl_openib_use_eager_rdma to 1 will not help. The
> reason is that it is 1 by default anyway, but Open MPI disables eager
> rdma because it can't find HCA description in the ini file and cannot
> distinguish between default value and value that user set explicitly.
Arrgh; that's a fun (read: annoying) bug. Well, it's not a total
loss -- you can still get the same performance in older Open MPI
versions by adding the following to the end of the $prefix/share/
openmpi/mca-btl-openib-hca-params.ini file:
[Mellanox Hermon]
vendor_id = 0x2c9,0x5ad,0x66a,0x8f1,0x1708
vendor_part_id = 25408,25418,25428
use_eager_rdma = 1
mtu = 2048
--
Jeff Squyres
Cisco Systems
|
![[Flag]](../../../../../community/mirrors/flag-us.gif){kind=small}