Sorry I missed this message before... it got lost in the deluge that
is my inbox.
Are you using the mpi_leave_pinned MCA parameter? That will make a
big difference on the typical ping-pong benchmarks:
mpirun --mca mpi_leave_pinned 1 ....
On Oct 11, 2007, at 11:44 AM, Matteo Cicuttin wrote:
> Il giorno 11/ott/07, alle ore 07:16, Neeraj Chourasia ha scritto:
>> Dear All,
>> Could anyone tell me the important tuning parameters in
>> openmpi with IB interconnect? I tried setting eager_rdma,
>> min_rdma_size, mpi_leave_pinned parameters from the mpirun command
>> line on 38 nodes cluster (38*2 processors) but in vain. I found
>> simple mpirun with no mca parameters performing better. I
>> conducted test on P2P send/receive with data size of 8MB.
>> Similarly i patched HPL linpack code with libnbc(non blocking
>> collectives) and found no performance benefits. I went through its
>> patch and found that, its probably not overlapping computation
>> with communication.
>> Any help in this direction would be appreciated.
> I'm Matteo, and I work for a company that produces HPC systems, in
> I'm new in that company and I'm looking for some help, and this
> thread seems to be good :)
> In the last days we're benchmarking a system, and I'm interested in
> some performance
> scores of the infiniband interconnect.
> The nodes are dual dual-core opteron machines and we use the PCI-X
> IB interfaces Mellanox Cougar Cub.
> Machines have the 8111 system controller and the 8131 PCI-X bridge.
> We reach a rate of about 600 MB/s in the point-to-point tests.
> This rate (more or less) is reported both by the ib_*_bw benchmarks
> and the IMB-MPI (sendrecv) benchmarks, version 3.
> MPI implementation is, of course, openmpi.
> I've read in a few places that a similar setup can reach about 800
> MB/s on machines similar to those descripted above.
> Someone can confirm this? Someone have similar hardware and the
> measured bandwidth is better than 600 MB/s?
> Thank you in advance,
> Best regards,
> Cicuttin Matteo
> Black holes are where god divided by zero
> users mailing list