Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] TCP Latency
From: Andy Georgi (Andy.Georgi_at_[hidden])
Date: 2008-07-29 04:52:01

Zitat von Jeff Squyres <jsquyres_at_[hidden]>:

> On Jul 28, 2008, at 2:53 PM, Andy Georgi wrote:
>> we use Chelsio S320E-CXA adapters
>> (
>> in one of our clusters. After tuning the kernel i measured the ping
>> pong latency via NetPIPE and got ~12us which is pretty good for TCP i
>> think. So i wrote a simple ping-pong-kernel and was really terrified
>> about the ~45us i got with OpenMPI 1.2.6. Are there any hints how we
>> can reduce the MPI latency? To increase the bandwidth we already set
>> the buffer sizes but we couldn't find a parameter which can be
>> relevant for the latency. Every hint is welcome.
> The upcoming Open MPI v1.3 series will support iWARP, which gives much
> better latency than that. I don't know all the Chelsio models offhand;
> are those iWARP-capable cards?

Thanks for the fast answer. So is this latency normal for TCP
communications over MPI!? Could RDMA maybe reduce the latency? It
should work with those cards but there are still problems with OFED.
iWARP is also one of the features they offer but if it works...

Dresden University of Technology
Center for Information Services
and High Performance Computing (ZIH)
D-01062 Dresden
e-mail: Andy.Georgi_at_[hidden]