Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] TCP Latency
From: George Bosilca (bosilca_at_[hidden])
Date: 2008-07-29 06:01:39

I just want to make sure that I correctly understand your statement.
You're saying that running NetPIPE (NPtcp) directly over TCP give you
a latency of 12us, but running NetPIPE (NPmpi) over Open MPI bring
this latency up to 45us ?


On Jul 29, 2008, at 10:52 AM, Andy Georgi wrote:

> Zitat von Jeff Squyres <jsquyres_at_[hidden]>:
>> On Jul 28, 2008, at 2:53 PM, Andy Georgi wrote:
>>> we use Chelsio S320E-CXA adapters (
>>> )
>>> in one of our clusters. After tuning the kernel i measured the ping
>>> pong latency via NetPIPE and got ~12us which is pretty good for
>>> TCP i
>>> think. So i wrote a simple ping-pong-kernel and was really terrified
>>> about the ~45us i got with OpenMPI 1.2.6. Are there any hints how we
>>> can reduce the MPI latency? To increase the bandwidth we already set
>>> the buffer sizes but we couldn't find a parameter which can be
>>> relevant for the latency. Every hint is welcome.
>> The upcoming Open MPI v1.3 series will support iWARP, which gives
>> much
>> better latency than that. I don't know all the Chelsio models
>> offhand;
>> are those iWARP-capable cards?
> Thanks for the fast answer. So is this latency normal for TCP
> communications over MPI!? Could RDMA maybe reduce the latency? It
> should work with those cards but there are still problems with OFED.
> iWARP is also one of the features they offer but if it works...
> --
> Dresden University of Technology
> Center for Information Services
> and High Performance Computing (ZIH)
> D-01062 Dresden
> Germany
> e-mail: Andy.Georgi_at_[hidden]
> WWW:
> _______________________________________________
> users mailing list
> users_at_[hidden]

  • application/pkcs7-signature attachment: smime.p7s