Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Strange TCP latency results on Amazon EC2
From: Roberto Rey (eros.83_at_[hidden])
Date: 2012-01-12 10:04:42

Hi again,

Today I was trying with another TCP benchmark included in the hpcbench
suite, and with a ping-pong test I'm also getting 100us of latency. Then, I
tried with netperf and the same result....

So, in summary, I'm measuring TCP latency with messages size between 1-32

Netperf over TCP -> 100us
Netpipe over TCP (NPtcp) -> 100us
HPCbench over TCP -> 100us
Netpipe over OpenMPI (NPmpi) -> 60us
HPCBench over OpenMPI -> 60us

Any clues?

Thanks a lot!

2012/1/10 Roberto Rey <eros.83_at_[hidden]>

> Hi,
> I'm running some tests on EC2 cluster instances with 10 Gigabit Ethernet
> hardware and I'm getting strange latency results with Netpipe and OpenMPI.
> If I run Netpipe over OpenMPI (NPmpi) I get a network latency around 60
> microseconds for small messages (less than 2kbytes). However, when I run
> Netpipe over TCP (NPtcp) I always get around 100 microseconds. For bigger
> messages everything seems to be OK.
> I'm using the BTL TCP in OpenMPI, so I can't understand why OpenMPI
> outperforms raw TCP performance for small messages (40us of difference). I
> also have run the PingPong test from the Intel Media Benchmarks and the
> latency results for OpenMPI are very similar (60us) to those obtained with
> NPmpi
> Can OpenMPI outperform Netpipe over TCP? Why? Is OpenMPI doing any
> optimization in BTL TCP?
> The results for OpenMPI aren't so good but we must take into account the
> network virtualization overhead under Xen
> Thanks for your reply

Roberto Rey Expósito