Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: [OMPI users] Strange TCP latency results on Amazon EC2
From: Roberto Rey (eros.83_at_[hidden])
Date: 2012-01-10 10:02:58


I'm running some tests on EC2 cluster instances with 10 Gigabit Ethernet
hardware and I'm getting strange latency results with Netpipe and OpenMPI.

If I run Netpipe over OpenMPI (NPmpi) I get a network latency around 60
microseconds for small messages (less than 2kbytes). However, when I run
Netpipe over TCP (NPtcp) I always get around 100 microseconds. For bigger
messages everything seems to be OK.

I'm using the BTL TCP in OpenMPI, so I can't understand why OpenMPI
outperforms raw TCP performance for small messages (40us of difference). I
also have run the PingPong test from the Intel Media Benchmarks and the
latency results for OpenMPI are very similar (60us) to those obtained with

Can OpenMPI outperform Netpipe over TCP? Why? Is OpenMPI doing any
optimization in BTL TCP?

The results for OpenMPI aren't so good but we must take into account the
network virtualization overhead under Xen

Thanks for your reply