Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] kernel 2.6.23 vs 2.6.24 - communication/wait times
From: Oliver Geisler (openmpi_at_[hidden])
Date: 2010-04-06 18:09:11


On 4/6/2010 2:53 PM, Jeff Squyres wrote:

>
> Try NetPIPE -- it has both MPI communication benchmarking and TCP benchmarking. Then you can see if there is a noticable difference between TCP and MPI (there shouldn't be). There's also a "memcpy" mode in netpipe, but it's not quite the same thing as shared memory message passing.
>

Using netpipe and comparing tcp and mpi communication I get the
following results:

TCP is much faster than MPI, approx. by factor 12
e.g a packet size of 4096 bytes deliveres in
97.11 usec with NPtcp and
15338.98 usec with NPmpi
or
packet size 262kb
0.05268801 sec NPtcp
0.00254560 sec NPmpi

Further our benchmark started with "--mca btl tcp,self" runs with short
communication times, even using kernel 2.6.33.1

Is there a way to see what type of communication is actually selected?

Can anybody imagine why shared memory leads to these problems?
Kernel configuration?

Thanks, Jeff, for insisting upon testing network performance.
Thanks all others, too ;-)

oli

-- 
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.