Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI devel] kernel 2.6.23 vs 2.6.24 - communication/wait times
From: Eugene Loh (eugene.loh_at_[hidden])
Date: 2010-04-09 23:51:14


Oliver Geisler wrote:

> This is the results of skampi pt2pt, first with shared memory allowed,
> second shared memory excluded.

Thanks for the data. The TCP results are not very interesting... they
look reasonable.

The shared-memory data is rather straightforward: results are just
plain ridiculously bad. The results for "eager" messages (messages
shorter than 4Kbytes) are around 12 millisec. The results for
"rendezvous" messages (longer than 4 Kbytes, signal the receiver, wait
for an acknowledgement, then send the message) are about 30 millisec.

I was also curious about "long-message bandwidth", but since SKaMPI is
only going up to 16 Kbyte messages, we can't really tell.

But, maybe all that is irrelevent.

Why is shared-memory performance about four orders of magnitude slower
than it should be? The processes are communicating via memory that's
shared by having the processes all mmap the same file into their address
spaces. Is it possible that with the newer kernels, operations to that
shared file are going all the way out to disk? Maybe you don't know the
answer, but hopefully someone on this mail list can provide some insight.