Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2007-08-13 11:07:59

On Aug 13, 2007, at 10:49 AM, George Bosilca wrote:

> You want a dirtier trick for benchmarks ... Here it is ...
> Implement a compression like algorithm based on checksum. The data-
> type engine can compute a checksum for each fragment and if the
> checksum match one in the peer [limitted] history (so we can claim
> our communication protocol is adaptive), then we replace the actual
> message content with the matched id in the common history. Checksums
> are fairly cheap, lookup in a balanced tree is cheap too, so we will
> end up with a lot of improvement (as instead of sending a full
> fragment we will end-up sending one int). Based on the way most of
> the benchmarks initialize the user data (when they don't everything
> is mostly 0), this trick might work on all cases for the
> benchmarks ...

Are you sure you didn't want to publish a paper about this before you
sent it across a public list? Now someone else is likely to "invent"
this scheme and get credit for it. ;-)

Such a scheme is certainly possible, but I see even less use for it
than use cases for the existing microbenchmarks. Specifically,
header caching *can* happen in real applications (i.e., repeatedly
send short messages with the same MPI signature), but repeatedly
sending to the same peer with exactly the same signature *and*
exactly the same "long-enough" data (i.e., more than a small number
of ints that an app could use for its own message data caching) is
indicative of a poorly-written MPI application IMHO.

> But don't complain if your Linpack run fails.

I assume you're talking about bugs in the implementation; not a
problem with the approach, right?

Jeff Squyres
Cisco Systems