Jeff Squyres wrote:
Ok, so I have to do it myself. Not a problem really because
On Apr 22, 2008, at 9:03 AM, Tomas Ukkonen wrote:
I read from somewhere that OpenMPI supports
some kind of data compression but I couldn't find
any information about it.
Is this true and how it can be used?
Nope, sorry -- not true.
This just came up in a different context, actually. We added some
preliminary compression on our startup/mpirun messages and found that
it really had no effect; any savings that you get in bandwidth (and
therefore overall wall clock time) are eaten up by the time necessary
to compress/uncompress the messages. There were a few more things we
could have tried, but frankly we had some higher priority items to
finish for the upcoming v1.3 series. :-(
there are only few places where the compression really seems to matter.
Yeah, already did something like this. I have a situation where all
Does anyone have any experiences about using it?
Is it possible to use compression in just some
subset of communications (communicator
specific compression settings)?
In our MPI application we are transferring large
amounts of sparse/redundant data that compresses
very well. Also my initial tests showed significant
improvements in performance.
If your particular data is well-suited for fast compression, you might
want to compress it before calling MPI_SEND / after calling MPI_RECV.
Use the MPI_BYTE datatype to send/receive the messages, and then MPI
won't do anything additional for datatype conversions, etc
the nodes are sending large amounts of redundant data at once. The
combination: "compress --> MPI_SEND --> MPI_RECV -->
works of course, but it forces one to allocate large amounts of memory
(or diskspace) for the compressed data. You can do it manually in parts
of course, but it would be nice if MPI library could do it behind the