Jeff Squyres wrote:
On Apr 22, 2008, at 9:03 AM, Tomas Ukkonen wrote:
  
I read from somewhere that OpenMPI supports

some kind of data compression but I couldn't find
any information about it.

Is this true and how it can be used?
    
Nope, sorry -- not true.

This just came up in a different context, actually.  We added some  
preliminary compression on our startup/mpirun messages and found that  
it really had no effect; any savings that you get in bandwidth (and  
therefore overall wall clock time) are eaten up by the time necessary  
to compress/uncompress the messages.  There were a few more things we  
could have tried, but frankly we had some higher priority items to  
finish for the upcoming v1.3 series.  :-(
  
Ok, so I have to do it myself. Not a problem really because
there are only few places where the compression really seems to matter.
Does anyone have any experiences about using it?

Is it possible to use compression in just some
subset of communications (communicator
specific compression settings)?

In our MPI application we are transferring large
amounts of sparse/redundant data that compresses
very well. Also my initial tests showed significant
improvements in performance.
    

If your particular data is well-suited for fast compression, you might  
want to compress it before calling MPI_SEND / after calling MPI_RECV.   
Use the MPI_BYTE datatype to send/receive the messages, and then MPI  
won't do anything additional for datatype conversions, etc
Yeah, already did something like this. I have a situation where all
the nodes are sending large amounts of redundant data at once. The
combination: "compress --> MPI_SEND --> MPI_RECV --> decompress"
works of course, but it forces one to allocate large amounts of memory
(or diskspace) for the compressed data. You can do it manually in parts
of course, but it would be nice if MPI library could do it behind the
scenes.

Thanks,
-- 
Tomas Ukkonen