On Apr 22, 2008, at 9:03 AM, Tomas Ukkonen wrote:
> I read from somewhere that OpenMPI supports
>
> some kind of data compression but I couldn't find
> any information about it.
>
> Is this true and how it can be used?
>
Nope, sorry -- not true.
This just came up in a different context, actually. We added some
preliminary compression on our startup/mpirun messages and found that
it really had no effect; any savings that you get in bandwidth (and
therefore overall wall clock time) are eaten up by the time necessary
to compress/uncompress the messages. There were a few more things we
could have tried, but frankly we had some higher priority items to
finish for the upcoming v1.3 series. :-(
> Does anyone have any experiences about using it?
>
> Is it possible to use compression in just some
> subset of communications (communicator
> specific compression settings)?
>
> In our MPI application we are transferring large
> amounts of sparse/redundant data that compresses
> very well. Also my initial tests showed significant
> improvements in performance.
>
If your particular data is well-suited for fast compression, you might
want to compress it before calling MPI_SEND / after calling MPI_RECV.
Use the MPI_BYTE datatype to send/receive the messages, and then MPI
won't do anything additional for datatype conversions, etc.
--
Jeff Squyres
Cisco Systems
|
![[Flag]](../../../../../community/mirrors/flag-us.gif){kind=small}