Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Message compression in OpenMPI
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2008-04-22 12:08:54


On Apr 22, 2008, at 9:03 AM, Tomas Ukkonen wrote:

> I read from somewhere that OpenMPI supports
>
> some kind of data compression but I couldn't find
> any information about it.
>
> Is this true and how it can be used?
>

Nope, sorry -- not true.

This just came up in a different context, actually. We added some
preliminary compression on our startup/mpirun messages and found that
it really had no effect; any savings that you get in bandwidth (and
therefore overall wall clock time) are eaten up by the time necessary
to compress/uncompress the messages. There were a few more things we
could have tried, but frankly we had some higher priority items to
finish for the upcoming v1.3 series. :-(

> Does anyone have any experiences about using it?
>
> Is it possible to use compression in just some
> subset of communications (communicator
> specific compression settings)?
>
> In our MPI application we are transferring large
> amounts of sparse/redundant data that compresses
> very well. Also my initial tests showed significant
> improvements in performance.
>

If your particular data is well-suited for fast compression, you might
want to compress it before calling MPI_SEND / after calling MPI_RECV.
Use the MPI_BYTE datatype to send/receive the messages, and then MPI
won't do anything additional for datatype conversions, etc.

-- 
Jeff Squyres
Cisco Systems