# Open MPI User's Mailing List Archives

 |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list
 Subject: Re: [OMPI users] Message compression in OpenMPI From: Tomas Ukkonen (tomas.ukkonen_at_[hidden]) Date: 2008-04-24 08:26:35 George Bosilca wrote: > The paper you cited, while presenting a particular implementation > doesn't bring present any new ideas. The compression of the data was > studied for long time, and [unfortunately] it always came back to the > same result. In the general case, not worth the effort ! > > Now of course, if one limit itself to very regular applications (such > as the one presented in the paper), where the matrices involved in the > computation are well conditioned (such as in the paper), and if you > only use MPI_DOUBLE (\cite{same_paper}), and finally if you only > expect to run over slow Ethernet (1Gbs) (\cite{same_paper_again})... > then yes one might get some benefit. > Yes, you are probably right that its not worth the effort in general and especially not in HPC environments where you have very fast network. But I can think of (rather important) special cases where it is important - non HPC environments with slow network: beowulf clusters and/or   internet + normal PCs where you use existing workstations and network   for computations. - communication/io-bound computations where you transfer   large redundant datasets between nodes So it would be nice to be able to turn on the compression (for spefic communicators and/or data transfers) when you need it. ```-- Tomas > george. > > On Apr 22, 2008, at 9:03 AM, Tomas Ukkonen wrote: > >> Hello >> >> I read from somewhere that OpenMPI supports >> some kind of data compression but I couldn't find >> any information about it. >> >> Is this true and how it can be used? >> >> Does anyone have any experiences about using it? >> >> Is it possible to use compression in just some >> subset of communications (communicator >> specific compression settings)? >> >> In our MPI application we are transferring large >> amounts of sparse/redundant data that compresses >> very well. Also my initial tests showed significant >> improvements in performance. >> >> There are also articles that suggest that compression >> should be used [1]. >> >> [1] J. Ke, M. Burtcher and E. Speight. >> Runtime Compression of MPI Messages to Improve the >> Performance and Scalability of Parallel Applications. >> >> >> Thanks in advance, >> Tomas >> >> _______________________________________________ >> users mailing list >> users_at_[hidden] >> http://www.open-mpi.org/mailman/listinfo.cgi/users > > ------------------------------------------------------------------------ > > _______________________________________________ > users mailing list > users_at_[hidden] > http://www.open-mpi.org/mailman/listinfo.cgi/users ``` text/html attachment: attachment