On Apr 24, 2008, at 8:26 AM, Tomas Ukkonen wrote:
> Yes, you are probably right that its not worth the effort in general
> especially not in HPC environments where you have very fast network.
> But I can think of (rather important) special cases where it is
> - non HPC environments with slow network: beowulf clusters and/or
> internet + normal PCs where you use existing workstations and
> for computations.
> - communication/io-bound computations where you transfer
> large redundant datasets between nodes
> So it would be nice to be able to turn on the compression (for spefic
> communicators and/or data transfers) when you need it.
Quite possibly so. Note that there are a few proposals going on in
MPI-2.2/MPI-3 about how to pass "hints" or "assertions" to the MPI
implementation. Compression could be one of these hints -- the MPI
may not be able to detect that it's in a situation that is favorable
for compression, so having the user/app tell it "use compression on
this communicator" could be helpful.
Would you be willing to contribute the work to Open MPI to enable
compression? Per a post yesterday (http://www.open-mpi.org/community/lists/users/2008/04/5473.php
), contributions are always welcome.