Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Tom Lund (lund_at_[hidden])
Date: 2007-01-09 16:53:24


Rob,
   Thank you for your informative reply. I had no luck finding the
external32 data representation in any of several mpi implementations and
thus I do need to devise an alternative strategy. Do you know of a good
reference explaining how to combine HDF5 with mpi?

   ---Tom

Robert Latham wrote:
> On Mon, Jan 08, 2007 at 02:32:14PM -0700, Tom Lund wrote:
>
>> Rainer,
>> Thank you for taking time to reply to my querry. Do I understand
>> correctly that external32 data representation for i/o is not
>> implemented? I am puzzled since the MPI-2 standard clearly indicates
>> the existence of external32 and has lots of words regarding how nice
>> this feature is for file interoperability. So do both Open MPI and
>> MPIch2 not adhere to the standard in this regard? If this is really the
>> case, how difficult is it to define a custom data representation that is
>> 32-bit big endian on all platforms? Do you know of any documentation
>> that explains how to do this?
>> Thanks again.
>>
>
> Hi Tom
>
> You do understand correctly. I do not know of an MPI-IO
> implementation that supports external32.
>
> When you say "custom data representation" do you mean an MPI-IO
> user-defined data representation?
>
> An alternate approach would be to use a higher level library like
> parallel-netcdf or HDF5 (configured for parallel i/o). Those
> libraries already define a file format and implement all the necessary
> data conversion routines, and they have a wealth of ancilary tools and
> programs to work with their respective file formats. Additionally,
> those higher-level libraries will offer you more features than MPI-IO
> such as the ability to define atributes on variables and datafiles.
> Even better, there is the potential that these libraries might offer
> some clever optimizations for your workload, saving you the effort.
> Further, you can use those higher-level libraries on top of any MPI-IO
> implementation, not just OpenMPI or MPICH2.
>
> This is a little bit of a diversion from your original question, but
> to sum it up, I'd say one potential answer to the lack of external32
> is to use a higher level library and sidestep the issue of MPI-IO data
> representations altogether.
>
> ==rob
>
>

-- 
===========================================================
   Thomas S. Lund
   Sr. Research Scientist
   Colorado Research Associates, a division of
   NorthWest Research Associates
   3380 Mitchell Ln.
   Boulder, CO 80301
   (303) 415-9701 X 209 (voice)
   (303) 415-9702       (fax)
   lund_at_[hidden]
===========================================================