On Mon, Nov 23, 2009 at 11:17:45AM +0100, Christoph Rackwitz wrote:
> >If I may ask a slightly different question: you've got periods of I/O
> >and periods of computation. Have you evaluated collective I/O?
> I thought about it and I know a way to make it happen too, but I put
> that on the "to do" pile for possible improvements later on, after
> I'd have gotten the asynchronous I/O working. My file format
> contains a struct followed by two matrices (same dimensions). Right
> now, I write the header via rank 0 and then each rank writes one
> stripe for each matrix, resulting in two Requests pending. I gather
> that I'd need to construct one or two more data types for
> split-collective I/O to be applicable, i.e., so the whole write
> happens in one call.
If the other processors need header data, perhaps rank 0 can broadcast
it to everyone else?
You won't get any overlap with split collectives in any ROMIO-based
MPI-IO: they have always been implemented in a blocking fashion.
You're on the right track: describe the I/O for the two matrices with
one datatype, then on rank zero use that datatype as one of the
datatype for a struct type.
It's definitely a different way of thinking about I/O than the async
Is it OK to mention MPICH2 on this list? I did prototype some MPI
extensions that allowed ROMIO to do true async I/O (at least as far
as the underlying operating system supports it). If you really need
to experiment with async I/O, I'd love to hear your experiences.
Mathematics and Computer Science Division
Argonne National Lab, IL USA