Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Trouble with MPI-IO
From: Tom Rosmond (rosmond_at_[hidden])
Date: 2011-05-24 11:31:23


Rob,

Thanks for the clarification. I had seen that point about
non-decreasing offsets in the standard and it was just beginning to dawn
on me that maybe it was my problem. I will rethink my mapping strategy
to comply with the restriction. Thanks again.

T. Rosmond

On Tue, 2011-05-24 at 10:09 -0500, Rob Latham wrote:
> On Fri, May 20, 2011 at 08:14:07AM -0400, Jeff Squyres wrote:
> > On May 20, 2011, at 6:23 AM, Jeff Squyres wrote:
> >
> > > Shouldn't ijlena and ijdisp be 1D arrays, not 2D arrays?
> >
> > Ok, if I convert ijlena and ijdisp to 1D arrays, I don't get the compile error (even though they're allocatable -- so allocate was a red herring, sorry). That's all that "use mpi" is complaining about -- that the function signatures didn't match.
> >
> > use mpi is your friend -- even if you don't use F90 constructs much. Compile-time checking is Very Good Thing (you were effectively "getting lucky" by passing in the 2D arrays, I think).
> >
> > Attached is my final version. And with this version, I see the hang when running it with the "T" parameter.
> >
> > That being said, I'm not an expert on the MPI IO stuff -- your code *looks* right to me, but I could be missing something subtle in the interpretation of MPI_FILE_SET_VIEW. I tried running your code with MPICH 1.3.2p1 and it also hung.
> >
> > Rob (ROMIO guy) -- can you comment this code? Is it correct?
>
> There's a kind of obscure but important rule in MPI-IO: the file view
> must describe monotonically non-decreasing offsets.
>
> the T type creates a file type with the following flattened
> representation (you can kind of think of the flattened representation
> as a type map, except everything is in terms of bytes):
>
> (0, 32), (96, 32), (32, 64)
>
> So, 32 bytes at offset 0, 32 bytes at offset 96 and 64 bytes at offset
> 32.
>
> That sort of looks like this:
> |xxxx~~~~~~~~~~~~zzzz~~~~yyyy|
>
> But you need the zzzz and yyyy pieces to be swapped in file view.
>
> It's an annoying part of the standard but as you can see if you
> violate that ROMIO will go off and spin in an infinite loop looking
> for the next piece of I/O (which in this case was "behind" the current
> piece).
>
> You can work around this by adjusting your memory datatype: data must
> be read off of the disk in this monotonically non-decreasing order but
> it can be jammed into memory any which way you want.
>
> ROMIO should be better about reporting file views that violate this
> part of the standard. We report it in a few places but clearly not
> enough.
>
> ==rob
>