Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Parallel file write in fortran (+mpi)
From: Laurence Marks (L-marks_at_[hidden])
Date: 2010-02-02 19:37:15


Agreed that it is not good (and I am recoding some programs to avoid
this), but (and here life gets interesting) is this a "bug" in Lustre?

On Tue, Feb 2, 2010 at 5:59 PM, David Singleton
<David.Singleton_at_[hidden]> wrote:
>
> But its a very bad idea on a "coherent", "POSIX" filesystem like Lustre.
> Locks have to bounce around between the nodes for every write.  This can
> be VERY slow (even for trivial amounts of "logging" IO) and thrash the
> filesystem for other users.   So, yes, at our site, we include this sort
> of "parallel IO" on our list of disallowed behaviour.  Not a good practice
> to adopt in general.
>
> David
>
> On 02/03/2010 10:40 AM, Laurence Marks wrote:
>>
>> I know it's wrong, but I don't think it is forbidden.... (which I
>> guess is what you are saying).
>>
>> On Tue, Feb 2, 2010 at 5:31 PM, Jeff Squyres<jsquyres_at_[hidden]>  wrote:
>>>
>>> +1 on Nick's responses.
>>>
>>> AFAIK, if you don't mind getting garbage in the output file, it should be
>>> fine to do.  Specifically: it should not cause OS issues (crash, reboot,
>>> corrupted filesystem, etc.) to do this -- but the file contents will likely
>>> be garbage.
>>>
>>> That being said, this situation likely falls into the "Doc, it hurts when
>>> I do this..." category.  Meaning: you know it's wrong, so you probably
>>> shouldn't be doing it anyway.  :-)
>>>
>>>
>>> On Feb 2, 2010, at 4:50 PM, Nicolas Bock wrote:
>>>
>>>> Hi Laurence,
>>>>
>>>> I don't know whether it's as bad as a deadly sin, but for us parallel
>>>> writes are a huge problem and we get complete garbage in the file. Take a
>>>> look at:
>>>>
>>>> Implementing MPI-IO Atomic Mode and Shared File Pointers Using MPI
>>>> One-Sided Communication, Robert Latham,Robert Ross, Rajeev Thakur,
>>>> International Journal of High Performance Computing Applications, 21, 132
>>>> (2007).
>>>>
>>>> They describe an implemenation of a "mutex" like object in MPI. If you
>>>> protect writes to the file with an exclusive lock you can serialize the
>>>> writes and make use of NFS's close to open cache coherence.
>>>>
>>>> nick
>>>>
>>>>
>>>> On Tue, Feb 2, 2010 at 08:27, Laurence Marks<L-marks_at_[hidden]>
>>>>  wrote:
>>>> I have a question concerning having many processors in a mpi job all
>>>> write to the same file -- not using mpi calls but with standard
>>>> fortran I/O. I know that this can lead to consistency issues, but it
>>>> can also lead to OS issues with some flavors of nfs.
>>>>
>>>> At least in fortran, there is nothing "wrong" with doing this. My
>>>> question is whether this is "One of the Seven Deadly Sins" of mpi
>>>> programming, or just frowned on. (That is, it should be OK even if it
>>>> leads to nonsense files, and not lead to OS issues.) If it is a sin, I
>>>> would appreciate a link to where this is spelt out in some "official"
>>>> document or similar.
>>>>
>>>> --
>>>> Laurence Marks
>>>> Department of Materials Science and Engineering
>>>> MSE Rm 2036 Cook Hall
>>>> 2220 N Campus Drive
>>>> Northwestern University
>>>> Evanston, IL 60208, USA
>>>> Tel: (847) 491-3996 Fax: (847) 491-7820
>>>> email: L-marks at northwestern dot edu
>>>> Web: www.numis.northwestern.edu
>>>> Chair, Commission on Electron Crystallography of IUCR
>>>> www.numis.northwestern.edu/
>>>> Electron crystallography is the branch of science that uses electron
>>>> scattering and imaging to study the structure of matter.
>>>> _______________________________________________
>>>> users mailing list
>>>> users_at_[hidden]
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>
>>>> _______________________________________________
>>>> users mailing list
>>>> users_at_[hidden]
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>
>>>
>>> --
>>> Jeff Squyres
>>> jsquyres_at_[hidden]
>>>
>>>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>

-- 
Laurence Marks
Department of Materials Science and Engineering
MSE Rm 2036 Cook Hall
2220 N Campus Drive
Northwestern University
Evanston, IL 60208, USA
Tel: (847) 491-3996 Fax: (847) 491-7820
email: L-marks at northwestern dot edu
Web: www.numis.northwestern.edu
Chair, Commission on Electron Crystallography of IUCR
www.numis.northwestern.edu/
Electron crystallography is the branch of science that uses electron
scattering and imaging to study the structure of matter.