Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Parallel file write in fortran (+mpi)
From: Laurence Marks (L-marks_at_[hidden])
Date: 2010-02-02 18:40:01


I know it's wrong, but I don't think it is forbidden.... (which I
guess is what you are saying).

On Tue, Feb 2, 2010 at 5:31 PM, Jeff Squyres <jsquyres_at_[hidden]> wrote:
> +1 on Nick's responses.
>
> AFAIK, if you don't mind getting garbage in the output file, it should be fine to do.  Specifically: it should not cause OS issues (crash, reboot, corrupted filesystem, etc.) to do this -- but the file contents will likely be garbage.
>
> That being said, this situation likely falls into the "Doc, it hurts when I do this..." category.  Meaning: you know it's wrong, so you probably shouldn't be doing it anyway.  :-)
>
>
> On Feb 2, 2010, at 4:50 PM, Nicolas Bock wrote:
>
>> Hi Laurence,
>>
>> I don't know whether it's as bad as a deadly sin, but for us parallel writes are a huge problem and we get complete garbage in the file. Take a look at:
>>
>> Implementing MPI-IO Atomic Mode and Shared File Pointers Using MPI One-Sided Communication, Robert Latham,Robert Ross, Rajeev Thakur, International Journal of High Performance Computing Applications, 21, 132 (2007).
>>
>> They describe an implemenation of a "mutex" like object in MPI. If you protect writes to the file with an exclusive lock you can serialize the writes and make use of NFS's close to open cache coherence.
>>
>> nick
>>
>>
>> On Tue, Feb 2, 2010 at 08:27, Laurence Marks <L-marks_at_[hidden]> wrote:
>> I have a question concerning having many processors in a mpi job all
>> write to the same file -- not using mpi calls but with standard
>> fortran I/O. I know that this can lead to consistency issues, but it
>> can also lead to OS issues with some flavors of nfs.
>>
>> At least in fortran, there is nothing "wrong" with doing this. My
>> question is whether this is "One of the Seven Deadly Sins" of mpi
>> programming, or just frowned on. (That is, it should be OK even if it
>> leads to nonsense files, and not lead to OS issues.) If it is a sin, I
>> would appreciate a link to where this is spelt out in some "official"
>> document or similar.
>>
>> --
>> Laurence Marks
>> Department of Materials Science and Engineering
>> MSE Rm 2036 Cook Hall
>> 2220 N Campus Drive
>> Northwestern University
>> Evanston, IL 60208, USA
>> Tel: (847) 491-3996 Fax: (847) 491-7820
>> email: L-marks at northwestern dot edu
>> Web: www.numis.northwestern.edu
>> Chair, Commission on Electron Crystallography of IUCR
>> www.numis.northwestern.edu/
>> Electron crystallography is the branch of science that uses electron
>> scattering and imaging to study the structure of matter.
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> --
> Jeff Squyres
> jsquyres_at_[hidden]
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>

-- 
Laurence Marks
Department of Materials Science and Engineering
MSE Rm 2036 Cook Hall
2220 N Campus Drive
Northwestern University
Evanston, IL 60208, USA
Tel: (847) 491-3996 Fax: (847) 491-7820
email: L-marks at northwestern dot edu
Web: www.numis.northwestern.edu
Chair, Commission on Electron Crystallography of IUCR
www.numis.northwestern.edu/
Electron crystallography is the branch of science that uses electron
scattering and imaging to study the structure of matter.