But its a very bad idea on a "coherent", "POSIX" filesystem like Lustre.
Locks have to bounce around between the nodes for every write. This can
be VERY slow (even for trivial amounts of "logging" IO) and thrash the
filesystem for other users. So, yes, at our site, we include this sort
of "parallel IO" on our list of disallowed behaviour. Not a good practice
to adopt in general.
On 02/03/2010 10:40 AM, Laurence Marks wrote:
> I know it's wrong, but I don't think it is forbidden.... (which I
> guess is what you are saying).
> On Tue, Feb 2, 2010 at 5:31 PM, Jeff Squyres<jsquyres_at_[hidden]> wrote:
>> +1 on Nick's responses.
>> AFAIK, if you don't mind getting garbage in the output file, it should be fine to do. Specifically: it should not cause OS issues (crash, reboot, corrupted filesystem, etc.) to do this -- but the file contents will likely be garbage.
>> That being said, this situation likely falls into the "Doc, it hurts when I do this..." category. Meaning: you know it's wrong, so you probably shouldn't be doing it anyway. :-)
>> On Feb 2, 2010, at 4:50 PM, Nicolas Bock wrote:
>>> Hi Laurence,
>>> I don't know whether it's as bad as a deadly sin, but for us parallel writes are a huge problem and we get complete garbage in the file. Take a look at:
>>> Implementing MPI-IO Atomic Mode and Shared File Pointers Using MPI One-Sided Communication, Robert Latham,Robert Ross, Rajeev Thakur, International Journal of High Performance Computing Applications, 21, 132 (2007).
>>> They describe an implemenation of a "mutex" like object in MPI. If you protect writes to the file with an exclusive lock you can serialize the writes and make use of NFS's close to open cache coherence.
>>> On Tue, Feb 2, 2010 at 08:27, Laurence Marks<L-marks_at_[hidden]> wrote:
>>> I have a question concerning having many processors in a mpi job all
>>> write to the same file -- not using mpi calls but with standard
>>> fortran I/O. I know that this can lead to consistency issues, but it
>>> can also lead to OS issues with some flavors of nfs.
>>> At least in fortran, there is nothing "wrong" with doing this. My
>>> question is whether this is "One of the Seven Deadly Sins" of mpi
>>> programming, or just frowned on. (That is, it should be OK even if it
>>> leads to nonsense files, and not lead to OS issues.) If it is a sin, I
>>> would appreciate a link to where this is spelt out in some "official"
>>> document or similar.
>>> Laurence Marks
>>> Department of Materials Science and Engineering
>>> MSE Rm 2036 Cook Hall
>>> 2220 N Campus Drive
>>> Northwestern University
>>> Evanston, IL 60208, USA
>>> Tel: (847) 491-3996 Fax: (847) 491-7820
>>> email: L-marks at northwestern dot edu
>>> Web: www.numis.northwestern.edu
>>> Chair, Commission on Electron Crystallography of IUCR
>>> Electron crystallography is the branch of science that uses electron
>>> scattering and imaging to study the structure of matter.
>>> users mailing list
>>> users mailing list
>> Jeff Squyres