Hi guys,
I was hoping someone here could shed some light on OpenMPI's use of /tmp
(or, I guess, TMPDIR) and save me from diving into the source.. ;)
The background is that I'm trying to run some applications on a system
which has a flaky parallel file system which TMPDIR is mapped to - so, on
start up, OpenMPI creates it's 'openmpi-sessions-<user>' directory there,
and under that, a few files. Sometimes I see 1 subdirectory (usually a 0),
sometimes a 0 and a 1, etc. In one of these, sometimes I see files such as
'shared_memory_pool.<host>', and 'shared_memory_module.<host>'.
My questions are, one, what are the various numbers / files for? (If
there's a write-up on this somewhere, just point me towards it!)
And two, the real question, are these 'files' used during runtime, or only
upon startup / shutdown? I'm having issues with various codes, especially
those heavy on messages and I/O, failing to complete a run, and haven't
resorted to sifting through strace's output yet. This doesn't happen all
the time, but I've seen it happen reliably now with one particular code -
it's success rate (it DOES succeed sometimes) is about 25% right now. My
best guess is that this is because the file system is overloaded, thus not
allowing timely I/O or access to OpenMPI's files, but I wanted to get a
quick understanding of how these files are used by OpenMPI and whether the
FS does indeed seem a likely culprit before going with that theory for sure.
Thanks very much,
- Brian
Brian Dobbins
Yale Engineering HPC
|
![[Flag]](../../../../../community/mirrors/flag-us.gif){kind=small}