Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Shared-memory problems
From: Ralph Castain (rhc_at_[hidden])
Date: 2011-11-03 12:33:27

I'm afraid this isn't correct. You definitely don't want the session directory in /dev/shm as this will almost always cause problems.

We look thru a progression of envars to find where to put the session directory:

1. the MCA param orte_tmpdir_base

2. the envar OMPI_PREFIX_ENV

3. the envar TMPDIR

4. the envar TEMP

5. the envar TMP

Check all those to see if one is set to /dev/shm. If so, you have a problem to resolve. For performance reasons, you probably don't want the session directory sitting on a network mounted location. What you need is a good local directory - anything you have permission to write in will work fine. Just set one of the above to point to it.

On Nov 3, 2011, at 10:04 AM, Durga Choudhury wrote:

> Since /tmp is mounted across a network and /dev/shm is (always) local,
> /dev/shm seems to be the right place for shared memory transactions.
> If you create temporary files using mktemp is it being created in
> /dev/shm or /tmp?
> On Thu, Nov 3, 2011 at 11:50 AM, Bogdan Costescu <bcostescu_at_[hidden]> wrote:
>> On Thu, Nov 3, 2011 at 15:54, Blosch, Edwin L <edwin.l.blosch_at_[hidden]> wrote:
>>> - /dev/shm is 12 GB and has 755 permissions
>>> ...
>>> % ls –l output:
>>> drwxr-xr-x 2 root root 40 Oct 28 09:14 shm
>> This is your problem: it should be something like drwxrwxrwt. It might
>> depend on the distribution, f.e. the following show this to be a bug:
>> and surely you can find some more on the subject with your favorite
>> search engine. Another source could be a paranoid sysadmin who has
>> changed the default (most likely correct) setting the distribution
>> came with - not only OpenMPI but any application using shmem would be
>> affected..
>> Cheers,
>> Bogdan
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
> _______________________________________________
> users mailing list
> users_at_[hidden]