Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Ralph Castain (rhc_at_[hidden])
Date: 2007-07-17 18:47:13


No, the session directory is created in the tmpdir - we don't create
anything anywhere else, nor do we write any executables anywhere.

Unfortunately, we don't have more debug statements internal to that
function. I'll have to create a patch for you that will add some so we can
better understand why it is failing - will try to send it to you on Wed.

Just out of curiosity: although I know you have different arch's on your
nodes, the tests you are running are all executing on the same arch,
correct???

Ralph

On 7/17/07 4:06 PM, "Bill Johnstone" <beejstone3_at_[hidden]> wrote:

> I made sure the TMPDIR environment variable was set to /tmp for
> non-interactive logins, and got the same result as before.
>
> Also specifying the "-mca tmpdir_base /tmp" command-line options gave
> the same result as well.
>
> I made a mistake in my previous e-mail however -- the user home
> directories are also writable by each node (again, via NFS). /var and
> /tmp are the only unique-per-node writable directories. I'm assuming
> that by default, the session directory structure is created in the run
> directory, or the user's home directory, or something similar?
>
> /tmp and the home directories are both mounted nosuid, but are mounted
> exec. Does mpirun write/run a suid executable in any of these
> directories?
>
> Thank you.
>
> --- Ralph Castain <rhc_at_[hidden]> wrote:
>
>> Open MPI needs to create a temporary directory structure that we call
>> the
>> "session directory". This error is telling you that Open MPI was
>> unable to
>> create that directory, probably due to a permission issue.
>>
>> We decide on the root directory for the session directory using a
>> progression. You can direct where you want it to go by setting the
>> TMPDIR
>> environment variable, or (to set it just for us) using -mca
>> tmpdir_base foo
>> on the mpirun command (or you can set OMPI_MCA_tmpidir_base=foo in
>> your
>> environment), where "foo" is the root of your tmp directory you want
>> us to
>> use (e.g., /tmp).
>>
>> Hope that helps
>> Ralph
>>
>>
>>
>> On 7/17/07 3:09 PM, "Bill Johnstone" <beejstone3_at_[hidden]> wrote:
>>
>>> When I run with --debug-daemons, I get:
>>>
>>> <Begin mpirun output>
>>>
>>> [node5.x86-64:09920] [0,0,1] ORTE_ERROR_LOG: Error in file
>>> runtime/orte_init_stage1.c at line 626
>>>
>>
> --------------------------------------------------------------------------
>>> It looks like orte_init failed for some reason; your parallel
>> process
>>> is
>>> likely to abort. There are many reasons that a parallel process
>> can
>>> fail during orte_init; some of which are due to configuration or
>>> environment problems. This failure appears to be an internal
>> failure;
>>> here's some additional information (which may only be relevant to
>> an
>>> Open MPI developer):
>>>
>>> orte_session_dir failed
>>> --> Returned value -1 instead of ORTE_SUCCESS
>>>
>>>
>>
> --------------------------------------------------------------------------
>>> [node5.x86-64:09920] [0,0,1] ORTE_ERROR_LOG: Error in file
>>> runtime/orte_system_init.c at line 42
>>> [node5.x86-64:09920] [0,0,1] ORTE_ERROR_LOG: Error in file
>>> runtime/orte_init.c at line 52
>>> Open RTE was unable to initialize properly. The error occured
>> while
>>> attempting to orte_init(). Returned value -1 instead of
>> ORTE_SUCCESS.
>>>
>>> <End mpirun output>
>>>
>>> Where would you suggest I look next?
>>>
>>> Also, if it makes any difference, /usr/local is on a read-only
>> NFSROOT.
>>> Only /tmp and /var are writeable per-node.
>>>
>>> Thank you very much for your help so far.
>>>
>>> --- George Bosilca <bosilca_at_[hidden]> wrote:
>>>
>>>> Sorry. The --debug was supposed to be --debug-devel. But I suspect
>>>> that if you have a normal build then there will be not much
>> output.
>>>> However, --debug-daemons should give enough output so we can at
>> least
>>>>
>>>> have a starting point.
>>>>
>>>> george.
>>>>
>>>> On Jul 17, 2007, at 2:46 PM, Bill Johnstone wrote:
>>>>
>>>>> George Bosilca wrote:
>>>>>
>>>>>> You can start by adding --debug-daemons and --debug to your
>> mpirun
>>>>>> command line. This will generate a lot of output related to the
>>>>>> operations done internally by the launcher. If you send this
>>>> output
>>>>>> to the list we might be able to help you a little bit more.
>>>>>
>>>>> OK, I added those, but got a message about needing to supply a
>>>>> suitable
>>>>> debugger. If I supply the "--debugger gdb" argument, I just get
>>>>> dumped
>>>>> into gdb. I'm not sure what I need to do next to get the
>> launcher
>>>>> output you mentioned. My knowledge of gdb is pretty rudimentary.
>>>>
>>>>> Do I
>>>>> need to set mpirun as the executable, and the use the gdb "run"
>>>>> command
>>>>> with the mpirun arguments?
>>>>>
>>>>> Do I need to rebuild openmpi with --enable-debug?
>>>
>>>
>>>
>>>
>>>
>>
> ______________________________________________________________________________
>>> ______
>>> Luggage? GPS? Comic books?
>>> Check out fitting gifts for grads at Yahoo! Search
>>>
>>
> http://search.yahoo.com/search?fr=oni_on_mail&p=graduation+gifts&cs=bz
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>
>
>
>
> ______________________________________________________________________________
> ______
> Moody friends. Drama queens. Your life? Nope! - their life, your story. Play
> Sims Stories at Yahoo! Games.
> http://sims.yahoo.com/
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users