Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] MPI_Comm_spawn and oreted
From: Jerome BENOIT (ml.jgmbenoit_at_[hidden])
Date: 2009-04-16 13:31:31


Thanks for the info.

meanwhile I have set:

mpi_param_check = 0

in my system-wide configuation file on workers

and

mpi_param_check = 1

on the master.

Jerome

Ralph Castain wrote:
> Thanks! That does indeed help clarify.
>
> You should also then configure OMPI with
> --disable-per-user-config-files. MPI procs will automatically look at
> the default MCA parameter file, which is probably on your master node
> (wherever mpirun was executed). However, they also look at the user's
> home directory for any user default param file and/or binary modules. So
> the home directory will again be automounted, this time by the MPI procs.
>
> We created that option specifically to address the problem you describe.
> Hope it helps.
>
>
> On Apr 16, 2009, at 8:57 AM, Jerome BENOIT wrote:
>
>> Hi,
>>
>> thanks for the reply.
>>
>> Ralph Castain wrote:
>>> The orteds don't pass anything from MPI_Info to srun during a
>>> comm_spawn. What the orteds do is to chdir to the specified wdir
>>> before spawning the child process to ensure that the child has the
>>> correct working directory, then the orted changes back to its default
>>> working directory.
>>> The orted working directory is set by the base environment. So your
>>> slurm arguments cause *all* orteds to use the specified directory as
>>> their "home base". They will then use any given wdir keyval when they
>>> launch their respective child processes, as described above.
>>> As a side note, it isn't clear to me why you care about the orted's
>>> working directory. The orteds don't write anything there, or do
>>> anything with respect to their "home base" - so why would this
>>> matter? Or are you trying to specify the executable's path relative
>>> to where the orted is sitting?
>>
>>
>> Let be specific. My worker nodes are homeless: the /home directory is
>> automounted
>> (when needed) from the master node: orteds dont write anything, but
>> they keep it mounted !
>> The idea is to avoid this by specifying a local working directory.
>>
>> Jerome
>>
>>
>>
>>> On Apr 16, 2009, at 4:02 AM, Jerome BENOIT wrote:
>>>> Hi !
>>>>
>>>> finally I got it:
>>>> passing the mca key/value `"plm_slurm_args"/"--chdir /local/folder"'
>>>> does the trick.
>>>>
>>>> As a matter of fact, my code pass the MPI_Info key/value
>>>> `"wdir"/"/local/folder"'
>>>> to MPI_Comm_spawn as well: the working directories on the nodes of
>>>> the spawned programs
>>>> are `nodes:/local/folder' as expected, but the working directory of
>>>> the oreted_s
>>>> is the working directory of the parent program. My guess is that the
>>>> MPI_Info key/vale
>>>> may also be passed to `srun'.
>>>>
>>>> hth,
>>>> Jerome
>>>>
>>>>
>>>>
>>>> Jerome BENOIT wrote:
>>>>> Hello Again,
>>>>> Jerome BENOIT wrote:
>>>>>> Hello List,
>>>>>>
>>>>>> I have just noticed that, when MPI_Comm_spawn is used to launch
>>>>>> programs around,
>>>>>> oreted working directory on the nodes is the working directory of
>>>>>> the spawnning program:
>>>>>> can we ask to oreted to use an another directory ?
>>>>> Changing the working the directory via chdir before spawning with
>>>>> MPI_Comm_spawn
>>>>> changes nothing: the oreted working directory on the nodes seems to
>>>>> be imposed
>>>>> by something else. As run OMPI on top of SLURM, I guess this is
>>>>> related to SLURM.
>>>>> Jerome
>>>>>>
>>>>>> Thanks in advance,
>>>>>> Jerome _______________________________________________
>>>>>> users mailing list
>>>>>> users_at_[hidden]
>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>
>>>> _______________________________________________
>>>> users mailing list
>>>> users_at_[hidden]
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>