Thanks! That does indeed help clarify.
You should also then configure OMPI with --disable-per-user-config-
files. MPI procs will automatically look at the default MCA parameter
file, which is probably on your master node (wherever mpirun was
executed). However, they also look at the user's home directory for
any user default param file and/or binary modules. So the home
directory will again be automounted, this time by the MPI procs.
We created that option specifically to address the problem you
describe. Hope it helps.
On Apr 16, 2009, at 8:57 AM, Jerome BENOIT wrote:
> thanks for the reply.
> Ralph Castain wrote:
>> The orteds don't pass anything from MPI_Info to srun during a
>> comm_spawn. What the orteds do is to chdir to the specified wdir
>> before spawning the child process to ensure that the child has the
>> correct working directory, then the orted changes back to its
>> default working directory.
>> The orted working directory is set by the base environment. So your
>> slurm arguments cause *all* orteds to use the specified directory
>> as their "home base". They will then use any given wdir keyval when
>> they launch their respective child processes, as described above.
>> As a side note, it isn't clear to me why you care about the orted's
>> working directory. The orteds don't write anything there, or do
>> anything with respect to their "home base" - so why would this
>> matter? Or are you trying to specify the executable's path relative
>> to where the orted is sitting?
> Let be specific. My worker nodes are homeless: the /home directory
> is automounted
> (when needed) from the master node: orteds dont write anything, but
> they keep it mounted !
> The idea is to avoid this by specifying a local working directory.
>> On Apr 16, 2009, at 4:02 AM, Jerome BENOIT wrote:
>>> Hi !
>>> finally I got it:
>>> passing the mca key/value `"plm_slurm_args"/"--chdir /local/
>>> folder"' does the trick.
>>> As a matter of fact, my code pass the MPI_Info key/value `"wdir"/"/
>>> to MPI_Comm_spawn as well: the working directories on the nodes of
>>> the spawned programs
>>> are `nodes:/local/folder' as expected, but the working directory
>>> of the oreted_s
>>> is the working directory of the parent program. My guess is that
>>> the MPI_Info key/vale
>>> may also be passed to `srun'.
>>> Jerome BENOIT wrote:
>>>> Hello Again,
>>>> Jerome BENOIT wrote:
>>>>> Hello List,
>>>>> I have just noticed that, when MPI_Comm_spawn is used to launch
>>>>> programs around,
>>>>> oreted working directory on the nodes is the working directory
>>>>> of the spawnning program:
>>>>> can we ask to oreted to use an another directory ?
>>>> Changing the working the directory via chdir before spawning with
>>>> changes nothing: the oreted working directory on the nodes seems
>>>> to be imposed
>>>> by something else. As run OMPI on top of SLURM, I guess this is
>>>> related to SLURM.
>>>>> Thanks in advance,
>>>>> Jerome _______________________________________________
>>>>> users mailing list
>>> users mailing list
>> users mailing list
> users mailing list