Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI users] few Problems
From: Gus Correa (gus_at_[hidden])
Date: 2009-04-22 17:06:57


Hi Luis, list

To complement Jeff's recommendation,
see if this recipe to setup passwordless ssh connections helps.
If you use RSA keys instead of DSA, replace all "dsa" by "rsa":

http://www.sshkeychain.org/mirrors/SSH-with-Keys-HOWTO/SSH-with-Keys-HOWTO-4.html#ss4.3

I hope this helps.

Gus Correa
---------------------------------------------------------------------
Gustavo Correa
Lamont-Doherty Earth Observatory - Columbia University
Palisades, NY, 10964-8000 - USA
---------------------------------------------------------------------

Jeff Squyres wrote:
> It looks like you need to fix your password-less ssh problems first:
>
> > Permission denied, please try again.
> > AH72000_at_cluster-srv2's password:
>
> As I mentioned earlier, you need to be able to be able to run
>
> ssh cluster-srv2 uptime
>
> without being prompted for a password before Open MPI will work properly.
>
> If you're still having problems after fixing this, please send all the
> information from the "help" URL I sent earlier.
>
> Thanks!
>
>
> On Apr 22, 2009, at 3:24 PM, Luis Vitorio Cargnini wrote:
>
>> ok this is the debug information debug running on 5 nodes (trying at
>> least), the process is locked until now:
>>
>> each node is composed by two quad-core microprocessors.
>>
>> (don't finish), one node yet asked me the password. I have the home
>> partition mounted (the same) in all nodes. so login in cluster-
>> srv[0-4] is the same thing, I generated the

  key in each node, in
>> different files and added all to the authorized_keys, it should be
>> working.
>>
>> That is it, all help is welcome.
>>
>>
>> this is the code been executed:
>>
>> #include <stdio.h>
>> #include <mpi.h>
>>
>>
>> int main (int argc, char *argv[])
>> {
>> int rank, size;
>>
>> MPI_Init (&argc, &argv); /* starts MPI */
>> MPI_Comm_rank (MPI_COMM_WORLD, &rank); /* get current process
>> id */
>> MPI_Comm_size (MPI_COMM_WORLD, &size); /* get number of
>> processes */
>> printf( "Hello world from process %d of %d\n", rank, size );
>> MPI_Finalize();
>> return 0;
>> }
>>
>>
>> ------
>> debug:
>>
>> -bash-3.2$ mpirun -v -d -hostfile chosts -np 32 /export/cluster/home/
>> AH72000/mpi/hello
>>
>> [cluster-srv0:21606] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/0/0
>> [cluster-srv0:21606] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/0
>> [cluster-srv0:21606] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>> [cluster-srv0:21606] tmp: /tmp
>> [cluster-srv0:21606] mpirun: reset PATH: /export/cluster/appl/x86_64/
>> llvm/bin:/bin:/sbin:/export/cluster/appl/x86_64/llvm/bin:/usr/local/
>> llvm/bin:/usr/local/bin:/usr/bin:/usr/sbin:/home/GTI420/AH72000/oe/
>> bitbake/bin
>> [cluster-srv0:21606] mpirun: reset LD_LIBRARY_PATH: /export/cluster/
>> appl/x86_64/llvm/lib:/lib64:/lib:/export/cluster/appl/x86_64/llvm/lib:/
>> usr/lib64:/usr/lib:/usr/local/lib64:/usr/local/lib
>> AH72000_at_cluster-srv1's password: AH72000_at_cluster-srv2's password:
>> AH72000_at_cluster-srv3's password:
>> [cluster-srv1:07406] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/0/1
>> [cluster-srv1:07406] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/0
>> [cluster-srv1:07406] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>> [cluster-srv1:07406] tmp: /tmp
>>
>>
>> Permission denied, please try again.
>> AH72000_at_cluster-srv2's password:
>> [cluster-srv3:14230] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/0/3
>> [cluster-srv3:14230] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/0
>> [cluster-srv3:14230] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>> [cluster-srv3:14230] tmp: /tmp
>>
>> Permission denied, please try again.
>> AH72000_at_cluster-srv2's password:
>> [cluster-srv2:17092] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/0/2
>> [cluster-srv2:17092] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/0
>> [cluster-srv2:17092] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>> [cluster-srv2:17092] tmp: /tmp
>> [cluster-srv0:21606] [[35335,0],0] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv0:21606] [[35335,0],0] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv0:21606] [[35335,0],0] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv0:21606] [[35335,0],0] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv0:21606] [[35335,0],0] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv1:07406] [[35335,0],1] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv1:07406] [[35335,0],1] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv1:07406] [[35335,0],1] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv1:07406] [[35335,0],1] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv1:07406] [[35335,0],1] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv2:17092] [[35335,0],2] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv2:17092] [[35335,0],2] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv2:17092] [[35335,0],2] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv2:17092] [[35335,0],2] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv2:17092] [[35335,0],2] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv0:21611] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1/0
>> [cluster-srv0:21611] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1
>> [cluster-srv0:21611] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>> [cluster-srv0:21611] tmp: /tmp
>> [cluster-srv0:21613] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1/2
>> [cluster-srv0:21613] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1
>> [cluster-srv0:21613] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>> [cluster-srv0:21613] tmp: /tmp
>> [cluster-srv0:21612] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1/1
>> [cluster-srv0:21612] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1
>> [cluster-srv0:21612] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>> [cluster-srv0:21612] tmp: /tmp
>> [cluster-srv0:21614] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1/3
>> [cluster-srv0:21614] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1
>> [cluster-srv0:21614] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>> [cluster-srv0:21614] tmp: /tmp
>> [cluster-srv0:21615] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1/4
>> [cluster-srv0:21615] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1
>> [cluster-srv0:21615] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>> [cluster-srv0:21615] tmp: /tmp
>> [cluster-srv0:21616] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1/5
>> [cluster-srv0:21616] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1
>> [cluster-srv0:21616] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>> [cluster-srv0:21616] tmp: /tmp
>> [cluster-srv0:21617] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1/6
>> [cluster-srv0:21617] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1
>> [cluster-srv0:21617] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>> [cluster-srv0:21617] tmp: /tmp
>> [cluster-srv0:21618] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1/7
>> [cluster-srv0:21618] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv0_0/35335/1
>> [cluster-srv0:21618] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>> [cluster-srv0:21618] tmp: /tmp
>> [cluster-srv1:07408] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1/8
>> [cluster-srv1:07408] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1
>> [cluster-srv1:07408] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>> [cluster-srv1:07408] tmp: /tmp
>> [cluster-srv3:14230] [[35335,0],3] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv3:14230] [[35335,0],3] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv3:14230] [[35335,0],3] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv3:14230] [[35335,0],3] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv3:14230] [[35335,0],3] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv1:07409] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1/9
>> [cluster-srv1:07409] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1
>> [cluster-srv1:07409] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>> [cluster-srv1:07409] tmp: /tmp
>> [cluster-srv1:07410] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1/10
>> [cluster-srv1:07410] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1
>> [cluster-srv1:07410] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>> [cluster-srv1:07410] tmp: /tmp
>> [cluster-srv0:21606] Info: Setting up debugger process table for
>> applications
>> MPIR_being_debugged = 0
>> MPIR_debug_state = 1
>> MPIR_partial_attach_ok = 1
>> MPIR_i_am_starter = 0
>> MPIR_proctable_size = 32
>> MPIR_proctable:
>> (i, host, exe, pid) = (0, cluster-srv0, /export/cluster/home/
>> AH72000/mpi/hello, 21611)
>> (i, host, exe, pid) = (1, cluster-srv0, /export/cluster/home/
>> AH72000/mpi/hello, 21612)
>> (i, host, exe, pid) = (2, cluster-srv0, /export/cluster/home/
>> AH72000/mpi/hello, 21613)
>> (i, host, exe, pid) = (3, cluster-srv0, /export/cluster/home/
>> AH72000/mpi/hello, 21614)
>> (i, host, exe, pid) = (4, cluster-srv0, /export/cluster/home/
>> AH72000/mpi/hello, 21615)
>> (i, host, exe, pid) = (5, cluster-srv0, /export/cluster/home/
>> AH72000/mpi/hello, 21616)
>> (i, host, exe, pid) = (6, cluster-srv0, /export/cluster/home/
>> AH72000/mpi/hello, 21617)
>> (i, host, exe, pid) = (7, cluster-srv0, /export/cluster/home/
>> AH72000/mpi/hello, 21618)
>> (i, host, exe, pid) = (8, cluster-srv1, /export/cluster/home/
>> AH72000/mpi/hello, 7408)
>> (i, host, exe, pid) = (9, cluster-srv1, /export/cluster/home/
>> AH72000/mpi/hello, 7409)
>> (i, host, exe, pid) = (10, cluster-srv1, /export/cluster/home/
>> AH72000/mpi/hello, 7410)
>> (i, host, exe, pid) = (11, cluster-srv1, /export/cluster/home/
>> AH72000/mpi/hello, 7411)
>> (i, host, exe, pid) = (12, cluster-srv1, /export/cluster/home/
>> AH72000/mpi/hello, 7412)
>> (i, host, exe, pid) = (13, cluster-srv1, /export/cluster/home/
>> AH72000/mpi/hello, 7413)
>> (i, host, exe, pid) = (14, cluster-srv1, /export/cluster/home/
>> AH72000/mpi/hello, 7414)
>> (i, host, exe, pid) = (15, cluster-srv1, /export/cluster/home/
>> AH72000/mpi/hello, 7415)
>> (i, host, exe, pid) = (16, cluster-srv2, /export/cluster/home/
>> AH72000/mpi/hello, 17096)
>> (i, host, exe, pid) = (17, cluster-srv2, /export/cluster/home/
>> AH72000/mpi/hello, 17097)
>> (i, host, exe, pid) = (18, cluster-srv2, /export/cluster/home/
>> AH72000/mpi/hello, 17098)
>> (i, host, exe, pid) = (19, cluster-srv2, /export/cluster/home/
>> AH72000/mpi/hello, 17099)
>> (i, host, exe, pid) = (20, cluster-srv2, /export/cluster/home/
>> AH72000/mpi/hello, 17100)
>> (i, host, exe, pid) = (21, cluster-srv2, /export/cluster/home/
>> AH72000/mpi/hello, 17101)
>> (i, host, exe, pid) = (22, cluster-srv2, /export/cluster/home/
>> AH72000/mpi/hello, 17102)
>> (i, host, exe, pid) = (23, cluster-srv2, /export/cluster/home/
>> AH72000/mpi/hello, 17103)
>> (i, host, exe, pid) = (24, cluster-srv3, /export/cluster/home/
>> AH72000/mpi/hello, 14232)
>> (i, host, exe, pid) = (25, cluster-srv3, /export/cluster/home/
>> AH72000/mpi/hello, 14233)
>> (i, host, exe, pid) = (26, cluster-srv3, /export/cluster/home/
>> AH72000/mpi/hello, 14234)
>> (i, host, exe, pid) = (27, cluster-srv3, /export/cluster/home/
>> AH72000/mpi/hello, 14235)
>> (i, host, exe, pid) = (28, cluster-srv3, /export/cluster/home/
>> AH72000/mpi/hello, 14236)
>> (i, host, exe, pid) = (29, cluster-srv3, /export/cluster/home/
>> AH72000/mpi/hello, 14237)
>> (i, host, exe, pid) = (30, cluster-srv3, /export/cluster/home/
>> AH72000/mpi/hello, 14238)
>> (i, host, exe, pid) = (31, cluster-srv3, /export/cluster/home/
>> AH72000/mpi/hello, 14239)
>> [cluster-srv1:07412] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1/12
>> [cluster-srv1:07412] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1
>> [cluster-srv1:07412] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>> [cluster-srv1:07412] tmp: /tmp
>> [cluster-srv1:07411] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1/11
>> [cluster-srv1:07411] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1
>> [cluster-srv1:07411] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>> [cluster-srv1:07411] tmp: /tmp
>> [cluster-srv1:07414] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1/14
>> [cluster-srv1:07414] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1
>> [cluster-srv1:07414] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>> [cluster-srv1:07414] tmp: /tmp
>> [cluster-srv1:07415] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1/15
>> [cluster-srv1:07415] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1
>> [cluster-srv1:07415] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>> [cluster-srv1:07415] tmp: /tmp
>> [cluster-srv1:07413] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1/13
>> [cluster-srv1:07413] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv1_0/35335/1
>> [cluster-srv1:07413] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>> [cluster-srv1:07413] tmp: /tmp
>> [cluster-srv2:17096] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1/16
>> [cluster-srv2:17096] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1
>> [cluster-srv2:17096] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>> [cluster-srv2:17096] tmp: /tmp
>> [cluster-srv2:17097] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1/17
>> [cluster-srv2:17097] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1
>> [cluster-srv2:17097] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>> [cluster-srv2:17097] tmp: /tmp
>> [cluster-srv2:17099] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1/19
>> [cluster-srv2:17099] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1
>> [cluster-srv2:17099] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>> [cluster-srv2:17099] tmp: /tmp
>> [cluster-srv2:17098] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1/18
>> [cluster-srv2:17098] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1
>> [cluster-srv2:17098] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>> [cluster-srv2:17098] tmp: /tmp
>> [cluster-srv2:17100] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1/20
>> [cluster-srv2:17100] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1
>> [cluster-srv2:17100] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>> [cluster-srv2:17100] tmp: /tmp
>> [cluster-srv2:17102] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1/22
>> [cluster-srv2:17102] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1
>> [cluster-srv2:17102] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>> [cluster-srv2:17102] tmp: /tmp
>> [cluster-srv2:17103] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1/23
>> [cluster-srv2:17103] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1
>> [cluster-srv2:17103] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>> [cluster-srv2:17103] tmp: /tmp
>> [cluster-srv2:17101] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1/21
>> [cluster-srv2:17101] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv2_0/35335/1
>> [cluster-srv2:17101] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>> [cluster-srv2:17101] tmp: /tmp
>> [cluster-srv3:14232] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1/24
>> [cluster-srv3:14232] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1
>> [cluster-srv3:14232] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>> [cluster-srv3:14232] tmp: /tmp
>> [cluster-srv3:14235] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1/27
>> [cluster-srv3:14235] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1
>> [cluster-srv3:14235] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>> [cluster-srv3:14235] tmp: /tmp
>> [cluster-srv3:14233] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1/25
>> [cluster-srv3:14233] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1
>> [cluster-srv3:14233] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>> [cluster-srv3:14233] tmp: /tmp
>> [cluster-srv3:14234] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1/26
>> [cluster-srv3:14234] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1
>> [cluster-srv3:14234] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>> [cluster-srv3:14234] tmp: /tmp
>> [cluster-srv3:14236] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1/28
>> [cluster-srv3:14236] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1
>> [cluster-srv3:14236] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>> [cluster-srv3:14236] tmp: /tmp
>> [cluster-srv3:14237] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1/29
>> [cluster-srv3:14237] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1
>> [cluster-srv3:14237] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>> [cluster-srv3:14237] tmp: /tmp
>> [cluster-srv3:14238] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1/30
>> [cluster-srv3:14238] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1
>> [cluster-srv3:14238] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>> [cluster-srv3:14238] tmp: /tmp
>> [cluster-srv3:14239] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1/31
>> [cluster-srv3:14239] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>> srv3_0/35335/1
>> [cluster-srv3:14239] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>> [cluster-srv3:14239] tmp: /tmp
>>
>> [cluster-srv0:21611] [[35335,1],0] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv0:21611] [[35335,1],0] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv0:21611] [[35335,1],0] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv0:21611] [[35335,1],0] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv0:21611] [[35335,1],0] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv0:21613] [[35335,1],2] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv0:21613] [[35335,1],2] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv0:21613] [[35335,1],2] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv0:21613] [[35335,1],2] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv0:21613] [[35335,1],2] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv0:21614] [[35335,1],3] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv0:21614] [[35335,1],3] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv0:21614] [[35335,1],3] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv0:21614] [[35335,1],3] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv0:21614] [[35335,1],3] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv0:21612] [[35335,1],1] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv0:21612] [[35335,1],1] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv0:21612] [[35335,1],1] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv0:21612] [[35335,1],1] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv0:21612] [[35335,1],1] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv0:21615] [[35335,1],4] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv0:21615] [[35335,1],4] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv0:21615] [[35335,1],4] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv0:21615] [[35335,1],4] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv0:21615] [[35335,1],4] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv1:07408] [[35335,1],8] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv1:07408] [[35335,1],8] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv1:07408] [[35335,1],8] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv1:07408] [[35335,1],8] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv1:07408] [[35335,1],8] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv1:07409] [[35335,1],9] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv1:07409] [[35335,1],9] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv1:07409] [[35335,1],9] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv1:07409] [[35335,1],9] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv1:07409] [[35335,1],9] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv1:07410] [[35335,1],10] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv1:07410] [[35335,1],10] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv1:07410] [[35335,1],10] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv1:07410] [[35335,1],10] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv1:07410] [[35335,1],10] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv2:17096] [[35335,1],16] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv2:17096] [[35335,1],16] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv2:17096] [[35335,1],16] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv2:17096] [[35335,1],16] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv2:17096] [[35335,1],16] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv2:17097] [[35335,1],17] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv2:17097] [[35335,1],17] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv2:17097] [[35335,1],17] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv2:17097] [[35335,1],17] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv2:17097] [[35335,1],17] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv0:21617] [[35335,1],6] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv0:21617] [[35335,1],6] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv0:21617] [[35335,1],6] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv0:21617] [[35335,1],6] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv0:21617] [[35335,1],6] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv0:21616] [[35335,1],5] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv0:21616] [[35335,1],5] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv0:21616] [[35335,1],5] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv0:21616] [[35335,1],5] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv0:21616] [[35335,1],5] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv1:07411] [[35335,1],11] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv1:07411] [[35335,1],11] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv1:07411] [[35335,1],11] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv1:07411] [[35335,1],11] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv1:07411] [[35335,1],11] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv2:17099] [[35335,1],19] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv2:17099] [[35335,1],19] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv2:17099] [[35335,1],19] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv2:17099] [[35335,1],19] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv2:17099] [[35335,1],19] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv0:21618] [[35335,1],7] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv0:21618] [[35335,1],7] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv0:21618] [[35335,1],7] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv0:21618] [[35335,1],7] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv0:21618] [[35335,1],7] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv2:17098] [[35335,1],18] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv2:17098] [[35335,1],18] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv2:17098] [[35335,1],18] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv2:17098] [[35335,1],18] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv2:17098] [[35335,1],18] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv1:07412] [[35335,1],12] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv1:07412] [[35335,1],12] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv1:07412] [[35335,1],12] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv1:07412] [[35335,1],12] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv1:07412] [[35335,1],12] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv2:17102] [[35335,1],22] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv2:17102] [[35335,1],22] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv2:17102] [[35335,1],22] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv2:17102] [[35335,1],22] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv2:17102] [[35335,1],22] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv2:17100] [[35335,1],20] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv2:17100] [[35335,1],20] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv2:17100] [[35335,1],20] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv2:17100] [[35335,1],20] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv2:17100] [[35335,1],20] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv1:07415] [[35335,1],15] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv1:07415] [[35335,1],15] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv1:07415] [[35335,1],15] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv1:07415] [[35335,1],15] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv1:07415] [[35335,1],15] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv1:07414] [[35335,1],14] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv1:07414] [[35335,1],14] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv1:07414] [[35335,1],14] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv1:07414] [[35335,1],14] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv1:07414] [[35335,1],14] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv2:17101] [[35335,1],21] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv2:17101] [[35335,1],21] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv2:17101] [[35335,1],21] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv2:17101] [[35335,1],21] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv2:17101] [[35335,1],21] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv2:17103] [[35335,1],23] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv2:17103] [[35335,1],23] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv2:17103] [[35335,1],23] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv2:17103] [[35335,1],23] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv2:17103] [[35335,1],23] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv1:07413] [[35335,1],13] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv1:07413] [[35335,1],13] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv1:07413] [[35335,1],13] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv1:07413] [[35335,1],13] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv1:07413] [[35335,1],13] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv3:14232] [[35335,1],24] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv3:14232] [[35335,1],24] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv3:14232] [[35335,1],24] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv3:14232] [[35335,1],24] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv3:14232] [[35335,1],24] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv3:14235] [[35335,1],27] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv3:14235] [[35335,1],27] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv3:14235] [[35335,1],27] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv3:14235] [[35335,1],27] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv3:14235] [[35335,1],27] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv3:14233] [[35335,1],25] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv3:14233] [[35335,1],25] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv3:14233] [[35335,1],25] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv3:14233] [[35335,1],25] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv3:14233] [[35335,1],25] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv3:14234] [[35335,1],26] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv3:14234] [[35335,1],26] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv3:14234] [[35335,1],26] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv3:14234] [[35335,1],26] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv3:14234] [[35335,1],26] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv3:14236] [[35335,1],28] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv3:14236] [[35335,1],28] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv3:14236] [[35335,1],28] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv3:14236] [[35335,1],28] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv3:14236] [[35335,1],28] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv3:14237] [[35335,1],29] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv3:14237] [[35335,1],29] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv3:14237] [[35335,1],29] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv3:14237] [[35335,1],29] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv3:14237] [[35335,1],29] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv3:14239] [[35335,1],31] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv3:14239] [[35335,1],31] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv3:14239] [[35335,1],31] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv3:14239] [[35335,1],31] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv3:14239] [[35335,1],31] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> [cluster-srv3:14238] [[35335,1],30] node[0].name cluster-srv0 daemon 0
>> arch ffc91200
>> [cluster-srv3:14238] [[35335,1],30] node[1].name cluster-srv1 daemon 1
>> arch ffc91200
>> [cluster-srv3:14238] [[35335,1],30] node[2].name cluster-srv2 daemon 2
>> arch ffc91200
>> [cluster-srv3:14238] [[35335,1],30] node[3].name cluster-srv3 daemon 3
>> arch ffc91200
>> [cluster-srv3:14238] [[35335,1],30] node[4].name cluster-srv4 daemon
>> INVALID arch ffc91200
>> Hello world from process 0 of 32
>> Hello world from process 4 of 32
>> Hello world from process 1 of 32
>> Hello world from process 7 of 32
>> Hello world from process 6 of 32
>> Hello world from process 24 of 32
>> Hello world from process 5 of 32
>> Hello world from process 27 of 32
>> Hello world from process 25 of 32
>> Hello world from process 21 of 32
>> Hello world from process 20 of 32
>> Hello world from process 18 of 32
>> Hello world from process 10 of 32
>> Hello world from process 16 of 32
>> Hello world from process 8 of 32
>> Hello world from process 12 of 32
>> Hello world from process 22 of 32
>> Hello world from process 13 of 32
>> Hello world from process 11 of 32
>> Hello world from process 14 of 32
>> Hello world from process 31 of 32
>> Hello world from process 19 of 32
>> Hello world from process 17 of 32
>> Hello world from process 26 of 32
>> Hello world from process 29 of 32
>> Hello world from process 9 of 32
>> Hello world from process 30 of 32
>> Hello world from process 2 of 32
>> Hello world from process 23 of 32
>> Hello world from process 3 of 32
>> Hello world from process 15 of 32
>> Hello world from process 28 of 32
>> [cluster-srv0:21615] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07409] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17099] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv0:21606] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14233] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17098] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17097] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17096] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv0:21617] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07415] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07414] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv0:21606] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07413] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14237] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv0:21613] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17101] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17102] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17100] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv0:21606] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14234] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14235] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17103] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07412] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07408] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07411] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14238] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv0:21616] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv0:21614] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14236] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14239] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv0:21606] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv0:21606] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14232] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07410] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv0:21612] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv0:21611] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv0:21606] sess_dir_finalize: proc session dir not empty -
>> leaving
>> [cluster-srv0:21606] sess_dir_finalize: proc session dir not empty -
>> leaving
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> Le 09-04-22 à 14:10, Jeff Squyres a écrit :
>>
>> > This isn't really enough information for us to help you. Can you
>> > send all the information here:
>> >
>> > http://www.open-mpi.org/community/help/
>> >
>> > Thanks.
>> >
>> >
>> > On Apr 21, 2009, at 10:34 AM, Luis Vitorio Cargnini wrote:
>> >
>> >> Hi,
>> >> Please someone can answer me which can be this problem ?
>> >> daemon INVALID arch ffc91200
>> >>
>> >>
>> >>
>> >>
>> >> the debug output:
>> >> [[41704,1],14] node[4].name cluster-srv4 daemon INVALID arch ffc91200
>> >> [cluster-srv3:09684] [[41704,1],13] node[0].name cluster-srv0
>> >> daemon 0
>> >> arch ffc91200
>> >> [cluster-srv3:09684] [[41704,1],13] node[1].name cluster-srv1
>> >> daemon 1
>> >> arch ffc91200
>> >> [cluster-srv3:09684] [[41704,1],13] node[2].name cluster-srv2
>> >> daemon 2
>> >> arch ffc91200
>> >> [cluster-srv3:09684] [[41704,1],13] node[3].name cluster-srv3
>> >> daemon 3
>> >> arch ffc91200
>> >> [cluster-srv3:09684] [[41704,1],13] node[4].name cluster-srv4 daemon
>> >> INVALID arch ffc91200
>> >>
>> >> ORTE_ERROR_LOG: A message is attempting to be sent to a process whose
>> >> contact information is unknown in file rml_oob_send.c at line 105
>> >> <PGP.sig>_______________________________________________
>> >> users mailing list
>> >> users_at_[hidden]
>> >> http://www.open-mpi.org/mailman/listinfo.cgi/users
>> >
>> >
>> > --
>> > Jeff Squyres
>> > Cisco Systems
>> >
>> > _______________________________________________
>> > users mailing list
>> > users_at_[hidden]
>> > http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>> <PGP.sig>_______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>