Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] few Problems
From: Luis Vitorio Cargnini (lvcargnini_at_[hidden])
Date: 2009-04-22 17:26:47


thank you all, I'll try to fix this ASAP, after I'll make a new test
round than I answer back, Thanks you all until here.

Le 09-04-22 à 17:06, Gus Correa a écrit :

> Hi Luis, list
>
> To complement Jeff's recommendation,
> see if this recipe to setup passwordless ssh connections helps.
> If you use RSA keys instead of DSA, replace all "dsa" by "rsa":
>
> http://www.sshkeychain.org/mirrors/SSH-with-Keys-HOWTO/SSH-with-Keys-HOWTO-4.html#ss4.3
>
> I hope this helps.
>
> Gus Correa
> ---------------------------------------------------------------------
> Gustavo Correa
> Lamont-Doherty Earth Observatory - Columbia University
> Palisades, NY, 10964-8000 - USA
> ---------------------------------------------------------------------
>
>
> Jeff Squyres wrote:
>> It looks like you need to fix your password-less ssh problems first:
>> > Permission denied, please try again.
>> > AH72000_at_cluster-srv2's password:
>> As I mentioned earlier, you need to be able to be able to run
>> ssh cluster-srv2 uptime
>> without being prompted for a password before Open MPI will work
>> properly.
>> If you're still having problems after fixing this, please send all
>> the information from the "help" URL I sent earlier.
>> Thanks!
>> On Apr 22, 2009, at 3:24 PM, Luis Vitorio Cargnini wrote:
>>> ok this is the debug information debug running on 5 nodes (trying at
>>> least), the process is locked until now:
>>>
>>> each node is composed by two quad-core microprocessors.
>>>
>>> (don't finish), one node yet asked me the password. I have the home
>>> partition mounted (the same) in all nodes. so login in cluster-
>>> srv[0-4] is the same thing, I generated the
>
> key in each node, in
>>> different files and added all to the authorized_keys, it should be
>>> working.
>>>
>>> That is it, all help is welcome.
>>>
>>>
>>> this is the code been executed:
>>>
>>> #include <stdio.h>
>>> #include <mpi.h>
>>>
>>>
>>> int main (int argc, char *argv[])
>>> {
>>> int rank, size;
>>>
>>> MPI_Init (&argc, &argv); /* starts MPI */
>>> MPI_Comm_rank (MPI_COMM_WORLD, &rank); /* get current
>>> process id */
>>> MPI_Comm_size (MPI_COMM_WORLD, &size); /* get number of
>>> processes */
>>> printf( "Hello world from process %d of %d\n", rank, size );
>>> MPI_Finalize();
>>> return 0;
>>> }
>>>
>>>
>>> ------
>>> debug:
>>>
>>> -bash-3.2$ mpirun -v -d -hostfile chosts -np 32 /export/cluster/
>>> home/
>>> AH72000/mpi/hello
>>>
>>> [cluster-srv0:21606] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/0/0
>>> [cluster-srv0:21606] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/0
>>> [cluster-srv0:21606] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>>> [cluster-srv0:21606] tmp: /tmp
>>> [cluster-srv0:21606] mpirun: reset PATH: /export/cluster/appl/
>>> x86_64/
>>> llvm/bin:/bin:/sbin:/export/cluster/appl/x86_64/llvm/bin:/usr/local/
>>> llvm/bin:/usr/local/bin:/usr/bin:/usr/sbin:/home/GTI420/AH72000/oe/
>>> bitbake/bin
>>> [cluster-srv0:21606] mpirun: reset LD_LIBRARY_PATH: /export/cluster/
>>> appl/x86_64/llvm/lib:/lib64:/lib:/export/cluster/appl/x86_64/llvm/
>>> lib:/
>>> usr/lib64:/usr/lib:/usr/local/lib64:/usr/local/lib
>>> AH72000_at_cluster-srv1's password: AH72000_at_cluster-srv2's password:
>>> AH72000_at_cluster-srv3's password:
>>> [cluster-srv1:07406] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/0/1
>>> [cluster-srv1:07406] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/0
>>> [cluster-srv1:07406] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>>> [cluster-srv1:07406] tmp: /tmp
>>>
>>>
>>> Permission denied, please try again.
>>> AH72000_at_cluster-srv2's password:
>>> [cluster-srv3:14230] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/0/3
>>> [cluster-srv3:14230] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/0
>>> [cluster-srv3:14230] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>>> [cluster-srv3:14230] tmp: /tmp
>>>
>>> Permission denied, please try again.
>>> AH72000_at_cluster-srv2's password:
>>> [cluster-srv2:17092] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/0/2
>>> [cluster-srv2:17092] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/0
>>> [cluster-srv2:17092] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>>> [cluster-srv2:17092] tmp: /tmp
>>> [cluster-srv0:21606] [[35335,0],0] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv0:21606] [[35335,0],0] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv0:21606] [[35335,0],0] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv0:21606] [[35335,0],0] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv0:21606] [[35335,0],0] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv1:07406] [[35335,0],1] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv1:07406] [[35335,0],1] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv1:07406] [[35335,0],1] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv1:07406] [[35335,0],1] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv1:07406] [[35335,0],1] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv2:17092] [[35335,0],2] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv2:17092] [[35335,0],2] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv2:17092] [[35335,0],2] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv2:17092] [[35335,0],2] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv2:17092] [[35335,0],2] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv0:21611] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1/0
>>> [cluster-srv0:21611] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1
>>> [cluster-srv0:21611] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>>> [cluster-srv0:21611] tmp: /tmp
>>> [cluster-srv0:21613] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1/2
>>> [cluster-srv0:21613] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1
>>> [cluster-srv0:21613] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>>> [cluster-srv0:21613] tmp: /tmp
>>> [cluster-srv0:21612] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1/1
>>> [cluster-srv0:21612] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1
>>> [cluster-srv0:21612] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>>> [cluster-srv0:21612] tmp: /tmp
>>> [cluster-srv0:21614] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1/3
>>> [cluster-srv0:21614] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1
>>> [cluster-srv0:21614] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>>> [cluster-srv0:21614] tmp: /tmp
>>> [cluster-srv0:21615] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1/4
>>> [cluster-srv0:21615] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1
>>> [cluster-srv0:21615] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>>> [cluster-srv0:21615] tmp: /tmp
>>> [cluster-srv0:21616] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1/5
>>> [cluster-srv0:21616] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1
>>> [cluster-srv0:21616] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>>> [cluster-srv0:21616] tmp: /tmp
>>> [cluster-srv0:21617] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1/6
>>> [cluster-srv0:21617] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1
>>> [cluster-srv0:21617] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>>> [cluster-srv0:21617] tmp: /tmp
>>> [cluster-srv0:21618] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1/7
>>> [cluster-srv0:21618] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv0_0/35335/1
>>> [cluster-srv0:21618] top: openmpi-sessions-AH72000_at_cluster-srv0_0
>>> [cluster-srv0:21618] tmp: /tmp
>>> [cluster-srv1:07408] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1/8
>>> [cluster-srv1:07408] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1
>>> [cluster-srv1:07408] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>>> [cluster-srv1:07408] tmp: /tmp
>>> [cluster-srv3:14230] [[35335,0],3] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv3:14230] [[35335,0],3] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv3:14230] [[35335,0],3] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv3:14230] [[35335,0],3] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv3:14230] [[35335,0],3] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv1:07409] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1/9
>>> [cluster-srv1:07409] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1
>>> [cluster-srv1:07409] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>>> [cluster-srv1:07409] tmp: /tmp
>>> [cluster-srv1:07410] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1/10
>>> [cluster-srv1:07410] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1
>>> [cluster-srv1:07410] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>>> [cluster-srv1:07410] tmp: /tmp
>>> [cluster-srv0:21606] Info: Setting up debugger process table for
>>> applications
>>> MPIR_being_debugged = 0
>>> MPIR_debug_state = 1
>>> MPIR_partial_attach_ok = 1
>>> MPIR_i_am_starter = 0
>>> MPIR_proctable_size = 32
>>> MPIR_proctable:
>>> (i, host, exe, pid) = (0, cluster-srv0, /export/cluster/home/
>>> AH72000/mpi/hello, 21611)
>>> (i, host, exe, pid) = (1, cluster-srv0, /export/cluster/home/
>>> AH72000/mpi/hello, 21612)
>>> (i, host, exe, pid) = (2, cluster-srv0, /export/cluster/home/
>>> AH72000/mpi/hello, 21613)
>>> (i, host, exe, pid) = (3, cluster-srv0, /export/cluster/home/
>>> AH72000/mpi/hello, 21614)
>>> (i, host, exe, pid) = (4, cluster-srv0, /export/cluster/home/
>>> AH72000/mpi/hello, 21615)
>>> (i, host, exe, pid) = (5, cluster-srv0, /export/cluster/home/
>>> AH72000/mpi/hello, 21616)
>>> (i, host, exe, pid) = (6, cluster-srv0, /export/cluster/home/
>>> AH72000/mpi/hello, 21617)
>>> (i, host, exe, pid) = (7, cluster-srv0, /export/cluster/home/
>>> AH72000/mpi/hello, 21618)
>>> (i, host, exe, pid) = (8, cluster-srv1, /export/cluster/home/
>>> AH72000/mpi/hello, 7408)
>>> (i, host, exe, pid) = (9, cluster-srv1, /export/cluster/home/
>>> AH72000/mpi/hello, 7409)
>>> (i, host, exe, pid) = (10, cluster-srv1, /export/cluster/home/
>>> AH72000/mpi/hello, 7410)
>>> (i, host, exe, pid) = (11, cluster-srv1, /export/cluster/home/
>>> AH72000/mpi/hello, 7411)
>>> (i, host, exe, pid) = (12, cluster-srv1, /export/cluster/home/
>>> AH72000/mpi/hello, 7412)
>>> (i, host, exe, pid) = (13, cluster-srv1, /export/cluster/home/
>>> AH72000/mpi/hello, 7413)
>>> (i, host, exe, pid) = (14, cluster-srv1, /export/cluster/home/
>>> AH72000/mpi/hello, 7414)
>>> (i, host, exe, pid) = (15, cluster-srv1, /export/cluster/home/
>>> AH72000/mpi/hello, 7415)
>>> (i, host, exe, pid) = (16, cluster-srv2, /export/cluster/home/
>>> AH72000/mpi/hello, 17096)
>>> (i, host, exe, pid) = (17, cluster-srv2, /export/cluster/home/
>>> AH72000/mpi/hello, 17097)
>>> (i, host, exe, pid) = (18, cluster-srv2, /export/cluster/home/
>>> AH72000/mpi/hello, 17098)
>>> (i, host, exe, pid) = (19, cluster-srv2, /export/cluster/home/
>>> AH72000/mpi/hello, 17099)
>>> (i, host, exe, pid) = (20, cluster-srv2, /export/cluster/home/
>>> AH72000/mpi/hello, 17100)
>>> (i, host, exe, pid) = (21, cluster-srv2, /export/cluster/home/
>>> AH72000/mpi/hello, 17101)
>>> (i, host, exe, pid) = (22, cluster-srv2, /export/cluster/home/
>>> AH72000/mpi/hello, 17102)
>>> (i, host, exe, pid) = (23, cluster-srv2, /export/cluster/home/
>>> AH72000/mpi/hello, 17103)
>>> (i, host, exe, pid) = (24, cluster-srv3, /export/cluster/home/
>>> AH72000/mpi/hello, 14232)
>>> (i, host, exe, pid) = (25, cluster-srv3, /export/cluster/home/
>>> AH72000/mpi/hello, 14233)
>>> (i, host, exe, pid) = (26, cluster-srv3, /export/cluster/home/
>>> AH72000/mpi/hello, 14234)
>>> (i, host, exe, pid) = (27, cluster-srv3, /export/cluster/home/
>>> AH72000/mpi/hello, 14235)
>>> (i, host, exe, pid) = (28, cluster-srv3, /export/cluster/home/
>>> AH72000/mpi/hello, 14236)
>>> (i, host, exe, pid) = (29, cluster-srv3, /export/cluster/home/
>>> AH72000/mpi/hello, 14237)
>>> (i, host, exe, pid) = (30, cluster-srv3, /export/cluster/home/
>>> AH72000/mpi/hello, 14238)
>>> (i, host, exe, pid) = (31, cluster-srv3, /export/cluster/home/
>>> AH72000/mpi/hello, 14239)
>>> [cluster-srv1:07412] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1/12
>>> [cluster-srv1:07412] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1
>>> [cluster-srv1:07412] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>>> [cluster-srv1:07412] tmp: /tmp
>>> [cluster-srv1:07411] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1/11
>>> [cluster-srv1:07411] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1
>>> [cluster-srv1:07411] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>>> [cluster-srv1:07411] tmp: /tmp
>>> [cluster-srv1:07414] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1/14
>>> [cluster-srv1:07414] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1
>>> [cluster-srv1:07414] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>>> [cluster-srv1:07414] tmp: /tmp
>>> [cluster-srv1:07415] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1/15
>>> [cluster-srv1:07415] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1
>>> [cluster-srv1:07415] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>>> [cluster-srv1:07415] tmp: /tmp
>>> [cluster-srv1:07413] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1/13
>>> [cluster-srv1:07413] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv1_0/35335/1
>>> [cluster-srv1:07413] top: openmpi-sessions-AH72000_at_cluster-srv1_0
>>> [cluster-srv1:07413] tmp: /tmp
>>> [cluster-srv2:17096] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1/16
>>> [cluster-srv2:17096] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1
>>> [cluster-srv2:17096] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>>> [cluster-srv2:17096] tmp: /tmp
>>> [cluster-srv2:17097] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1/17
>>> [cluster-srv2:17097] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1
>>> [cluster-srv2:17097] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>>> [cluster-srv2:17097] tmp: /tmp
>>> [cluster-srv2:17099] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1/19
>>> [cluster-srv2:17099] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1
>>> [cluster-srv2:17099] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>>> [cluster-srv2:17099] tmp: /tmp
>>> [cluster-srv2:17098] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1/18
>>> [cluster-srv2:17098] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1
>>> [cluster-srv2:17098] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>>> [cluster-srv2:17098] tmp: /tmp
>>> [cluster-srv2:17100] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1/20
>>> [cluster-srv2:17100] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1
>>> [cluster-srv2:17100] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>>> [cluster-srv2:17100] tmp: /tmp
>>> [cluster-srv2:17102] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1/22
>>> [cluster-srv2:17102] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1
>>> [cluster-srv2:17102] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>>> [cluster-srv2:17102] tmp: /tmp
>>> [cluster-srv2:17103] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1/23
>>> [cluster-srv2:17103] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1
>>> [cluster-srv2:17103] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>>> [cluster-srv2:17103] tmp: /tmp
>>> [cluster-srv2:17101] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1/21
>>> [cluster-srv2:17101] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv2_0/35335/1
>>> [cluster-srv2:17101] top: openmpi-sessions-AH72000_at_cluster-srv2_0
>>> [cluster-srv2:17101] tmp: /tmp
>>> [cluster-srv3:14232] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1/24
>>> [cluster-srv3:14232] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1
>>> [cluster-srv3:14232] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>>> [cluster-srv3:14232] tmp: /tmp
>>> [cluster-srv3:14235] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1/27
>>> [cluster-srv3:14235] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1
>>> [cluster-srv3:14235] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>>> [cluster-srv3:14235] tmp: /tmp
>>> [cluster-srv3:14233] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1/25
>>> [cluster-srv3:14233] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1
>>> [cluster-srv3:14233] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>>> [cluster-srv3:14233] tmp: /tmp
>>> [cluster-srv3:14234] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1/26
>>> [cluster-srv3:14234] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1
>>> [cluster-srv3:14234] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>>> [cluster-srv3:14234] tmp: /tmp
>>> [cluster-srv3:14236] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1/28
>>> [cluster-srv3:14236] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1
>>> [cluster-srv3:14236] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>>> [cluster-srv3:14236] tmp: /tmp
>>> [cluster-srv3:14237] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1/29
>>> [cluster-srv3:14237] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1
>>> [cluster-srv3:14237] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>>> [cluster-srv3:14237] tmp: /tmp
>>> [cluster-srv3:14238] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1/30
>>> [cluster-srv3:14238] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1
>>> [cluster-srv3:14238] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>>> [cluster-srv3:14238] tmp: /tmp
>>> [cluster-srv3:14239] procdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1/31
>>> [cluster-srv3:14239] jobdir: /tmp/openmpi-sessions-AH72000_at_cluster-
>>> srv3_0/35335/1
>>> [cluster-srv3:14239] top: openmpi-sessions-AH72000_at_cluster-srv3_0
>>> [cluster-srv3:14239] tmp: /tmp
>>>
>>> [cluster-srv0:21611] [[35335,1],0] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv0:21611] [[35335,1],0] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv0:21611] [[35335,1],0] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv0:21611] [[35335,1],0] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv0:21611] [[35335,1],0] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv0:21613] [[35335,1],2] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv0:21613] [[35335,1],2] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv0:21613] [[35335,1],2] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv0:21613] [[35335,1],2] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv0:21613] [[35335,1],2] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv0:21614] [[35335,1],3] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv0:21614] [[35335,1],3] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv0:21614] [[35335,1],3] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv0:21614] [[35335,1],3] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv0:21614] [[35335,1],3] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv0:21612] [[35335,1],1] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv0:21612] [[35335,1],1] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv0:21612] [[35335,1],1] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv0:21612] [[35335,1],1] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv0:21612] [[35335,1],1] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv0:21615] [[35335,1],4] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv0:21615] [[35335,1],4] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv0:21615] [[35335,1],4] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv0:21615] [[35335,1],4] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv0:21615] [[35335,1],4] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv1:07408] [[35335,1],8] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv1:07408] [[35335,1],8] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv1:07408] [[35335,1],8] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv1:07408] [[35335,1],8] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv1:07408] [[35335,1],8] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv1:07409] [[35335,1],9] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv1:07409] [[35335,1],9] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv1:07409] [[35335,1],9] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv1:07409] [[35335,1],9] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv1:07409] [[35335,1],9] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv1:07410] [[35335,1],10] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv1:07410] [[35335,1],10] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv1:07410] [[35335,1],10] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv1:07410] [[35335,1],10] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv1:07410] [[35335,1],10] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv2:17096] [[35335,1],16] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv2:17096] [[35335,1],16] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv2:17096] [[35335,1],16] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv2:17096] [[35335,1],16] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv2:17096] [[35335,1],16] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv2:17097] [[35335,1],17] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv2:17097] [[35335,1],17] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv2:17097] [[35335,1],17] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv2:17097] [[35335,1],17] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv2:17097] [[35335,1],17] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv0:21617] [[35335,1],6] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv0:21617] [[35335,1],6] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv0:21617] [[35335,1],6] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv0:21617] [[35335,1],6] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv0:21617] [[35335,1],6] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv0:21616] [[35335,1],5] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv0:21616] [[35335,1],5] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv0:21616] [[35335,1],5] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv0:21616] [[35335,1],5] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv0:21616] [[35335,1],5] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv1:07411] [[35335,1],11] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv1:07411] [[35335,1],11] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv1:07411] [[35335,1],11] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv1:07411] [[35335,1],11] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv1:07411] [[35335,1],11] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv2:17099] [[35335,1],19] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv2:17099] [[35335,1],19] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv2:17099] [[35335,1],19] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv2:17099] [[35335,1],19] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv2:17099] [[35335,1],19] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv0:21618] [[35335,1],7] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv0:21618] [[35335,1],7] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv0:21618] [[35335,1],7] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv0:21618] [[35335,1],7] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv0:21618] [[35335,1],7] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv2:17098] [[35335,1],18] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv2:17098] [[35335,1],18] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv2:17098] [[35335,1],18] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv2:17098] [[35335,1],18] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv2:17098] [[35335,1],18] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv1:07412] [[35335,1],12] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv1:07412] [[35335,1],12] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv1:07412] [[35335,1],12] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv1:07412] [[35335,1],12] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv1:07412] [[35335,1],12] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv2:17102] [[35335,1],22] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv2:17102] [[35335,1],22] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv2:17102] [[35335,1],22] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv2:17102] [[35335,1],22] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv2:17102] [[35335,1],22] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv2:17100] [[35335,1],20] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv2:17100] [[35335,1],20] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv2:17100] [[35335,1],20] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv2:17100] [[35335,1],20] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv2:17100] [[35335,1],20] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv1:07415] [[35335,1],15] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv1:07415] [[35335,1],15] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv1:07415] [[35335,1],15] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv1:07415] [[35335,1],15] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv1:07415] [[35335,1],15] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv1:07414] [[35335,1],14] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv1:07414] [[35335,1],14] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv1:07414] [[35335,1],14] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv1:07414] [[35335,1],14] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv1:07414] [[35335,1],14] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv2:17101] [[35335,1],21] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv2:17101] [[35335,1],21] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv2:17101] [[35335,1],21] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv2:17101] [[35335,1],21] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv2:17101] [[35335,1],21] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv2:17103] [[35335,1],23] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv2:17103] [[35335,1],23] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv2:17103] [[35335,1],23] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv2:17103] [[35335,1],23] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv2:17103] [[35335,1],23] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv1:07413] [[35335,1],13] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv1:07413] [[35335,1],13] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv1:07413] [[35335,1],13] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv1:07413] [[35335,1],13] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv1:07413] [[35335,1],13] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv3:14232] [[35335,1],24] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv3:14232] [[35335,1],24] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv3:14232] [[35335,1],24] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv3:14232] [[35335,1],24] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv3:14232] [[35335,1],24] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv3:14235] [[35335,1],27] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv3:14235] [[35335,1],27] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv3:14235] [[35335,1],27] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv3:14235] [[35335,1],27] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv3:14235] [[35335,1],27] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv3:14233] [[35335,1],25] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv3:14233] [[35335,1],25] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv3:14233] [[35335,1],25] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv3:14233] [[35335,1],25] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv3:14233] [[35335,1],25] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv3:14234] [[35335,1],26] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv3:14234] [[35335,1],26] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv3:14234] [[35335,1],26] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv3:14234] [[35335,1],26] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv3:14234] [[35335,1],26] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv3:14236] [[35335,1],28] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv3:14236] [[35335,1],28] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv3:14236] [[35335,1],28] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv3:14236] [[35335,1],28] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv3:14236] [[35335,1],28] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv3:14237] [[35335,1],29] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv3:14237] [[35335,1],29] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv3:14237] [[35335,1],29] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv3:14237] [[35335,1],29] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv3:14237] [[35335,1],29] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv3:14239] [[35335,1],31] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv3:14239] [[35335,1],31] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv3:14239] [[35335,1],31] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv3:14239] [[35335,1],31] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv3:14239] [[35335,1],31] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> [cluster-srv3:14238] [[35335,1],30] node[0].name cluster-srv0
>>> daemon 0
>>> arch ffc91200
>>> [cluster-srv3:14238] [[35335,1],30] node[1].name cluster-srv1
>>> daemon 1
>>> arch ffc91200
>>> [cluster-srv3:14238] [[35335,1],30] node[2].name cluster-srv2
>>> daemon 2
>>> arch ffc91200
>>> [cluster-srv3:14238] [[35335,1],30] node[3].name cluster-srv3
>>> daemon 3
>>> arch ffc91200
>>> [cluster-srv3:14238] [[35335,1],30] node[4].name cluster-srv4 daemon
>>> INVALID arch ffc91200
>>> Hello world from process 0 of 32
>>> Hello world from process 4 of 32
>>> Hello world from process 1 of 32
>>> Hello world from process 7 of 32
>>> Hello world from process 6 of 32
>>> Hello world from process 24 of 32
>>> Hello world from process 5 of 32
>>> Hello world from process 27 of 32
>>> Hello world from process 25 of 32
>>> Hello world from process 21 of 32
>>> Hello world from process 20 of 32
>>> Hello world from process 18 of 32
>>> Hello world from process 10 of 32
>>> Hello world from process 16 of 32
>>> Hello world from process 8 of 32
>>> Hello world from process 12 of 32
>>> Hello world from process 22 of 32
>>> Hello world from process 13 of 32
>>> Hello world from process 11 of 32
>>> Hello world from process 14 of 32
>>> Hello world from process 31 of 32
>>> Hello world from process 19 of 32
>>> Hello world from process 17 of 32
>>> Hello world from process 26 of 32
>>> Hello world from process 29 of 32
>>> Hello world from process 9 of 32
>>> Hello world from process 30 of 32
>>> Hello world from process 2 of 32
>>> Hello world from process 23 of 32
>>> Hello world from process 3 of 32
>>> Hello world from process 15 of 32
>>> Hello world from process 28 of 32
>>> [cluster-srv0:21615] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07409] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17099] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv0:21606] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14233] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17098] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17097] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17096] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv0:21617] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07415] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07414] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv0:21606] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07413] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14237] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv0:21613] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17101] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17102] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17100] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv0:21606] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14234] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14235] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17103] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv2:17092] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07412] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07408] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07411] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14238] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv0:21616] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv0:21614] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14236] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14239] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv0:21606] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv0:21606] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14232] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07410] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv3:14230] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv0:21612] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv1:07406] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv0:21611] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv0:21606] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>> [cluster-srv0:21606] sess_dir_finalize: proc session dir not empty -
>>> leaving
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> Le 09-04-22 à 14:10, Jeff Squyres a écrit :
>>>
>>> > This isn't really enough information for us to help you. Can you
>>> > send all the information here:
>>> >
>>> > http://www.open-mpi.org/community/help/
>>> >
>>> > Thanks.
>>> >
>>> >
>>> > On Apr 21, 2009, at 10:34 AM, Luis Vitorio Cargnini wrote:
>>> >
>>> >> Hi,
>>> >> Please someone can answer me which can be this problem ?
>>> >> daemon INVALID arch ffc91200
>>> >>
>>> >>
>>> >>
>>> >>
>>> >> the debug output:
>>> >> [[41704,1],14] node[4].name cluster-srv4 daemon INVALID arch
>>> ffc91200
>>> >> [cluster-srv3:09684] [[41704,1],13] node[0].name cluster-srv0
>>> >> daemon 0
>>> >> arch ffc91200
>>> >> [cluster-srv3:09684] [[41704,1],13] node[1].name cluster-srv1
>>> >> daemon 1
>>> >> arch ffc91200
>>> >> [cluster-srv3:09684] [[41704,1],13] node[2].name cluster-srv2
>>> >> daemon 2
>>> >> arch ffc91200
>>> >> [cluster-srv3:09684] [[41704,1],13] node[3].name cluster-srv3
>>> >> daemon 3
>>> >> arch ffc91200
>>> >> [cluster-srv3:09684] [[41704,1],13] node[4].name cluster-srv4
>>> daemon
>>> >> INVALID arch ffc91200
>>> >>
>>> >> ORTE_ERROR_LOG: A message is attempting to be sent to a process
>>> whose
>>> >> contact information is unknown in file rml_oob_send.c at line 105
>>> >> <PGP.sig>_______________________________________________
>>> >> users mailing list
>>> >> users_at_[hidden]
>>> >> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>> >
>>> >
>>> > --
>>> > Jeff Squyres
>>> > Cisco Systems
>>> >
>>> > _______________________________________________
>>> > users mailing list
>>> > users_at_[hidden]
>>> > http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>
>>> <PGP.sig>_______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users




  • application/pkcs7-signature attachment: smime.p7s