[saturna.cluster:19174] progressed_wait: base/plm_base_launch_support.c 459 Daemon was launched on xserve01.cluster - beginning to initialize Daemon [[28401,0],1] checking in as pid 43824 on host xserve01.cluster Daemon [[28401,0],1] not using static ports [saturna.cluster:19174] defining message event: base/plm_base_launch_support.c 423 [xserve01.cluster:43824] [[28401,0],1] orted: up and running - waiting for commands! [saturna.cluster:19174] defining message event: grpcomm_bad_module.c 183 [saturna.cluster:19174] progressed_wait: base/plm_base_launch_support.c 712 [saturna.cluster:19174] [[28401,0],0] orte:daemon:cmd:processor called by [[28401,0],0] for tag 1 [saturna.cluster:19174] [[28401,0],0] node[0].name saturna daemon 0 arch ffc90200 [saturna.cluster:19174] [[28401,0],0] node[1].name xserve01 daemon 1 arch ffc90200 [saturna.cluster:19174] [[28401,0],0] orted_cmd: received add_local_procs [saturna.cluster:19174] defining message event: base/odls_base_default_fns.c 1219 [saturna.cluster:19174] [[28401,0],0] orte:daemon:send_relay [saturna.cluster:19174] [[28401,0],0] orte:daemon:send_relay sending relay msg to 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,0],0] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],0] for tag 1 [xserve01.cluster:43824] [[28401,0],1] node[0].name saturna daemon 0 arch ffc90200 [xserve01.cluster:43824] [[28401,0],1] node[1].name xserve01 daemon 1 arch ffc90200 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received add_local_procs [saturna.cluster:19174] defining message event: base/plm_base_launch_support.c 668 [xserve01.cluster:43824] [[28401,0],1] orte:daemon:send_relay [xserve01.cluster:43824] [[28401,0],1] orte:daemon:send_relay - recipient list is empty! [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],0] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],0] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync+nidmap from local proc [[28401,1],0] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],1] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync+nidmap from local proc [[28401,1],1] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],2] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],2] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync+nidmap from local proc [[28401,1],2] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],3] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],3] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync+nidmap from local proc [[28401,1],3] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],4] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],4] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync+nidmap from local proc [[28401,1],4] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],5] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],5] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync+nidmap from local proc [[28401,1],5] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],6] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],6] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync+nidmap from local proc [[28401,1],6] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [saturna.cluster:19174] defining message event: base/routed_base_receive.c 153 28401,1],7] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],7] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync+nidmap from local proc [[28401,1],7] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],0] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],0] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],2] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],2] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],1] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],3] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],3] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],4] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],4] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],6] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],6] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],5] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],5] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [saturna.cluster:19174] [[28401,0],0] orted_recv_cmd: received message from [[28401,0],1] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],7] f[saturna.cluster:19174] defining message event: orted/orted_comm.c 159 lective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [saturna.cluster:19174] [[28401,0],0] orted_recv_cmd: reissued recv [saturna.cluster:19174] [[28401,0],0] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [saturna.cluster:19174] [[28401,0],0] orted_cmd: received collective data cmd [saturna.cluster:19174] defining message event: grpcomm_bad_module.c 183 [saturna.cluster:19174] [[28401,0],0] orte:daemon:cmd:processor: processing commands completed [saturna.cluster:19174] [[28401,0],0] orte:daemon:cmd:processor called by [[28401,0],0] for tag 1 [saturna.cluster:19174] [[28401,0],0] orted_cmd: received message_local_procs [saturna.cluster:19174] [[28401,0],0] orted:comm:message_local_procs delivering message to job [28401,1] tag 15 [saturna.cluster:19174] [[28401,0],0] orte:daemon:send_relay [saturna.cluster:19174] [[28401,0],0] orte:daemon:send_relay sending relay msg to 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,0],0] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],0] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received message_local_procs [xserve01.cluster:43824] [[28401,0],1] orted:comm:message_local_procs delivering message to job [28401,1] tag 15 [xserve01.cluster:43824] [[28401,0],1] orte:daemon:send_relay [xserve01.cluster:43824] [[28401,0],1] orte:daemon:send_relay - recipient list is empty! [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],2] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],2] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],3] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],3] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],4] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],4] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],7] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],7] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],6] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],6] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],5] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],5] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],0][saturna.cluster:19174] defining message event: orted/orted_comm.c 159 9 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],0] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],1] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: proce[saturna.cluster:19174] [[28401,0],0] orted_recv_cmd: reissued recv [saturna.cluster:19174] [[28401,0],0] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [saturna.cluster:19174] [[28401,0],0] orted_cmd: received collective data cmd [saturna.cluster:19174] defining message event: grpcomm_bad_module.c 183 [saturna.cluster:19174] [[28401,0],0] orte:daemon:cmd:processor: processing commands completed [saturna.cluster:19174] [[28401,0],0] orte:daemon:cmd:processor called by [[28401,0],0] for tag 1 [saturna.cluster:19174] [[28401,0],0] orted_cmd: received message_local_procs [saturna.cluster:19174] [[28401,0],0] orted:comm:message_local_procs delivering message to job [28401,1] tag 17 [saturna.cluster:19174] [[28401,0],0] orte:daemon:send_relay [saturna.cluster:19174] [[28401,0],0] orte:daemon:send_relay sending relay msg to 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,0],0] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],0] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received message_local_procs [xserve01.cluster:43824] [[28401,0],1] orted:comm:message_local_procs delivering message to job [28401,1] tag 17 [xserve01.cluster:43824] [[28401,0],1] orte:daemon:send_relay [xserve01.cluster:43824] [[28401,0],1] orte:daemon:send_relay - recipient list is empty! [saturna.cluster:19174] defining message event: iof_hnp_receive.c 227 Hello, world, I am 0 of 8 [saturna.cluster:19174] defining message event: iof_hnp_receive.c 227 [saturna.cluster:19174] defining message event: iof_hnp_receive.c 227 Hello, world, I am 2 of 8 [saturna.cluster:19174] defining message event: iof_hnp_receive.c 227 Hello, world, I am 4 of 8 [saturna.cluster:19174] defining message event: iof_hnp_receive.c 227 Hello, world, I am 5 of 8 [saturna.cluster:19174] defining message event: iof_hnp_receive.c 227 Hello, world, I am 6 of 8 [saturna.cluster:19174] defining message event: iof_hnp_receive.c 227 Hello, world, I am 7 of 8 [saturna.cluster:19174] defining message event: iof_hnp_receive.c 227 Hello, world, I am 1 of 8 [28401,0],1] orted_recv_cmd: received message from [[28401,1],0] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],0] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],4] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],4] for[saturna.cluster:19174] [[28401,0],0] orted_recv_cmd: reissued recv llective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],1] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],2] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],2] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],3] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],3] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],5] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],5] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [saturna.cluster:19174] [[28401,0],0] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 serve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],6] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],6] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],7] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],7] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received collective data cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [saturna.cluster:19174] [[28401,0],0] orted_cmd: received collective data cmd [saturna.cluster:19174] defining message event: grpcomm_bad_module.c 183 [saturna.cluster:19174] [[28401,0],0] orte:daemon:cmd:processor: processing commands completed [saturna.cluster:19174] [[28401,0],0] orte:daemon:cmd:processor called by [[28401,0],0] for tag 1 [saturna.cluster:19174] [[28401,0],0] orted_cmd: received message_local_procs [saturna.cluster:19174] [[28401,0],0] orted:comm:message_local_procs delivering message to job [28401,1] tag 17 [saturna.cluster:19174] [[28401,0],0] orte:daemon:send_relay [saturna.cluster:19174] [[28401,0],0] orte:daemon:send_relay sending relay msg to 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,0],0] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],0] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received message_local_procs [xserve01.cluster:43824] [[28401,0],1] orted:comm:message_local_procs delivering message to job [28401,1] tag 17 [xserve01.cluster:43824] [[28401,0],1] orte:daemon:send_relay [xserve01.cluster:43824] [[28401,0],1] orte:daemon:send_relay - recipient list is empty! [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],0] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],0] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync from local proc [[28401,1],0] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],3] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],3] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync from local proc [[28401,1],3] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],4] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],4] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync from local proc [[28401,1],4] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],5] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],5] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync from local proc [[28401,1],5] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],6] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],6] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync from local proc [[28401,1],6] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],7] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],7] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync from local proc [[28401,1],7] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],1] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync from local proc [[28401,1],1] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from [[28401,1],2] [xserve01.cluster:43824] defining message event: orted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,1],2] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv: received sync from local proc [[28401,1],2] [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] defining message event: base/odls_base_default_fns.c 2055 [xserve01.cluster:43824] defining message event: iof_orted_read.c 211 [xserve01.cluster:43824] defining message event: iof_orted_read.c 211 [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received waitpid_fired cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] defining message event: base/odls_base_default_fns.c 2055 [xserve01.cluster:43824] defining message event: base/odls_base_default_fns.c 2055 [xserve01.cluster:43824] defining message event: base/odls_base_default_fns.c 2055 [xserve01.cluster:43824] defining message event: base/odls_base_default_fns.c 2055 [xserve01.cluster:43824] defining message event: base/odls_base_default_fns.c 2055 [xserve01.cluster:43824] defining message event: base/odls_base_default_fns.c 2055 [xserve01.cluster:43824] defining message event: base/odls_base_default_fns.c 2055 [xserve01.cluster:43824] defining message event: iof_orted_read.c 211 [xserve01.cluster:43824] defining message event: iof_orted_read.c 211 [xserve01.cluster:43824] defining message event: iof_orted_read.c 211 [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received iof_complete cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received iof_complete cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] defining message event: iof_orted_read.c 211 [xserve01.cluster:43824] defining message event: iof_orted_read.c 211 [xserve01.cluster:43824] defining message event: iof_orted_read.c 211 [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received waitpid_fired cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received waitpid_fired cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received waitpid_fired cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received waitpid_fired cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received waitpid_fired cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received waitpid_fired cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received waitpid_fired cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received iof_complete cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received iof_complete cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received iof_complete cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received iof_complete cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received iof_complete cmd [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor: processing commands completed [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],1] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received iof_complete cmd [saturna.cluster:19174] defining message event: base/plm_base_receive.c 327 commands completed [saturna.cluster:19174] [[28401,0],0] calling job_complete trigger [saturna.cluster:19174] defining message event: grpcomm_bad_module.c 183 [saturna.cluster:19174] [[28401,0],0] orte:daemon:cmd:processor called by [[28401,0],0] for tag 1 [saturna.cluster:19174] [[28401,0],0] orted_cmd: received exit [saturna.cluster:19174] [[28401,0],0] orte:daemon:send_relay [saturna.cluster:19174] [[28401,0],0] orte:daemon:send_relay sending relay msg to 1 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: received message from[saturna.cluster:19174] [[28401,0],0] calling orted_exit trigger rted/orted_comm.c 159 [xserve01.cluster:43824] [[28401,0],1] orted_recv_cmd: reissued recv [xserve01.cluster:43824] [[28401,0],1] orte:daemon:cmd:processor called by [[28401,0],0] for tag 1 [xserve01.cluster:43824] [[28401,0],1] orted_cmd: received exit [xserve01.cluster:43824] [[28401,0],1] orte:daemon:send_relay [xserve01.cluster:43824] [[28401,0],1] orte:daemon:send_relay - recipient list is empty! [xserve01.cluster:43824] [[28401,0],1] calling orted_shutdown trigger [xserve01.cluster:43824] [[28401,0],1] orted: finalizing