Just committed a potential fix to the trunk - please let me know if it worked for you

On May 14, 2014, at 11:44 AM, Siegmar Gross <Siegmar.Gross@informatik.hs-fulda.de> wrote:

Hi Ralph,

Hmmm...well, that's an interesting naming scheme :-)

Try adding "-mca oob_base_verbose 10 --report-uri -" on your cmd line
and let's see what it thinks is happening


tyr fd1026 105 mpiexec -np 3 --host tyr,sunpc1,linpc1 --mca oob_base_verbose 10 --report-uri - hostname
[tyr.informatik.hs-fulda.de:06877] mca: base: components_register: registering oob components
[tyr.informatik.hs-fulda.de:06877] mca: base: components_register: found loaded component tcp
[tyr.informatik.hs-fulda.de:06877] mca: base: components_register: component tcp register function successful
[tyr.informatik.hs-fulda.de:06877] mca: base: components_open: opening oob components
[tyr.informatik.hs-fulda.de:06877] mca: base: components_open: found loaded component tcp
[tyr.informatik.hs-fulda.de:06877] mca: base: components_open: component tcp open function successful
[tyr.informatik.hs-fulda.de:06877] mca:oob:select: checking available component tcp
[tyr.informatik.hs-fulda.de:06877] mca:oob:select: Querying component [tcp]
[tyr.informatik.hs-fulda.de:06877] oob:tcp: component_available called
[tyr.informatik.hs-fulda.de:06877] WORKING INTERFACE 1 KERNEL INDEX 1 FAMILY: V4
[tyr.informatik.hs-fulda.de:06877] WORKING INTERFACE 2 KERNEL INDEX 2 FAMILY: V4
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] oob:tcp:init creating module for V4 address on interface bge0
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] creating OOB-TCP module for interface bge0
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] oob:tcp:init adding 193.174.24.39 to our list of V4 connections
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] TCP STARTUP
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] attempting to bind to IPv4 port 0
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] assigned IPv4 port 55567
[tyr.informatik.hs-fulda.de:06877] mca:oob:select: Adding component to end
[tyr.informatik.hs-fulda.de:06877] mca:oob:select: Found 1 active transports
3170566144.0;tcp://193.174.24.39:55567
[sunpc1:07690] mca: base: components_register: registering oob components
[sunpc1:07690] mca: base: components_register: found loaded component tcp
[sunpc1:07690] mca: base: components_register: component tcp register function successful
[sunpc1:07690] mca: base: components_open: opening oob components
[sunpc1:07690] mca: base: components_open: found loaded component tcp
[sunpc1:07690] mca: base: components_open: component tcp open function successful
[sunpc1:07690] mca:oob:select: checking available component tcp
[sunpc1:07690] mca:oob:select: Querying component [tcp]
[sunpc1:07690] oob:tcp: component_available called
[sunpc1:07690] WORKING INTERFACE 1 KERNEL INDEX 1 FAMILY: V4
[sunpc1:07690] WORKING INTERFACE 2 KERNEL INDEX 2 FAMILY: V4
[sunpc1:07690] [[48379,0],1] oob:tcp:init creating module for V4 address on interface nge0
[sunpc1:07690] [[48379,0],1] creating OOB-TCP module for interface nge0
[sunpc1:07690] [[48379,0],1] oob:tcp:init adding 193.174.26.210 to our list of V4 connections
[sunpc1:07690] [[48379,0],1] TCP STARTUP
[sunpc1:07690] [[48379,0],1] attempting to bind to IPv4 port 0
[sunpc1:07690] [[48379,0],1] assigned IPv4 port 39616
[sunpc1:07690] mca:oob:select: Adding component to end
[sunpc1:07690] mca:oob:select: Found 1 active transports
[sunpc1:07690] [[48379,0],1]: set_addr to uri 3170566144.0;tcp://193.174.24.39:55567
[sunpc1:07690] [[48379,0],1]:set_addr checking if peer [[48379,0],0] is reachable via component tcp
[sunpc1:07690] [[48379,0],1] oob:tcp: working peer [[48379,0],0] address tcp://193.174.24.39:55567
[sunpc1:07690] [[48379,0],1] UNFOUND KERNEL INDEX -13 FOR ADDRESS 193.174.24.39
[sunpc1:07690] [[48379,0],1] PEER [[48379,0],0] MAY BE REACHABLE BY ROUTING - ASSIGNING MODULE AT KINDEX 2 INTERFACE nge0
[sunpc1:07690] [[48379,0],1] PASSING ADDR 193.174.24.39 TO INTERFACE nge0 AT KERNEL INDEX 2
[sunpc1:07690] [[48379,0],1]:tcp set addr for peer [[48379,0],0]
[sunpc1:07690] [[48379,0],1]: peer [[48379,0],0] is reachable via component tcp
[sunpc1:07690] [[48379,0],1] OOB_SEND: ../../../../../openmpi-1.8.2a1r31742/orte/mca/rml/oob/rml_oob_send.c:199
[sunpc1:07690] [[48379,0],1]:tcp:processing set_peer cmd for interface nge0
[sunpc1:07690] [[48379,0],1] oob:base:send to target [[48379,0],0]
[sunpc1:07690] [[48379,0],1] oob:tcp:send_nb to peer [[48379,0],0]:10
[sunpc1:07690] [[48379,0],1] tcp:send_nb to peer [[48379,0],0]
[sunpc1:07690] [[48379,0],1]:[../../../../../openmpi-1.8.2a1r31742/orte/mca/oob/tcp/oob_tcp.c:508] post send to [[48379,0],0]
[sunpc1:07690] [[48379,0],1]:[../../../../../openmpi-1.8.2a1r31742/orte/mca/oob/tcp/oob_tcp.c:442] processing send to peer 
[[48379,0],0]:10
[sunpc1:07690] [[48379,0],1]:[../../../../../openmpi-1.8.2a1r31742/orte/mca/oob/tcp/oob_tcp.c:476] queue pending to [[48379,0],0]
[sunpc1:07690] [[48379,0],1] tcp:send_nb: initiating connection to [[48379,0],0]
[sunpc1:07690] [[48379,0],1]:[../../../../../openmpi-1.8.2a1r31742/orte/mca/oob/tcp/oob_tcp.c:490] connect to [[48379,0],0]
[sunpc1:07690] [[48379,0],1] orte_tcp_peer_try_connect: attempting to connect to proc [[48379,0],0] via interface nge0
[sunpc1:07690] [[48379,0],1] oob:tcp:peer creating socket to [[48379,0],0]
[sunpc1:07690] [[48379,0],1] orte_tcp_peer_try_connect: attempting to connect to proc [[48379,0],0] via interface nge0 on socket 10
[sunpc1:07690] [[48379,0],1] orte_tcp_peer_try_connect: attempting to connect to proc [[48379,0],0] on 193.174.24.39:55567 - 0 retries
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] mca_oob_tcp_listen_thread: new connection: (15, 0) 193.174.26.210:39617
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] connection_handler: working connection (15, 11) 193.174.26.210:39617
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] CONNECTION REQUEST ON UNKNOWN INTERFACE
[sunpc1:07690] [[48379,0],1] waiting for connect completion to [[48379,0],0] - activating send event
[sunpc1:07690] [[48379,0],1] tcp:send_handler called to send to peer [[48379,0],0]
[sunpc1:07690] [[48379,0],1] tcp:send_handler CONNECTING
[sunpc1:07690] [[48379,0],1]:tcp:complete_connect called for peer [[48379,0],0] on socket 10
[sunpc1:07690] [[48379,0],1] tcp_peer_complete_connect: sending ack to [[48379,0],0]
[sunpc1:07690] [[48379,0],1] SEND CONNECT ACK
[sunpc1:07690] [[48379,0],1] send blocking of 48 bytes to socket 10
[sunpc1:07690] [[48379,0],1] connect-ack sent to socket 10
[sunpc1:07690] [[48379,0],1] tcp_peer_complete_connect: setting read event on connection to [[48379,0],0]
[linpc1:21511] mca: base: components_register: registering oob components
[linpc1:21511] mca: base: components_register: found loaded component tcp
[linpc1:21511] mca: base: components_register: component tcp register function successful
[linpc1:21511] mca: base: components_open: opening oob components
[linpc1:21511] mca: base: components_open: found loaded component tcp
[linpc1:21511] mca: base: components_open: component tcp open function successful
[linpc1:21511] mca:oob:select: checking available component tcp
[linpc1:21511] mca:oob:select: Querying component [tcp]
[linpc1:21511] oob:tcp: component_available called

[linpc1:21511] WORKING INTERFACE 1 KERNEL INDEX 1 FAMILY: V4
[linpc1:21511] WORKING INTERFACE 2 KERNEL INDEX 2 FAMILY: V4
[linpc1:21511] [[48379,0],2] oob:tcp:init creating module for V4 address on interface eth0
[linpc1:21511] [[48379,0],2] creating OOB-TCP module for interface eth0
[linpc1:21511] [[48379,0],2] oob:tcp:init adding 193.174.26.208 to our list of V4 connections
[linpc1:21511] [[48379,0],2] TCP STARTUP
[linpc1:21511] [[48379,0],2] attempting to bind to IPv4 port 0
[linpc1:21511] [[48379,0],2] assigned IPv4 port 39724
[linpc1:21511] mca:oob:select: Adding component to end
[linpc1:21511] mca:oob:select: Found 1 active transports
[linpc1:21511] [[48379,0],2]: set_addr to uri 3170566144.0;tcp://193.174.24.39:55567
[linpc1:21511] [[48379,0],2]:set_addr checking if peer [[48379,0],0] is reachable via component tcp
[linpc1:21511] [[48379,0],2] oob:tcp: working peer [[48379,0],0] address tcp://193.174.24.39:55567
[linpc1:21511] [[48379,0],2] UNFOUND KERNEL INDEX -13 FOR ADDRESS 193.174.24.39
[linpc1:21511] [[48379,0],2] PEER [[48379,0],0] MAY BE REACHABLE BY ROUTING - ASSIGNING MODULE AT KINDEX 2 INTERFACE eth0
[linpc1:21511] [[48379,0],2] PASSING ADDR 193.174.24.39 TO INTERFACE eth0 AT KERNEL INDEX 2
[linpc1:21511] [[48379,0],2]:tcp set addr for peer [[48379,0],0]
[linpc1:21511] [[48379,0],2]: peer [[48379,0],0] is reachable via component tcp
[linpc1:21511] [[48379,0],2] OOB_SEND: ../../../../../openmpi-1.8.2a1r31742/orte/mca/rml/oob/rml_oob_send.c:199
[linpc1:21511] [[48379,0],2]:tcp:processing set_peer cmd for interface eth0
[linpc1:21511] [[48379,0],2] oob:base:send to target [[48379,0],0]
[linpc1:21511] [[48379,0],2] oob:tcp:send_nb to peer [[48379,0],0]:10
[linpc1:21511] [[48379,0],2] tcp:send_nb to peer [[48379,0],0]
[linpc1:21511] [[48379,0],2]:[../../../../../openmpi-1.8.2a1r31742/orte/mca/oob/tcp/oob_tcp.c:508] post send to [[48379,0],0]
[linpc1:21511] [[48379,0],2]:[../../../../../openmpi-1.8.2a1r31742/orte/mca/oob/tcp/oob_tcp.c:442] processing send to peer 
[[48379,0],0]:10
[linpc1:21511] [[48379,0],2]:[../../../../../openmpi-1.8.2a1r31742/orte/mca/oob/tcp/oob_tcp.c:476] queue pending to [[48379,0],0]
[linpc1:21511] [[48379,0],2] tcp:send_nb: initiating connection to [[48379,0],0]
[linpc1:21511] [[48379,0],2]:[../../../../../openmpi-1.8.2a1r31742/orte/mca/oob/tcp/oob_tcp.c:490] connect to [[48379,0],0]
[linpc1:21511] [[48379,0],2] orte_tcp_peer_try_connect: attempting to connect to proc [[48379,0],0] via interface eth0
[linpc1:21511] [[48379,0],2] oob:tcp:peer creating socket to [[48379,0],0]
[linpc1:21511] [[48379,0],2] orte_tcp_peer_try_connect: attempting to connect to proc [[48379,0],0] via interface eth0 on socket 9
[linpc1:21511] [[48379,0],2] orte_tcp_peer_try_connect: attempting to connect to proc [[48379,0],0] on 193.174.24.39:55567 - 0 retries
[linpc1:21511] [[48379,0],2] waiting for connect completion to [[48379,0],0] - activating send event
[linpc1:21511] [[48379,0],2] tcp:send_handler called to send to peer [[48379,0],0]
[linpc1:21511] [[48379,0],2] tcp:send_handler CONNECTING
[linpc1:21511] [[48379,0],2]:tcp:complete_connect called for peer [[48379,0],0] on socket 9
[linpc1:21511] [[48379,0],2] tcp_peer_complete_connect: sending ack to [[48379,0],0]
[linpc1:21511] [[48379,0],2] SEND CONNECT ACK
[linpc1:21511] [[48379,0],2] send blocking of 48 bytes to socket 9
[linpc1:21511] [[48379,0],2] connect-ack sent to socket 9
[linpc1:21511] [[48379,0],2] tcp_peer_complete_connect: setting read event on connection to [[48379,0],0]
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] mca_oob_tcp_listen_thread: new connection: (16, 11) 193.174.26.208:53741
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] connection_handler: working connection (16, 11) 193.174.26.208:53741
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] CONNECTION REQUEST ON UNKNOWN INTERFACE
^CKilled by signal 2.
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] OOB_SEND: ../../../../../openmpi-1.8.2a1r31742/orte/mca/rml/oob/rml_oob_send.c:199
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] OOB_SEND: ../../../../../openmpi-1.8.2a1r31742/orte/mca/rml/oob/rml_oob_send.c:199
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] oob:base:send to target [[48379,0],1]
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] oob:base:send unknown peer [[48379,0],1]
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] is NOT reachable by TCP
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] oob:base:send to target [[48379,0],2]
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] oob:base:send unknown peer [[48379,0],2]
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] is NOT reachable by TCP
Killed by signal 2.
[tyr.informatik.hs-fulda.de:06877] [[48379,0],0] TCP SHUTDOWN
[tyr.informatik.hs-fulda.de:06877] mca: base: close: component tcp closed
[tyr.informatik.hs-fulda.de:06877] mca: base: close: unloading component tcp
tyr fd1026 106 


Thank you very much for your help in advance. Do you need anything else?


Kind regards

Siegmar



On May 14, 2014, at 9:06 AM, Siegmar Gross <Siegmar.Gross@informatik.hs-fulda.de> wrote:

Hi Ralph,

What are the interfaces on these machines?

tyr fd1026 111 ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
      inet 127.0.0.1 netmask ff000000 
bge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
      inet 193.174.24.39 netmask ffffffe0 broadcast 193.174.24.63
tyr fd1026 112 


tyr fd1026 112 ssh sunpc1 ifconfig -a
lo0: flags=2001000849<UP,LOOPBACK,RUNNING,MULTICAST,IPv4,VIRTUAL> mtu 8232 index 1
      inet 127.0.0.1 netmask ff000000 
nge0: flags=1000843<UP,BROADCAST,RUNNING,MULTICAST,IPv4> mtu 1500 index 2
      inet 193.174.26.210 netmask ffffffc0 broadcast 193.174.26.255
tyr fd1026 113 


tyr fd1026 113 ssh linpc1 /sbin/ifconfig -a
eth0      Link encap:Ethernet  HWaddr 00:14:4F:23:FD:A8  
        inet addr:193.174.26.208  Bcast:193.174.26.255  Mask:255.255.255.192
        UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
        RX packets:18052524 errors:127 dropped:0 overruns:0 frame:127
        TX packets:15917888 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:1000 
        RX bytes:4158294157 (3965.6 Mb)  TX bytes:12060556809 (11501.8 Mb)
        Interrupt:23 Base address:0x4000 

eth1      Link encap:Ethernet  HWaddr 00:14:4F:23:FD:A9  
        BROADCAST MULTICAST  MTU:1500  Metric:1
        RX packets:0 errors:0 dropped:0 overruns:0 frame:0
        TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:1000 
        RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
        Interrupt:45 Base address:0xa000 

lo        Link encap:Local Loopback  
        inet addr:127.0.0.1  Mask:255.0.0.0
        UP LOOPBACK RUNNING  MTU:16436  Metric:1
        RX packets:1083 errors:0 dropped:0 overruns:0 frame:0
        TX packets:1083 errors:0 dropped:0 overruns:0 carrier:0
        collisions:0 txqueuelen:0 
        RX bytes:329323 (321.6 Kb)  TX bytes:329323 (321.6 Kb)

tyr fd1026 114 


Do you need something else?


Kind regards

Siegmar




On May 14, 2014, at 7:45 AM, Siegmar Gross <Siegmar.Gross@informatik.hs-fulda.de> wrote:

Hi,

I just installed openmpi-1.8.2a1r31742 on my machines (Solaris 10
Sparc, Solaris 10 x86_64, and openSUSE Linux 12.1 x86_64) with
Sun C5.12 and still have the following problem.

tyr fd1026 102 which mpiexec
/usr/local/openmpi-1.8.2_64_cc/bin/mpiexec
tyr fd1026 103 mpiexec -np 3 --host tyr,sunpc1,linpc1 hostname
[tyr.informatik.hs-fulda.de:12827] [[37949,0],0] CONNECTION
REQUEST ON UNKNOWN INTERFACE
[tyr.informatik.hs-fulda.de:12827] [[37949,0],0] CONNECTION
REQUEST ON UNKNOWN INTERFACE
^CKilled by signal 2.
Killed by signal 2.
tyr fd1026 104 


The command works fine with openmpi-1.6.6rc1.

tyr fd1026 102 which mpiexec
/usr/local/openmpi-1.6.6_64_cc/bin/mpiexec
tyr fd1026 103 mpiexec -np 3 --host tyr,sunpc1,linpc1 hostname
tyr.informatik.hs-fulda.de
linpc1
sunpc1
tyr fd1026 104 


I have reported the problem before and I would be grateful, if
somebody could solve it. Please let me know if I can provide any
other information.


Kind regards

Siegmar

_______________________________________________
users mailing list
users@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users



_______________________________________________
users mailing list
users@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users


_______________________________________________
users mailing list
users@open-mpi.org
http://www.open-mpi.org/mailman/listinfo.cgi/users