Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2006-12-20 23:01:03


On Dec 20, 2006, at 7:04 PM, Jeff Squyres wrote:

>> I've been asked by the owner of the cluster "How can you prove to me
>> that this openmpi job is using the Infiniband network?"
>>
>> At first I thought a simple netstat -an on the compute nodes might
>> tell
>> me, however I don't see the Infiniband IP's in the list so I'm
>> thinking
>> maybe I need to be looking elsewhere.
>
> That's correct.

I completely forgot to explain this comment, sorry.

Open MPI does not use TCP over IB -- it uses the native "verbs"
interface (i.e., a lower lever than TCP over IB), and therefore a)
doesn't show up in tcpdump/netstat/etc., and b) gets much better
performance (TCP demands much processing on the host CPU, whereas
verbs can be offloaded to the processor on the HCA).

-- 
Jeff Squyres
Server Virtualization Business Unit
Cisco Systems