Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2006-12-20 23:01:03

On Dec 20, 2006, at 7:04 PM, Jeff Squyres wrote:

>> I've been asked by the owner of the cluster "How can you prove to me
>> that this openmpi job is using the Infiniband network?"
>> At first I thought a simple netstat -an on the compute nodes might
>> tell
>> me, however I don't see the Infiniband IP's in the list so I'm
>> thinking
>> maybe I need to be looking elsewhere.
> That's correct.

I completely forgot to explain this comment, sorry.

Open MPI does not use TCP over IB -- it uses the native "verbs"
interface (i.e., a lower lever than TCP over IB), and therefore a)
doesn't show up in tcpdump/netstat/etc., and b) gets much better
performance (TCP demands much processing on the host CPU, whereas
verbs can be offloaded to the processor on the HCA).

Jeff Squyres
Server Virtualization Business Unit
Cisco Systems