Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Michael John Hanby (mhanby_at_[hidden])
Date: 2006-12-21 09:53:27


Thanks Jeff and Andrew, those responses were well thought out and very
informative.

I'll run the same test explicitly using TCP for a couple runs and then
mvapi a couple times. The results should give the users a warm fuzzy
that OpenMPI is in fact using the expensive faster network (rather than
cheep gig-e).

Thanks again.

-----Original Message-----
From: users-bounces_at_[hidden] [mailto:users-bounces_at_[hidden]] On
Behalf Of Jeff Squyres
Sent: Wednesday, December 20, 2006 10:01 PM
To: Jeff Squyres
Cc: Open MPI Users
Subject: Re: [OMPI users] Infiniband - Any suggestions on "How can you
proveto me that OpenMPI is using it?"

On Dec 20, 2006, at 7:04 PM, Jeff Squyres wrote:

>> I've been asked by the owner of the cluster "How can you prove to me
>> that this openmpi job is using the Infiniband network?"
>>
>> At first I thought a simple netstat -an on the compute nodes might
>> tell
>> me, however I don't see the Infiniband IP's in the list so I'm
>> thinking
>> maybe I need to be looking elsewhere.
>
> That's correct.

I completely forgot to explain this comment, sorry.

Open MPI does not use TCP over IB -- it uses the native "verbs"
interface (i.e., a lower lever than TCP over IB), and therefore a)
doesn't show up in tcpdump/netstat/etc., and b) gets much better
performance (TCP demands much processing on the host CPU, whereas
verbs can be offloaded to the processor on the HCA).

-- 
Jeff Squyres
Server Virtualization Business Unit
Cisco Systems
_______________________________________________
users mailing list
users_at_[hidden]
http://www.open-mpi.org/mailman/listinfo.cgi/users