Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI users] How to use Multiple links with OpenMPI??????????????????
From: shan axida (axida2009_at_[hidden])
Date: 2009-05-28 23:06:09

Thank you! Mr. Jeff Squyres, I have conducted a simple MPI_Bcast experiment in out cluster. The results are shown in the file attached on this e-mail. The hostfile is : ----------------- hostname1 slots=4 hostname2 slots=4 hostname3 slots=4 .... .... hostname16 slots=4 ----------------- As we can see in the figure, it is little faster than single link when we use 2,3,4 links between nodes. My question is what would be the reason to make almost the same performance when we use 2,3,4 links ? Thank you! Axida ________________________________ From: Jeff Squyres <jsquyres_at_[hidden]> To: Open MPI Users <users_at_[hidden]> Sent: Wednesday, May 27, 2009 11:28:42 PM Subject: Re: [OMPI users] How to use Multiple links with OpenMPI?????????????????? Open MPI considers hosts differently than network links. So you should only list the actual hostname in the hostfile, with slots equal to the number of processors (4 in your case, I think?). Once the MPI processes are launched, they each look around on the host that they're running and find network paths to each of their peers. If they are multiple paths between pairs of peers, Open MPI will round-robin stripe messages across each of the links. We don't really have an easy setting for each peer pair only using 1 link. Indeed, since connectivity is bidirectional, the traffic patterns become less obvious if you want MPI_COMM_WORLD rank X to only use link Y -- what does that mean to the other 4 MPI processes on the other host (with whom you have assumedly assigned their own individual links as well)? On May 26, 2009, at 12:24 AM, shan axida wrote: > Hi everyone, > I want to ask how to use multiple links (multiple NICs) with OpenMPI. > For example, how can I assign a link to each process, if there are 4 links > and 4 processors on each node in our cluster? > Is this a correct way? > hostfile: > ---------------------- > host1-eth0 slots=1 > host1-eth1 slots=1 > host1-eth2 slots=1 > host1-eth3 slots=1 > host2-eth0 slots=1 > host2-eth1 slots=1 > host2-eth2 slots=1 > host2-eth3 slots=1 > ... ... > ... ... > host16-eth0 slots=1 > host16-eth1 slots=1 > host16-eth2 slots=1 > host16-eth3 slots=1 > ------------------------ > > > > > > > > > > > _______________________________________________ > users mailing list > users_at_[hidden] > --Jeff Squyres Cisco Systems _______________________________________________ users mailing list users_at_[hidden]