Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Maestas, Christopher Daniel (cdmaest_at_[hidden])
Date: 2006-12-12 11:18:47


Ralph,

I figured I should of run an mpi program ...here's what it does (seems
to be by-X-slot style):

---
$ /apps/x86_64/system/mpiexec-0.82/bin/mpiexec -npernode 2 mpi_hello
Hello, I am node an41 with rank 0
Hello, I am node an41 with rank 1
Hello, I am node an39 with rank 4
Hello, I am node an40 with rank 2
Hello, I am node an38 with rank 6
Hello, I am node an39 with rank 5
Hello, I am node an38 with rank 7
Hello, I am node an40 with rank 3
---
What you describe makes sense to me.  Thanks!
> -----Original Message-----
> From: users-bounces_at_[hidden] 
> [mailto:users-bounces_at_[hidden]] On Behalf Of Ralph Castain
> Sent: Monday, December 11, 2006 10:27 PM
> To: Open MPI Users
> Subject: Re: [OMPI users] Pernode request
> 
> Hi Chris
> 
> I have also implemented --npernode N now as well - it is in 
> the trunk as of
> r12826.
> 
> The testing you show below using mpiexec really doesn't tell 
> us the story -
> we need to know the rank of the various processes (and unfortunately,
> hostname just tells us the host). There is no way to tell the 
> rank from just
> the order in which the host names are printed to the screen. 
> I have a test
> program in our distribution (see 
> orte/test/mpi/hello_hostname.c) that will
> output both the rank and hostname - it would give us the 
> required info.
> 
> Regardless, I think it would make sense to provide the flexibility you
> describe. What if we selected this "by-X-slot" style by using 
> the --npernode
> option, and allowing the user to combine it with the existing 
> "--byslot"
> option? This would still launch N procs/node, but with the 
> ranking done by
> slot. If the user doesn't specify "byslot", then we default 
> to assigning
> ranks by node.
> 
> Make sense? If so, I can probably have that going before the holiday.
> 
> Ralph
> 
> 
> 
> On 12/11/06 7:51 PM, "Maestas, Christopher Daniel" 
> <cdmaest_at_[hidden]>
> wrote:
> 
> > Hello Ralph,
> > 
> > This is great news!  Thanks for doing this.  I will try and 
> get around
> > to it soon before the holiday break.
> > 
> > The allocation scheme always seems to get to me.  From what 
> you describe
> > that is how I would have seen it.  As I've gotten to know 
> osc mpiexec
> > through the years I think they like to do a first fit 
> approach, but now
> > that I test it I think the feature needs more testing or 
> I'm not testing
> > appropriately :-)
> > ---
> > $ /apps/x86_64/system/mpiexec-0.82/bin/mpiexec -comm=none 
> -npernode 2
> > grep HOSTNAME /etc/sysconfig/network
> > HOSTNAME="an56"
> > HOSTNAME="an56"
> > HOSTNAME="an55"
> > HOSTNAME="an53"
> > HOSTNAME="an54"
> > HOSTNAME="an55"
> > HOSTNAME="an53"
> > HOSTNAME="an54"
> > ---
> > 
> > I guess I would wonder if it would be possible to switch 
> from the method
> > what you suggest and also allow a "by-X-slot" style of 
> launch where you
> > would see for npernode = X and N nodes:
> >  proc1 - node1
> >  proc2 - node1
> >  ...
> >  proc(X*1) - node1
> > ...
> >  proc(X+1) - node2
> >  proc(X+2) - node2
> > ...
> >  proc(X*2) - node2
> > ...
> >  proc(N*X-(X-0)) - nodeN
> >  proc(N*X-(X-1)) - nodeN
> >  ...
> >  proc(X*N-1) - nodeN
> >  proc(X*N) - nodeN
> > 
> > I think that's how to best describe it.  Basically you load 
> until there
> > are X processes on each node before moving to the next.  
> This may prove
> > to be more challenging, and I can understand if it would 
> not be deemed
> > "worthy." :-)
> > 
> > -cdm 
> > 
> >> -----Original Message-----
> >> From: users-bounces_at_[hidden]
> >> [mailto:users-bounces_at_[hidden]] On Behalf Of Ralph Castain
> >> Sent: Monday, December 11, 2006 5:41 PM
> >> To: Open MPI Users
> >> Subject: Re: [OMPI users] Pernode request
> >> 
> >> Hi Chris
> >> 
> >> Okay, we have modified the pernode behavior as you requested
> >> (on the trunk
> >> as of r12821)- give it a shot and see if that does it. I have
> >> not yet added
> >> the npernode option, but hope to get that soon.
> >> 
> >> I have a question for you about the npernode option. I am
> >> assuming that you
> >> want n procs/node, but that you want it mapped by NODE. For
> >> example, proc 0
> >> goes on the first node, proc 1 goes on the second node, etc.
> >> until I get one
> >> on each node; then I wrap back to the beginning and do this
> >> again until I
> >> get the specified number of procs on each node.
> >> 
> >> Correct?
> >> Ralph
> >> 
> >>> Ralph,
> >> 
> >>> I agree with what you stated in points 1-4. That is what we
> >> are looking
> >>> for.
> >>> I understand your point now about the non-MPI users too. :-)
> >>> 
> >>> Thanks,
> >>> -cdm
> >> 
> >>>> -----Original Message-----
> >>>> From: users-bounces_at_[hidden]
> >> [mailto:users-bounces_at_[hidden]] On
> >>>> Behalf Of Ralph Castain
> >>>> Sent: Wednesday, November 29, 2006 8:01 AM
> >>>> To: Open MPI Users
> >>>> Subject: Re: [OMPI users] Pernode request
> >>>> 
> >>>> Hi Chris
> >>>> 
> >>>> Thanks for the patience and the clarification - much 
> appreciated. In
> >>>> fact, I have someone that needs to learn more about the
> >> code base, so I
> >>>> think I will assign this to him. At the least, he will have
> >> to learn a
> >>>> lot more about the mapper!
> >>>> 
> >>>> I have no problem with modifying the pernode behavior to
> >> deal with the
> >>>> case of someone specifying -np as you describe. It would be
> >> relatively
> >>>> easy to check. As I understand it, you want the behavior to be:
> >>>> 
> >>>> 1. if no -np is specified, launch one proc/node across
> >> entire allocation
> >>>> 
> >>>> 2. if -np n is specified AND n is less than the number 
> of allocated
> >>>> nodes, then launch one proc/node up to the specified
> >> number. Of course,
> >>>> this is identical to just doing -np n -bynode, but that's
> >> immaterial.
> >>>> ;-)
> >>>> 
> >>>> 3. if -np n is specified AND n is greater than the number
> >> of allocated
> >>>> nodes, error message and exit
> >>>> 
> >>>> 4. add a -npernode n option that launches n procs/node,
> >> subject to the
> >>>> same tests above.
> >>>> 
> >>>> Can you confirm?
> >>>> 
> >>>> Finally, I think you misunderstood my comment about the MPI
> >> folks. Our
> >>>> non-MPI users couldn't care less about commonality of 
> command line
> >>>> arguments across MPI implementations. Hence, I leave issues
> >> in that area
> >>>> to the MPI members of the team - they are the ones that
> >> decide how to
> >>>> deal with the myriad of different option syntaxes in the
> >> MPI community.
> >>>> 
> >>>> Gives me too much of a headache! :-)
> >>>> 
> >>>> Ralph 
> >> 
> >> 
> >> _______________________________________________
> >> users mailing list
> >> users_at_[hidden]
> >> http://www.open-mpi.org/mailman/listinfo.cgi/users
> >> 
> > 
> > 
> > _______________________________________________
> > users mailing list
> > users_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
> 
> 
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>