Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] using rankfiles to control mixed mode programming
From: Ralph Castain (rhc_at_[hidden])
Date: 2010-06-17 09:31:55


IIRC, I added relative node syntax to the rankfile mapper a while back. You should then be able to specify the rank->node mapping by indicating that (e.g.) rank 0 -> first node in file, instead of specifying the node by name.

On Jun 16, 2010, at 2:21 PM, Brock Palen wrote:

> We use Torque and Moab as our resource manager,
> I have a user who has to mix programming models, threads and MPI, but not on all ranks,
>
> Example:
> Rank 0 needs 4 cores
> Rank 1 needs 4 cores
> Ranks 2-5 need 1
>
> There is no way to give this to Torque/Moab in a way that when using TM openMPI correctly starts the layout I need.
> I found I can use a rankfile to do this,
>
> cat rankfile
> rank 0=nyx0456 slot=0
> rank 1=nyx0457 slot=0
> rank 2=nyx0458 slot=0
> rank 3=nyx0458 slot=1
> rank 4=nyx0458 slot=2
> rank 5=nyx0458 slot=3
>
> qsub -I -l nodes=4:ppn=4
> [brockp_at_nyx0456 tmp]$ mpirun -np 6 -rf rankfile hostname | sort | uniq -c
> 1 nyx0456.engin.umich.edu
> 1 nyx0457.engin.umich.edu
> 4 nyx0458.engin.umich.edu
>
> (ignore the cpu count mismatch, I am not using all 16 cores in this example)
>
> I could make a script to build this rankfile, I want to know if there is a better way to do this,
> Also we don't want to use affinity, as we don't know what 'cpus' the bath (pbs/moab) system gave us, is the above rankfile going to do the right thing? That is the processes will float? Is there a way to tell if affinity is set? Or to say "rank 0 goes to this host use any available cores"
>
> Thanks!
>
> Brock Palen
> www.umich.edu/~brockp
> Center for Advanced Computing
> brockp_at_[hidden]
> (734)936-1985
>
>
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users