Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] pinning processes by default
From: Ralph Castain (rhc_at_[hidden])
Date: 2014-05-23 09:32:44


Note that the lama mapper described in those slides may not work as it hasn't been maintained in a while. However, you can use the map-by and bind-to options to do the same things.

If you want to disable binding, you can do so by adding "--bind-to none" to the cmd line, or via the MCA param "hwloc_base_binding_policy=none"

If you want to bind your process to multiple cores (say one per thread), then you can use "--map-by core:pe=N". Many hybrid users prefer to bind to a socket, with one process for each socket - that can be done with "--map-by socket --bind-to socket". This keeps all the threads in the same NUMA domain. If you aren't sure that each socket is its own NUMA domain, you can alternatively "--map-by numa --bind-to numa" - that'll keep you in your own NUMA domain regardless of whether that's at the socket level or elsewhere.

I'm working on adding a full description of the mapping/binding system to our web site.

HTH
Ralph

On May 23, 2014, at 6:22 AM, Brock Palen <brockp_at_[hidden]> wrote:

> Albert,
>
> Actually doing affinity correctly for hybrid got easier in OpenMPI 1.7 and newer, In the past you had to make a lot of assumptions, stride by node etc,
>
> Now you can define a layout:
>
> http://blogs.cisco.com/performance/eurompi13-cisco-slides-open-mpi-process-affinity-user-interface/
>
> Brock Palen
> www.umich.edu/~brockp
> CAEN Advanced Computing
> XSEDE Campus Champion
> brockp_at_[hidden]
> (734)936-1985
>
>
>
> On May 23, 2014, at 9:19 AM, Albert Solernou <albert.solernou_at_[hidden]> wrote:
>
>> Hi,
>> after compiling and installing OpenMPI 1.8.1, I find that OpenMPI is pinning processes onto cores. Although this may be
>> desirable on some cases, it is a complete disaster when runnning hybrid OpenMP-MPI applications. Therefore, I want to disable this behaviour, but don't know how.
>>
>> I configured OpenMPI with:
>> ./configure \
>> --prefix=$OPENMPI_HOME \
>> --with-psm \
>> --with-tm=/system/software/arcus/lib/PBS/11.3.0.121723 \
>> --enable-mpirun-prefix-by-default \
>> --enable-mpi-thread-multiple
>>
>> and:
>> ompi_info | grep paffinity
>> does not report anything. However,
>> mpirun -np 2 --report-bindings ./wrk
>> reports bindings:
>> [login3:04574] MCW rank 1 bound to socket 0[core 1[hwt 0-1]]: [../BB/../../../../../..][../../../../../../../..]
>> [login3:04574] MCW rank 0 bound to socket 0[core 0[hwt 0-1]]: [BB/../../../../../../..][../../../../../../../..]
>> but they cannot be disabled as:
>> mpirun -np 2 --bind-to-none ./wrk
>> returns:
>> mpirun: Error: unknown option "--bind-to-none"
>>
>> Any idea on what went wrong?
>>
>> Best,
>> Albert
>>
>> --
>> ---------------------------------
>> Dr. Albert Solernou
>> Research Associate
>> Oxford Supercomputing Centre,
>> University of Oxford
>> Tel: +44 (0)1865 610631
>> ---------------------------------
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users