Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] problem when binding to socket on a single socket node
From: Terry Dontje (terry.dontje_at_[hidden])
Date: 2010-04-09 11:33:27


Nadia Derbey wrote:
> On Fri, 2010-04-09 at 08:41 -0600, Ralph Castain wrote:
>
>> Just to check: is this with the latest trunk? Brad and Terry have been making changes to this section of code, including modifying the PROCESS_IS_BOUND test...
>>
>>
>>
>
> Well, it was on the v1.5. But I just checked: looks like
> 1. the call to OPAL_PAFFINITY_PROCESS_IS_BOUND is still there in
> odls_default_fork_local_proc()
> 2. OPAL_PAFFINITY_PROCESS_IS_BOUND() is defined the same way
>
> But, I'll give it a try with the latest trunk.
>
> Regards,
> Nadia
>
>
The changes, I've done do not touch OPAL_PAFFINITY_PROCESS_IS_BOUND at
all. Also, I am only touching code related to the "bind-to-core" option
so I really doubt if my changes are causing issues here.

--td
>> On Apr 9, 2010, at 3:39 AM, Nadia Derbey wrote:
>>
>>
>>> Hi,
>>>
>>> I am facing a problem with a test that runs fine on some nodes, and
>>> fails on others.
>>>
>>> I have a heterogenous cluster, with 3 types of nodes:
>>> 1) Single socket , 4 cores
>>> 2) 2 sockets, 4cores per socket
>>> 3) 2 sockets, 6 cores/socket
>>>
>>> I am using:
>>> . salloc to allocate the nodes,
>>> . mpirun binding/mapping options "-bind-to-socket -bysocket"
>>>
>>> # salloc -N 1 mpirun -n 4 -bind-to-socket -bysocket sleep 900
>>>
>>> This command fails if the allocated node is of type #1 (single socket/4
>>> cpus).
>>> BTW, in that case orte_show_help is referencing a tag
>>> ("could-not-bind-to-socket") that does not exist in
>>> help-odls-default.txt.
>>>
>>> While it succeeds when run on nodes of type #2 or 3.
>>> I think a "bind to socket" should not return an error on a single socket
>>> machine, but rather be a noop.
>>>
>>> The problem comes from the test
>>> OPAL_PAFFINITY_PROCESS_IS_BOUND(mask, &bound);
>>> called in odls_default_fork_local_proc() after the binding to the
>>> processors socket has been done:
>>> ========
>>> <snip>
>>> OPAL_PAFFINITY_CPU_ZERO(mask);
>>> for (n=0; n < orte_default_num_cores_per_socket; n++) {
>>> <snip>
>>> OPAL_PAFFINITY_CPU_SET(phys_cpu, mask);
>>> }
>>> /* if we did not bind it anywhere, then that is an error */
>>> OPAL_PAFFINITY_PROCESS_IS_BOUND(mask, &bound);
>>> if (!bound) {
>>> orte_show_help("help-odls-default.txt",
>>> "odls-default:could-not-bind-to-socket", true);
>>> ORTE_ODLS_ERROR_OUT(ORTE_ERR_FATAL);
>>> }
>>> ========
>>> OPAL_PAFFINITY_PROCESS_IS_BOUND() will return true if there bits set in
>>> the mask *AND* the number of bits set is lesser than the number of cpus
>>> on the machine. Thus on a single socket, 4 cores machine the test will
>>> fail. While on other the kinds of machines it will succeed.
>>>
>>> Again, I think the problem could be solved by changing the alogrithm,
>>> and assuming that ORTE_BIND_TO_SOCKET, on a single socket machine =
>>> noop.
>>>
>>> Another solution could be to call the test
>>> OPAL_PAFFINITY_PROCESS_IS_BOUND() at the end of the loop only if we are
>>> bound (orte_odls_globals.bound). Actually that is the only case where I
>>> see a justification to this test (see attached patch).
>>>
>>> And may be both solutions could be mixed.
>>>
>>> Regards,
>>> Nadia
>>>
>>>
>>> --
>>> Nadia Derbey <Nadia.Derbey_at_[hidden]>
>>> <001_fix_process_binding_test.patch>_______________________________________________
>>> devel mailing list
>>> devel_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>
>> _______________________________________________
>> devel mailing list
>> devel_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>
>>

-- 
Oracle
Terry D. Dontje | Principal Software Engineer
Developer Tools Engineering | +1.650.633.7054
Oracle * - Performance Technologies*
95 Network Drive, Burlington, MA 01803
Email terry.dontje_at_[hidden] <mailto:terry.dontje_at_[hidden]>



picture