Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] [hwloc-users] hwloc_get_last_cpu_location on AIX
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2012-07-09 09:39:57

Thanks a lot Hendryk, I committed this to trunk. I am now looking at
making *set*_cpubind better at changing old-style binding. We'll have
all this in hwloc v1.5 (hopefully by the end of the month).

On 09/07/2012 10:41, Hendryk Bockelmann wrote:
> Hello Brice,
> I just tested your patch with the nightly build
> hwloc-1.5a1r4597.tar.gz on our POWER6 cluster for some OpenMP and some
> hybrid code. In both cases the reported binding is exactly what I get
> from the IBM tool and also exactly what it should look like (since I
> asked the LoadLeveler to distribute my MPI tasks and OpenMP threads in
> an unusual way).
> So far so good, thank you for the changed within hwloc. I will now use
> the hwloc API to work on binding issues in some user codes.
> Best,
> Hendryk
> On 15.06.2012 10:45, Brice Goglin wrote:
>> Hello Hendryk,
>> I am re-adding hwloc-devel to CC since we have a non-trivial patch
>> attached and discussion below.
>> I talked to your IBM contact and he was indeed able to help, thanks. So
>> there are indeed two different binding interfaces on AIX. hwloc only
>> support rsets, that's why we don't see binding made with the other
>> (older) interface (bindprocessor) in the XL OpenMP runtime. Fortunately,
>> both interfaces cannot be use in a contradictory way, so we just need to
>> fallback to the other interface when we find no rset binding.
>> The attached patch is supposed to implement this. I didn't test all
>> cases (current/other process/thread) yet. Let me know if it works fine
>> in your MPI+OpenMP program.
>> On set_cpubind() side, we should likely add some code to handle
>> conflicts with the older interface. If we get EPERM when setting a rset,
>> we should unbind with the old interface and try again. Unfortunately,
>> only the entire process can be unbound, single threads can't, from what
>> I understand. So we could get problems if the application binds multiple
>> threads with the old interface and then binds a single thread with
>> hwloc. But that'd be rare and bad anyway.
>> What will remains is get_last_cpu_location() for anything but the
>> current thread. I don't see any way to implement this.
>> Brice