Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] [hwloc-users] hwloc 1.5, freebsd and linux output on the same hardware
From: Sebastian Kuzminsky (seb_at_[hidden])
Date: 2012-10-11 12:39:41


This patch (against r4884) fixes the issue on my system. It moves the
lstopo process to cpuset 0, which includes all the CPUs in the system.

--- r4884/hwloc-trunk.svn/tests/ports/topology-freebsd.c 2012-10-02
16:13:06.000000000 -0600
+++ cpuset/hwloc-trunk.svn/tests/ports/topology-freebsd.c 2012-10-11
10:36:18.326408333 -0600
@@ -197,6 +197,7 @@
 void
 hwloc_set_freebsd_hooks(struct hwloc_topology *topology)
 {
+ cpuset_setid(CPU_WHICH_PID, -1, 0);
 #if defined(HAVE_SYS_CPUSET_H) && defined(HAVE_CPUSET_SETAFFINITY)
   topology->set_thisproc_cpubind = hwloc_freebsd_set_thisproc_cpubind;
   topology->get_thisproc_cpubind = hwloc_freebsd_get_thisproc_cpubind;

On Thu, Oct 11, 2012 at 10:33 AM, Sebastian Kuzminsky <
seb_at_[hidden]> wrote:

> The modification of the default cpuset (to exclude CPUs 0-9) is done by
> our local installation. I assume the vanilla FreeBSD 7.3 that we're based
> on does not monkey with the default cpuset.
>
> I believe modifying your own cpuset is not a privileged operation in
> FreeBSD. The cpuset executable is not setuid root and is usable by regular
> users.
>
> I think you can move the lstopo process to cpuset 0 (which includes all
> CPUs) by calling cpuset_setid(), and that should take care of the issue.
>
>
> On Thu, Oct 11, 2012 at 10:13 AM, Brice Goglin <Brice.Goglin_at_[hidden]>wrote:
>
>> (forwarding your mail to the list, and replying)
>>
>> Good to know that it works, thanks for testing. I released 1.5.1rc1
>> today, it should work fine as well.
>>
>> You say that the cpuset does not contain 0-9 by default on this hardware.
>> This is something specific to your installation, I guess? Nothing that
>> happens by default on a default FreeBSD install, right?
>>
>> Is the cpuset-modification a root-only operation on FreeBSD? If so lstopo
>> wouldn't be able to expand the cpuset at startup.
>>
>> lstopo has a --whole-system option to ignore such limitations.
>> Unfortunately the x86 backend that hwloc uses on FreeBSD requires that we
>> bind to each individual core to get its locality information, so that won't
>> help unless lstopo can indeed remove the cpuset first.
>>
>> Brice
>>
>>
>>
>> -------- Message original -------- Sujet: Re: [hwloc-users] hwloc 1.5,
>> freebsd and linux output on the same hardware Date : Thu, 11 Oct 2012
>> 10:02:55 -0600 De : Sebastian Kuzminsky <seb_at_[hidden]><seb_at_[hidden]> Pour :
>> Brice Goglin <Brice.Goglin_at_[hidden]> <Brice.Goglin_at_[hidden]>
>>
>>
>> Ok, it's our fault, sort of.
>>
>> We use cpusets, and by default on this hardware CPUs 0-9 are denied to
>> most processes (including lstopo). If I explicitly change the cpuset of
>> lstopo to include CPUs 0-9, it runs correctly.
>>
>> Maybe lstopo should expand its cpuset to be fully inclusive at startup?
>> I'll be happy to test patches if you want.
>>
>>
>>
>>
>> On Thu, Oct 11, 2012 at 9:27 AM, Sebastian Kuzminsky <
>> seb_at_[hidden]> wrote:
>>
>>> On Thu, Oct 11, 2012 at 2:39 AM, Brice Goglin <Brice.Goglin_at_[hidden]>wrote:
>>>
>>>> Le 06/10/2012 01:03, Sebastian Kuzminsky a écrit :
>>>> > Hm. It may be that we're doing something funny and reserving those
>>>> > CPUs. I'll run some tests on Monday and get back to you.
>>>>
>>>> (replying outside of the hwloc mailing list)
>>>>
>>>> Did you have a chance to look at this?
>>>> I am about to release a hwloc 1.5.1rc1. It'd be good to get your
>>>> feedback before we do the final 1.5.1 (likely next week).
>>>>
>>>
>>> Oops, thanks for the reminder... I got side tracked on other stuff,
>>> but I have some time this morning to devote to this. I'll get back to you
>>> soon.
>>>
>>>
>>> --
>>> Sebastian Kuzminsky
>>> Sr Software Engineer, Linerate Systems
>>>
>>>
>>
>>
>> --
>> Sebastian Kuzminsky
>> Sr Software Engineer, Linerate Systems
>>
>>
>
>
> --
> Sebastian Kuzminsky
> Sr Software Engineer, Linerate Systems
>
>

-- 
Sebastian Kuzminsky
Sr Software Engineer, Linerate Systems