Open MPI logo

Hardware Locality Users' Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Users mailing list

Subject: Re: [hwloc-users] hwloc on systems with more than 64 cpus?
From: Jirka Hladky (jhladky_at_[hidden])
Date: 2010-05-17 11:41:37


On Monday 17 May 2010 07:37:58 am Brice Goglin wrote:
> On 17/05/2010 00:23, Jirka Hladky wrote:
> > Hi Brice,
> >
> > thanks a lot for the clarification!
> >
> > I got access to 64 cores system and you are indeed right! There is
> > however an issue that taskset does not support 0x80000000,0x0 format.
> >
> > taskset 0x80000000,0x0 sleep 100
> >
> > failed to parse CPU mask 0x80000000,0x0
> >
> > However,
> >
> > taskset 0x8000000000000000 sleep 100
> >
> > works fine:-)
> >
> > Can I suggest an enhancement to hwloc to support taskset format?
> > taskset is currently standard utility to set CPU affinity. Some
> > colleagues of mine don't want to switch to hwloc-bind yet, so
> > supporting taskset format would be great. You can certainly get around
> > with
> >
> > hwloc-calc --proclist --physical 0x80000000,0x0
> >
> > but it will make unnecessarily complex.
> >
> > Could you either add new option --cpuset-taskset-compatible or perhaps
> > change --cpuset output from 0x80000000,0x0
> >
> > to 0x8000000000000000 ?
>
> What does taskset format look like when you want the 128th CPUs ? Does
> it change if you switch from a 32bits architecture to a 64bits
> architecture ?
>
> We have chosen the aforementioned cpuset string format so that it does
> not depend on the architecture. It makes XML files re-usable from one
> machine to another for instance (which is exactly the main point of
> exporting XML files, by the way).
>
> If we want to support taskset format, I think adding an option to lstopo
> wouldn't be enough since some people might want the same option in
> hwloc-distrib or hwloc-calc. So we'd end up adding a new API function.
> And then somebody else will ask for support for another external tool
> and we'd add another dedicated API.
>
> The real question is: why do you still want to use taskset ? hwloc-bind
> should do everything taskset does, and it also support cpusets as
> "node:2.proc:1" or "proc:23-35".
>
> Brice

Hi Brice,

taskset format is same on 32bits architecture on a 64bits architecture.

I don't have system with 128 CPUs, but according to man page it's
0x80000000000000000000000000000000

I have attached the source code as well. See str_to_cpuset function. It
supports cpuset hex mask of any length.

I understand your point to use hwloc-bind. However, I still believe that
supporting taskset would be a useful feature and very easy to implement (you
basically just convert physical CPU # into hex). taskset is currently THE
user-space program to set CPU affinity. I understand that it's probably too
late for this release. But perhaps you can consider it for the next release.

BTW, is there any time-plan for hwloc 1.0 to be released?

Thanks a lot!
Jirka

static int str_to_cpuset(struct bitmask *mask, const char* str)
{
        int len = strlen(str);
        const char *ptr = str + len - 1;
        int base = 0;

        /* skip 0x, it's all hex anyway */
        if (len > 1 && !memcmp(str, "0x", 2L))
                str += 2;

        bitmask_clearall(mask);
        while (ptr >= str) {
                char val = char_to_val(*ptr);
                if (val == (char) -1)
                        return -1;
                if (val & 1)
                        bitmask_setbit(mask, base);
                if (val & 2)
                        bitmask_setbit(mask, base + 1);
                if (val & 4)
                        bitmask_setbit(mask, base + 2);
                if (val & 8)
                        bitmask_setbit(mask, base + 3);
                len--;
                ptr--;
                base += 4;
        }

        return 0;
}