Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] hwloc powerpc rhel5 and power7 patch
From: Alexey Kardashevskiy (aik_at_[hidden])
Date: 2010-09-16 01:57:47


On 16/09/10 15:41, Brice Goglin wrote:
> Le 16/09/2010 06:10, Alexey Kardashevskiy a écrit :
>
>
>> 2. The HWLOC expects numa nodes to be numbered consecutively, like
>> 1-2-3-4-5.... However this is not necessary true for PowerPC with
>> LPARs or on systems with numa hotswap (do they exist? don't know).
>>
> Yes, I've never implemented any sparse-aware code since I haven't ever
> seen sparse-numbered system :)
>
>

I posted tgz - you may take a look :)

>
>> - where do I put IBM-specific code?
>>
> Is the device tree linux-specific ? If so, it can stay in linux file as
> long as it's not 30k lines :) We already have both sysfs and
> /proc/cpuinfo code there anyway.
>

It is powerpc-specific. It is mapped from the system firmware (aka bios)
by the powerpc kernel. However it is just a folder within /proc so it is
usual linux folder. But PowerPC might be not the only architecture which
uses the same pathname for the same thing.

>> - may be there is a better way to detect that no cache info was
>> fetched from sysfs
>>
> That's something that's not clear to me yet. There will likely be other
> cases in the future where we will fetch some info from different
> backends, and merging them may not be easy. Do you think the device tree
> generally contains more information than sysfs? If so, we could start by
> disabling cache info from sysfs when a device-tree is found, and maybe
> have a way to change that default (we already have a hidden en variable
> to use cpuinfo when sysfs is available).
>
>
See my note above about the system firmware :) Almost every powerpc
system has device-tree no matter which OS you run on it (sony ps3 is
probably the only exception). /proc/device-tree is the only source for
sysfs on powerpc linux.

>> - is the coding style ok? :-)
>>
> It doesn't look bad.
>

The current topology-linux.c consist of several coding styles so I could
not detect which one is primary :)

> One question though: Is the device tree completely save-able for
> external reuse? We like being able to save /proc and /sys so as to debug
> distant machines locally. Doing the same for the device tree would be
> great. If so, could you send a tarball of a machine with sparse-numa
> numbers? And we'll likely make gather-topology.sh store it too.
>

I am attaching a result of the "tar czf 256cpu_device_tree.tgz
/proc/device-tree/" command. Looks good. TAR complained multiple times
that "file changed as we read it" though.

ps: I get every mail twice. I know I screwed up with subscription but
how do I or somebody fix that? I need to have only aik_at_[hidden]
subscribed. Thank you :)