On 2-apr-10, at 13:14, Brice Goglin wrote:
> Fawzi Mohamed wrote:
>> Building tools
>> rather new autoconf/automake/libtool are requested
>> I had to install them even on new clusters, nothing terrible, but I
>> just noted it...
> Yes, but there are nightly snapshots in
> You don't need autotools there.
ok good to know, but I like svn... especially before the release...
>> pages info, before there were free pages, now I suppose that the
>> page_types array might contain several pages, and so also the free
>> pages, any place to get information about the meaning of the various
>> page types? or is it still possible to get the free pages?
> We only had the number of free pages for huge pages if I remember
> correctly. Now we're supposed to have only the total number of pages,
> huge or not, not the number of free ones. But Linux only reports the
> number of free huge pages so that's what we actually show there.
> The different page types are just (one page size + how many such pages
> are available at boot). There's nothing specific about each page type
> except its page size. So when you have hugepages, you have a page type
> for 4kB pages, and another one for 2MB pages for instance. This was
> designed to better cope with multiple page size since most modern
> architectures support multiple huge page sizes. And some OS are
> able to use them. So in the end some machine may show 4 page type
> one for 4kB, one for 64kB, one for 2MB, one for 1GB, ...
ok makes sense, so that one-off scan is enough, a pity I liked having
free huge_pages (not that I really used that), and yes I realize that
even then it was not updated.
>> C bitfields are used, normally they are avoided because they are slow
>> (a compiler can bit or at compile time the constants and check/set
>> several at once.
>> Speed is not a concern here, but in any case I am wrapping to D that
>> does not support bitfields directly, so I used flags, I was just
>> wondering why bitfields were used...
> Which bitfields are you concerned about?
well not really concerned, but I was talking about
hw_topology_support where instead of enum with flags a C bitfield
struct is used...
> Topology flags shouldn't be performance critical. Binding could be a
> more performance critical, but I suspect that even a slow compiler
> generate code that's faster at parsing binding bitfields than the
> binding system call.
>> HWLOC_OBJ_SYSTEM seems on the way out
>> I treated it just as a MACHINE anyway, but I wonder, as the constant
>> is still there, does it have special attributes?, has it machine
>> attributes? the documentation could be clearer...
> This behavior changes completely in 1.0. SYSTEM will only be used when
> assembling multiple independent machines, for instance with kerrighed,
> or maybe for network topologies. MACHINE is always used now, and
> should be very rare.
> SYSTEM doesn't have any specific attribute, but it may have the
> obj->name field set to != NULL so as to tell you what kind of
> multiple-machine-assembly we're talking about.
ok now it is clear
> We're slowly trying to move out of the type-specific attributes as
> as possible. In 1.1, we'll probably have an array of strings to
> custom attributes such as machine name, model, system, ...
good, yes one has always to strike a balance between uniformity/speed,
and flexibility, for things like names,... a generic attribute system
is probably the best choice
>> I am looking forward to 1.0 ...
> Me too, it's been way too long already :) We moved some features out
> 1.0 back to 1.1 (memory binding for instance) because we wanted 1.0
> early. We would probably have had time top implement them twice in the
> meantime ;)
still it is the best lib that I found to detect the structure of a
machine and then bind to it in a cross platform way.
I would take advantage more info about the possible numa node
connectivity (to know where to steal tasks), but I don't have access
to machines that would really take advantage of that, and probably
even then using the HW structure as topology would not bad.
> Could you point me at some precise documentation that is unclear or
> missing so that I try to improve them before the 1.0 final release
I will try to look at what can be improved later...
no thank you for the quick and extensive answers...