On 2-apr-10, at 13:14, Brice Goglin wrote:

Fawzi Mohamed wrote:
Building tools
rather new autoconf/automake/libtool are requested
I had to install them even on new clusters, nothing terrible, but I
just noted it...

Yes, but there are nightly snapshots in
http://www.open-mpi.org/software/hwloc/nightly/trunk/
You don't need autotools there.

ok good to know, but I like svn... especially before the release...

pages info, before there were free pages, now I suppose that the
page_types array might contain several pages, and so also the free
pages, any place to get information about the meaning of the various
page types? or is it still possible to get the free pages?

We only had the number of free pages for huge pages if I remember
correctly. Now we're supposed to have only the total number of pages,
huge or not, not the number of free ones. But Linux only reports the
number of free huge pages so that's what we actually show there.

The different page types are just (one page size + how many such pages
are available at boot). There's nothing specific about each page type
except its page size. So when you have hugepages, you have a page type
for 4kB pages, and another one for 2MB pages for instance. This was
designed to better cope with multiple page size since most modern
architectures support multiple huge page sizes. And some OS are already
able to use them. So in the end some machine may show 4 page type slots,
one for 4kB, one for 64kB, one for 2MB, one for 1GB, ...

ok makes sense, so that one-off scan is enough, a pity I liked having free huge_pages (not that I really used that), and yes I realize that even then it was not updated.

[...]
C bitfields are used, normally they are avoided because they are slow
(a compiler can bit or at compile time the constants and check/set
several at once.
Speed is not a concern here, but in any case I am wrapping to D that
does not support bitfields directly, so I used flags, I was just
wondering why bitfields were used...

Which bitfields are you concerned about?

well not really concerned, but I was talking about

Topology flags shouldn't be performance critical. Binding could be a bit
more performance critical, but I suspect that even a slow compiler will
generate code that's faster at parsing binding bitfields than the actual
binding system call.

HWLOC_OBJ_SYSTEM seems on the way out
I treated it just as a MACHINE anyway, but I wonder, as the constant
is still there, does it have special attributes?, has it machine
attributes? the documentation could be clearer...

This behavior changes completely in 1.0. SYSTEM will only be used when
assembling multiple independent machines, for instance with kerrighed,
or maybe for network topologies. MACHINE is always used now, and SYSTEM
should be very rare.

SYSTEM doesn't have any specific attribute, but it may have the
obj->name field set to != NULL so as to tell you what kind of
multiple-machine-assembly we're talking about.

ok now it is clear

We're slowly trying to move out of the type-specific attributes as much
as possible. In 1.1, we'll probably have an array of strings to contain
custom attributes such as machine name, model, system, ...

good, yes one has always to strike a balance between uniformity/speed, and flexibility, for things like names,... a generic attribute system is probably the best choice

I am looking forward to 1.0 ...

Me too, it's been way too long already :) We moved some features out of
1.0 back to 1.1 (memory binding for instance) because we wanted 1.0
early. We would probably have had time top implement them twice in the
meantime ;)

still it is the best lib that I found to detect the structure of a machine and then bind to it in a cross platform way.

I would take advantage more info about the possible numa node connectivity (to know where to steal tasks), but I don't have access to machines that would really take advantage of that, and probably even then using the HW structure as topology would not bad.