Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] comments on API changes
From: Fawzi Mohamed (fawzi_at_[hidden])
Date: 2010-04-02 08:38:26


On 2-apr-10, at 13:14, Brice Goglin wrote:

> Fawzi Mohamed wrote:
>> Building tools
>> rather new autoconf/automake/libtool are requested
>> I had to install them even on new clusters, nothing terrible, but I
>> just noted it...
>
> Yes, but there are nightly snapshots in
> http://www.open-mpi.org/software/hwloc/nightly/trunk/
> You don't need autotools there.

ok good to know, but I like svn... especially before the release...

>> pages info, before there were free pages, now I suppose that the
>> page_types array might contain several pages, and so also the free
>> pages, any place to get information about the meaning of the various
>> page types? or is it still possible to get the free pages?
>
> We only had the number of free pages for huge pages if I remember
> correctly. Now we're supposed to have only the total number of pages,
> huge or not, not the number of free ones. But Linux only reports the
> number of free huge pages so that's what we actually show there.
>
> The different page types are just (one page size + how many such pages
> are available at boot). There's nothing specific about each page type
> except its page size. So when you have hugepages, you have a page type
> for 4kB pages, and another one for 2MB pages for instance. This was
> designed to better cope with multiple page size since most modern
> architectures support multiple huge page sizes. And some OS are
> already
> able to use them. So in the end some machine may show 4 page type
> slots,
> one for 4kB, one for 64kB, one for 2MB, one for 1GB, ...

ok makes sense, so that one-off scan is enough, a pity I liked having
free huge_pages (not that I really used that), and yes I realize that
even then it was not updated.

> [...]
>> C bitfields are used, normally they are avoided because they are slow
>> (a compiler can bit or at compile time the constants and check/set
>> several at once.
>> Speed is not a concern here, but in any case I am wrapping to D that
>> does not support bitfields directly, so I used flags, I was just
>> wondering why bitfields were used...
>
> Which bitfields are you concerned about?

well not really concerned, but I was talking about
hw_topology_support where instead of enum with flags a C bitfield
struct is used...

> Topology flags shouldn't be performance critical. Binding could be a
> bit
> more performance critical, but I suspect that even a slow compiler
> will
> generate code that's faster at parsing binding bitfields than the
> actual
> binding system call.
>
>> HWLOC_OBJ_SYSTEM seems on the way out
>> I treated it just as a MACHINE anyway, but I wonder, as the constant
>> is still there, does it have special attributes?, has it machine
>> attributes? the documentation could be clearer...
>
> This behavior changes completely in 1.0. SYSTEM will only be used when
> assembling multiple independent machines, for instance with kerrighed,
> or maybe for network topologies. MACHINE is always used now, and
> SYSTEM
> should be very rare.
>
> SYSTEM doesn't have any specific attribute, but it may have the
> obj->name field set to != NULL so as to tell you what kind of
> multiple-machine-assembly we're talking about.

ok now it is clear

> We're slowly trying to move out of the type-specific attributes as
> much
> as possible. In 1.1, we'll probably have an array of strings to
> contain
> custom attributes such as machine name, model, system, ...

good, yes one has always to strike a balance between uniformity/speed,
and flexibility, for things like names,... a generic attribute system
is probably the best choice

>> I am looking forward to 1.0 ...
>
> Me too, it's been way too long already :) We moved some features out
> of
> 1.0 back to 1.1 (memory binding for instance) because we wanted 1.0
> early. We would probably have had time top implement them twice in the
> meantime ;)

still it is the best lib that I found to detect the structure of a
machine and then bind to it in a cross platform way.

I would take advantage more info about the possible numa node
connectivity (to know where to steal tasks), but I don't have access
to machines that would really take advantage of that, and probably
even then using the HW structure as topology would not bad.

> Could you point me at some precise documentation that is unclear or
> missing so that I try to improve them before the 1.0 final release
> arrives?

I will try to look at what can be improved later...

> thanks,

no thank you for the quick and extensive answers...

ciao
Fawzi