Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] SWIG bindings
From: Bernd Kallies (kallies_at_[hidden])
Date: 2010-12-02 16:25:44

On Thu, 2010-12-02 at 14:42 -0500, Jeff Squyres wrote:
> On Dec 2, 2010, at 11:07 AM, Bernd Kallies wrote:
> > To implement a hash there
> > exist two ways:
> >
> > a) the easy one: provide a new method like hwloc_get_obj_data(), that
> > gets a "pointer" to the C object and returns a hashref. This would be a
> > simple extension of my current implementation by leaving the children
> > and parent pointers as-is to be used in future calls.
> Do you mean that with this idea, you could do something like:
> my $obj = hwloc_get_...($t, ...);
> my $data = hwloc_get_obj_data($obj);
> print Dumper($data);
> $VAR = {
> type => HWLOC_OJB_CORE,
> os_index => 42,
> name => 'core number 42',
> ...
> };
> Something like that? (I typed that in email, so it's just an approximation) And then that would open the door to something like this:
> my $child_obj = $data{child}[1];
> my $child_data = hwloc_get_obj_data($child_obj);
> print Dumper($child_data);
> $VAR = {
> type => HWLOC_OJB_PU,
> os_index => 98,
> name => 'PU number 98',
> ...
> };
> Or are you saying something else?

Your understanding was right. There may be a method hwloc_get_obj_data
which returns a hasref containing (at least most of) the hwloc_obj_t
struct members in a perl representation. The hwloc_obj_t struct members
that are of type hwloc_obj_t would be still the C pointer values, which
are meaningless outside the API.

> > b) the complicated one: make the perl wrapper lib object oriented. One
> > may translate a hwloc_obj_t struct into a perl object. This sounds
> > straightforward at first sight, and indeed I started with this idea. But
> > very soon I got stuck about what to do with the child and parent
> > pointers and arrays. In the OO implementation these should also be perl
> > objects. One ends up with representing the whole topology as perl
> > objects once the topo gets loaded. In addition, one has to take care
> > about circular references because of the perl reference counters, which
> > makes all this very slow and clumsy.
> Ah, yes, this is what I was thinking of. $obj{parent} would be a pointer to the object of the parent, and $obj{children} would be an array of pointers to the children objects. And so on.
> Do you have much of a feel for how slow this would make it on your machines where lstopo is already expensive?

I tried to represent the whole topology as usual perl hashes with
references to other hashes. Then the problem is the garbage collector of
perl. You know that perl maintains reference counters for everything
that is not a scalar. It frees the memory that is allocated for an
object once its reference counter is zero. This is heavy house-keeping,
especially for the hwloc topology tree (a child object has a reference
to its ancestor, which has references to its children, and each child
has references to its neighbors, which reference their ancestor ...). I
don't remember exactly how much time it took to destroy a big topology,
when it is build up completely from perl objects, but I remember that I
throwed this away because it looked senseless to deal with this idea
further. In addition, circular references in perl have a high potential
to cause memory leaks.

But probably there exists a better solution for the problem by
implementing accessor functions for the attributes of a hwloc_obj_t
object, instead of storing the attribute in a perl hash. I'll think a
bit over this. This requires a bit more knowledge about perl internals
than I currently have.

> > In addition, in the OO
> > implementation one should do true OO, that means one should implement
> > object methods that access object data. The hwloc API is not of this
> > kind.
> Agreed.

> > These were my ideas. Comments are welcome. Note that we have very big
> > machines with complicated topologies. Executing lstopo (pure C API)
> > takes up to a second there. It made no sense to me to slow this down
> > more.
> Also agreed.
> Do you have any feel for if there are particular bottlenecks in hwloc / lstopo that make it take so long? I wonder if we should just attack those (if possible)...? Samuel and Brice have done all the work in the guts of the API, so they might know offhand if there are places that can be optimized or not...

Hmm. I did no profiling. The machines in question have 64 NUMA nodes
with 16 logical CPUs, each. The topology depth is 10. So parsing
of /sys/devices/system/node/* and evaluating the distance matrix to
fiddle out the topology tree should be quite expensive. But I guess this
statement is trivial and does not help very much.

Dr. Bernd Kallies
Konrad-Zuse-Zentrum für Informationstechnik Berlin
Takustr. 7
14195 Berlin
Tel: +49-30-84185-270
Fax: +49-30-84185-311
e-mail: kallies_at_[hidden]