Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: [hwloc-devel] userdata import/export
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2012-08-26 05:52:43


(breaking the thread since we're far from valarrays now).

I've played with this idea and came with the attached interface change.
Aside from low-level backend changes, everything was very easy to implement.

Let's take an example. I modified lstopo to add some userdata to the
root object and define the following export callback:

static void export(void *reserved, hwloc_topology_t topo, hwloc_obj_t obj)
{
  /* we export random stuff instead of the actual userdata content */
  hwloc_export_obj_userdata(reserved, topo, obj, NULL, "coincoin", 8); /* no name */
  hwloc_export_obj_userdata(reserved, topo, obj, "MyName", "foobar", 6);
  hwloc_export_obj_userdata(reserved, topo, obj, "MyValue", "foobarbarbar", 10); /* truncated to 10 chars */
}

This callback is only called for the root object since others have
userdata=NULL. It exports three lines to XML:

    <userdata length="8">coincoin</userdata>
    <userdata name="MyName" length="6">foobar</userdata>
    <userdata name="MyValue" length="10">foobarbarb</userdata>

The idea behind multiple userdata lines per object is that it helps the
application structure its userdata without having to create a single
contigous buffer.

When you import this with an import() callback, you get three
invocations of the callback:

##### name (null) value coincoin length 8
##### name MyName value foobar length 6
##### name MyValue value foobarbarb length 10

The optional "name" attribute lets the application recognize things (but
they always looked in-order in my testing).

My user that wants valarrays would likely export:
* the number of elements in his arrays as a first line
* the content of his latency array as a second line
* the content of his bandwidth array as a third line

We set import() and export() callbacks with two different function
because it makes the doc much more clear and the requirements are
different (import() must be set before load(), export() must use
export_userdata()).

I am satisfied with all this, I don't have any concern with the
genericity of the interface anymore.

Now, I still think we should offer some optional basic encoding
capability since many users have no clue about XDR or base64. For
instance, add export_userdata_encoded() on the side of
export_userdata(). On the import side, we could let the application
decode with a dedicated hwloc function. But I think I would rather mark
the XML line as "encoded" so that hwloc transparently decodes before
passing the data to the import() callback.

Brice