Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] merging the valarray branch (with a better name)
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2012-08-24 18:52:54

Le 24/08/2012 23:44, Jeff Squyres a écrit :
> On Aug 24, 2012, at 5:17 PM, George Bosilca wrote:
>>> hwloc_obj_t already has a "void *userdata" for this. But we cannot store
>>> it in XML unless we know what it contains.
>> Contiguous binary structures with a known size can be stored in a XML file. Instead of forcing the upper level to store their meaningful information as arrays of one of a few predefined types, let them deal with the content of the binary object, and focus on preserving the binary objects in the XML structure.
>>> Exporting to XML is strictly required here since people want to add
>>> values to the XML topology in a preliminary benchmarking programs, and
>>> later read it back in their actual application (a charm++ scheduler).
>> XDR exists for ensuring consistency. Let them store what they want not just arrays.
> These are good points. Perhaps userdata is enough, and if they want the userdata to be in the XML, they can provide a callback to return an XML blob (which may be a set of new attributes, or perhaps just one big binary blob). If they don't provide a callback, then the user data is not included in the blob.

This could be:
* hwloc_set_topology_userdata_blob_callbacks(topology, exportcb, importcb)
* On XML export, hwloc calls export(obj, &buffer, &len), the application
callback allocates a contigous buffer if needed, put stuff in there,
returns it, and we store buffer/len in XML, and we free the buffer if
needed (force the application to always allocate? or add a callback flag
to know whether we have to free or not?)
* On XML import, we read the XML stuff and call import(obj, buffer,
len). The application would usually read the blob and store it back in
the object. We'll have to document that the topology isn't ready for
consulting there yet (or queue callbacks to the end of the XML parsing).

We'd likely use something like base64 to encode these blobs so that the
minimalistic parser doesn't get lost (it doesn't like unusual characters).

Another thing I am not sure about is whether a single userdata pointer
is enough. Maybe allow multiple (name,data) key pairs exactly like we do
for string infos? If so, there's no need for topology-wide
hwloc_set_topology_userdata_blob_callback(), the application gives the
right callbacks when it adds the (key,data) pair.