Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: [hwloc-devel] [RFC] multinode topology
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2011-08-23 13:22:23


I am working on a prototype implementation for agregating multiple node
topologies into a single global one. I was initially thinking of doing a
    int hwloc_topology_insert_from_xml(newtopology, newparent, xmlsrc)
but it would require non-trivial changes to the core without being very
generic. So I came with the following solution:
    int hwloc_topology_insert_topology(newtopology, newparent, oldtopology);

The advantage is that you can insert topologies from wherever you want:
existing topology in memory, xml buffer, xml file, synthetic, ... You
just need the corresponding topology object.
hwloc_topology_insert_topology() just duplicates the existing topo
inside another one under the "newparent" object.

The drawback is that it's slower because you load the topology once and
then duplicate it into the other one. But I don't expect topology
agregation to be performance critical so it should be OK.

Here's an example that already works in my tree after fixing a couple
things in the core:

  hwloc_topology_t local, global;
  hwloc_obj_t sw1, sw2, root;

  printf("Loading the local topology...\n");
  hwloc_topology_init(&local);
  hwloc_topology_load(local);

  printf("Creating a custom topology...\n");
  hwloc_topology_init(&global);
  hwloc_topology_set_custom(global);

  printf("Inserting the local topology into the global one...\n");
  root = hwloc_get_root_obj(global);
  sw1 = hwloc_topology_insert_misc_object_by_parent(global, root, "Switch");
  hwloc_topology_insert_topology(global, sw1, local);
  hwloc_topology_insert_topology(global, sw1, local);
  sw2 = hwloc_topology_insert_misc_object_by_parent(global, root, "Switch");
  hwloc_topology_insert_topology(global, sw2, local);
  hwloc_topology_insert_topology(global, sw2, local);

  hwloc_topology_destroy(local);

  printf("Building the global topology...\n");
  hwloc_topology_load(global);
  hwloc_topology_export_xml(global, "foo.xml");
  hwloc_topology_destroy(global);

Guess what: lstopo -i foo.xml shows one System object above two Misc
objects (representing switches) above four identical machines.

hwloc_topology_set_custom() initializes my new "custom" backend. It
basically means that I am going to insert Misc objects and/or topologies
that hwloc_topology_load() will just have to connect together later.

Brice