Open MPI logo

PLPA Users' Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all PLPA Users mailing list

From: Bert Wesarg (wesarg_at_[hidden])
Date: 2007-04-28 11:05:20


Now I'm confused too, because of that I start a new thread, and with my
current thinking in exporting the cpu topology.

First some naming conventions:

processor ID:
  the linux virtual processor ID, as in the cpu/ directory (yes, when a
  wrote about a cpu0, I meant cpu with processor ID 0)

node ID:
  the linux node id as in the node/ directory (again, if I wrote sometimes
  node0, I meant the node with ID 0)

(socket,core,thread):
  a tuple that identify exactly one processor ID

Now follow all relations that have been "discovered" so far:

 (a) one-to-one: (socket,core,thread) -> processor ID
 (b) one-to-one: (socket,core,thread) -> node ID
 (c) one-to-one: processor ID -> (socket,core,thread)
 (d) one-to-one: processor ID -> node ID
 (e) one-to-many: node ID -> (socket,core,thread)
 (f) one-to-many: node ID -> processor ID

And now, to something completely different:

No, just joking, here is my API proposal for these relations:

/* Map (socket,core,thread) tuple to virtual processor ID, rel. (a) */
map_to_processor_id(int socket, int core, int thread, int *processor_id);

/* Map a processor ID to (socket,core,thread) tuple, rel. (c) */
map_to_tuple(int processor_id, int *socket, int *core, int *thread);

/* Return the node ID for the given processor ID, rel. (d) */
map_to_node_id(int processor_id, int *node_id);

/* Return the set of processor IDs for a node ID, rel. (f) */
map_from_node_id(int node_id, cpu_set_t *cpuset);

Now comes something new, these replace the max_XXX() functions:
(note: the names are invented on the fly while writing)

Here are the exported relations:

 (g) one-to-many: node ID -> socket
 (h) one-to-many: socket -> core
 (i) one-to-many: (socket,core) -> thread

/* Return the set of all possible processor IDs
   if bit i is set in cpuset, a processor ID with i exist */
get_processor_set(cpu_set_t *cpuset);

/* Return the set of all possible node IDs
   if bit i is set in nodeset, a node ID with i exist */
get_node_set(cpu_set_t *nodeset);

/* Return the set of all sockets
   if bit i is set in socketset, at least one tuple (s, c, t) exist
   with s == i */
get_socket_set(cpu_set_t *socketset);

/* Return the set of all sockets from a node ID
   if bit i is set in socketset, at least one tuple (s, c, t) exist
   with s == i and this tuple is in node node_id
   NOTE: this is the only function that brings the tuples and nodes
         directly into relation
   relation (g) */
get_socket_set_in_node(int node_id, cpu_set_t *socketset);

/* Return the set of all possible cores in a socket
   if bit i is set in coreset, at least one tuple (s, c, t) exist
   with s == socket and c == i
   relation (h) */
get_core_set(int socket, cpu_set_t *coreset);

/* Return the set of all possible threads in a (socket,core) tuple
   if bit i is set in threadset, there exist exactly one tuple (s, c, t)
   with s == socket and c == core and t == i
   relation (i) */
get_thread_set(int socket, int core, cpu_set_t *threadset);

This solves the problem we see on this dual xeon machine with socket id 0
and 3, because we expose no "maximum" to the user.

Here is some pseudo code, that produces a nice topology tree:

cpu_set_t nodeset, socketset, coreset, threadset;
int node, socket, core, thread, cpu;

get_node_set(&nodeset);
cpu_forall(node, &nodeset) {
  get_socket_set_from_node(node, &socketset);
  printf("node %d: with %d sockets\n", node, cpu_count(&socketset));

  cpu_forall(socket, socketset) {
    get_core_set(socket, &coreset);
    printf(" socket %d: with %d cores\n", socket, cpu_count(&coreset));

    cpu_forall(core, &coreset) {
      get_thread_set(socket, core, &threadset);
      printf(" core %d: with %d threads\n", core,
             cpu_count(&threadset));

      cpu_forall(thread, &threadset) {
        map_to_processor_id(socket, core, thread, &cpu);
        printf(" cpu%d <-> (%d, %d, %d)\n",
               cpu, socket, core, thread);
      }
    }
  }
}

Bert