Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: [hwloc-devel] specifying I/O devices on the command-line
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2011-04-12 08:10:10


I am looking for a good way to specify PCI and OS devices on the
command-line (for hwloc-calc and hwloc-bind).

The trunk currently supports:
* os:foobar with for OS device named foobar (eth0, mlx4_0, ...)
* pci:0000:00:00.0 or pci:00:00.0 for a given PCI device
* pci:aaaa:bbbb:c for the c-th PCI device with vendor ID aaaa and device
ID bbbb

The idea is basically to make it easy to bind processes near some
high-performance devices:
   hwloc-bind os:mlx4_0 <mympibenchmark>
   hwloc-bind pci:nvidia:tesla:0 <mycudabenchmark>

Ideally, the os:foobar notation would be enough. But as long as we don't
have any OS name associated with (proprietary) GPUs, people will have to
identify GPUs by their PCI ids.

Other ideas that we may want so support:
* PCI devices by name: something like the 2nd PCI device whose name
contains "tesla C2070" so that people don't have to dig into lspci
manually to find out the vendor/device IDs or busids (mostly useful for
GPUs that have no OS names)
* OS devices by class: something like os:net:2 for the 2nd network
interface (not sure it's useful)

If we want to make all the above even more unreadable, we could also
support specifying a range of indexes (pcifoo:start:amount or
pcifoo:start-end) like we already do for non-I/O objects. But I am not
sure this is actually useful. People may still specify pcifoo:0 pcifoo:1
pcifoo:2 whenever needed.

I/O devices will not be supported through the generic hierarchical
notation "socket:1.core:2..." anyway. So we could make their
command-line specification totally different from the usual one.

It's actually the first time we select objects on something different
than just a type or a depth and some indexes. So we could introduce a
new syntax here. For instance:
   <type>[attributename=attributevalue,...]:index
   <type>[attributename=attributevalue,...]:firstindex:lastindex
   <type>[attributename=attributevalue,...]:firstindex:amount
Not sure it's worth doing this.

Brice