On Apr 12, 2011, at 8:10 AM, Brice Goglin wrote:
> I am looking for a good way to specify PCI and OS devices on the
> command-line (for hwloc-calc and hwloc-bind).
> The trunk currently supports:
> * os:foobar with for OS device named foobar (eth0, mlx4_0, ...)
> * pci:0000:00:00.0 or pci:00:00.0 for a given PCI device
> * pci:aaaa:bbbb:c for the c-th PCI device with vendor ID aaaa and device
> ID bbbb
> The idea is basically to make it easy to bind processes near some
> high-performance devices:
> hwloc-bind os:mlx4_0 <mympibenchmark>
> hwloc-bind pci:nvidia:tesla:0 <mycudabenchmark>
Can you list multiple devices? E.g.:
hwloc-bind os:mlx4_0 os:mlx4_1 my_mpi_benchmark
Also, is there a CLI way to retrieve which numa nodes / OS processors are near such devices? I can imagine wanting to script up something like:
- retrieve a mask / list of processors near OS device <foo>
- binding N processes, one per processor, to the processors near that device
> Ideally, the os:foobar notation would be enough. But as long as we don't
> have any OS name associated with (proprietary) GPUs, people will have to
> identify GPUs by their PCI ids.
> Other ideas that we may want so support:
> * PCI devices by name: something like the 2nd PCI device whose name
> contains "tesla C2070" so that people don't have to dig into lspci
> manually to find out the vendor/device IDs or busids (mostly useful for
> GPUs that have no OS names)
I immediately had that question when I read your 2nd example, above (i.e., where did you get the names from?). Are these names in the lstopo output?
> * OS devices by class: something like os:net:2 for the 2nd network
> interface (not sure it's useful)
I'm not sure it is -- isn't the ordering of PCI devices non-deterministic between cold boots?
> If we want to make all the above even more unreadable, we could also
> support specifying a range of indexes (pcifoo:start:amount or
> pcifoo:start-end) like we already do for non-I/O objects. But I am not
> sure this is actually useful. People may still specify pcifoo:0 pcifoo:1
> pcifoo:2 whenever needed.
For consistency, it doesn't sound like a bad idea.
> I/O devices will not be supported through the generic hierarchical
> notation "socket:1.core:2..." anyway. So we could make their
> command-line specification totally different from the usual one.
> It's actually the first time we select objects on something different
> than just a type or a depth and some indexes. So we could introduce a
> new syntax here. For instance:
> Not sure it's worth doing this.
It might be better to just put out basic functionality in 1.3 and *not* do advanced syntax like this (i.e., only do basic syntax). And then see what people ask for.
For corporate legal information go to: