Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] Merging the PCI branch?
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2011-03-31 16:22:16

Le 31/03/2011 18:06, Jeff Squyres a écrit :
> Good. Although I think we should plan to make this the default in some future version (i.e., say that in the docs).

I agree with Samuel on this. Keep things people for basic users. Without
I/O devices, the topology is usually very simple, making users' code
very simple too (all objects have a cpuset, all children have the same
type, topology is symmetric, ...). If you enable I/O, you need to make
your code much more complex, which is good, but may prevent some users
from using the C API.

Note that I/O is enabled by default in lstopo :)

>> I also added some GPU-related OS devices by looking at DRM objects
>> (card0 and controlD64 in the attached picture). This only works with
>> free graphics drivers. Ideally we would have some Cuda or OpenCL device
>> ID there, but we'll likely need some specific plugins to do so. I don't
>> know if the current DRM objects are useful, we'll be able to remove them
>> later if needed.
> We should ping Intel, NVIDIA, ... others for assistance with this.

The solution will likely not be in sysfs because only GPL drivers can
put stuff in sysfs. That's why only the free graphics drivers are
already visible with the current code.

For CUDA, we already have something similar to what we need in
hwloc/cuda.h. However, this requires the cuda library.

For OpenCL, there's nothing to help you translate from PCI devices into
software handles in the 1.1 API iirc.

> Do iWARP and RoCE devices show up, too? I.e., should it be

>From I understand to the kernel code, the sysfs file we are reading come
from ib_register_device, which is called by IB, IBoE/RoCE and iWARP
drivers. The only difference is the node_type attribute. So indeed,
openfabrics might be a better name.