Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [hwloc-devel] Merging the PCI branch?
From: Brice Goglin (Brice.Goglin_at_[hidden])
Date: 2011-03-31 16:22:16


Le 31/03/2011 18:06, Jeff Squyres a écrit :
>
> Good. Although I think we should plan to make this the default in some future version (i.e., say that in the docs).
>

I agree with Samuel on this. Keep things people for basic users. Without
I/O devices, the topology is usually very simple, making users' code
very simple too (all objects have a cpuset, all children have the same
type, topology is symmetric, ...). If you enable I/O, you need to make
your code much more complex, which is good, but may prevent some users
from using the C API.

Note that I/O is enabled by default in lstopo :)

>> I also added some GPU-related OS devices by looking at DRM objects
>> (card0 and controlD64 in the attached picture). This only works with
>> free graphics drivers. Ideally we would have some Cuda or OpenCL device
>> ID there, but we'll likely need some specific plugins to do so. I don't
>> know if the current DRM objects are useful, we'll be able to remove them
>> later if needed.
>>
> We should ping Intel, NVIDIA, ... others for assistance with this.
>

The solution will likely not be in sysfs because only GPL drivers can
put stuff in sysfs. That's why only the free graphics drivers are
already visible with the current code.

For CUDA, we already have something similar to what we need in
hwloc/cuda.h. However, this requires the cuda library.

For OpenCL, there's nothing to help you translate from PCI devices into
software handles in the 1.1 API iirc.

> Do iWARP and RoCE devices show up, too? I.e., should it be
> "INFINIBAND" or "OPENFABRICS"?

>From I understand to the kernel code, the sysfs file we are reading come
from ib_register_device, which is called by IB, IBoE/RoCE and iWARP
drivers. The only difference is the node_type attribute. So indeed,
openfabrics might be a better name.

Brice