Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] 1.3.1 and 1.4rc1
From: Samuel Thibault (samuel.thibault_at_[hidden])
Date: 2011-12-13 12:45:17

Brice Goglin, le Tue 13 Dec 2011 18:13:52 +0100, a écrit :
> Le 13/12/2011 18:02, Samuel Thibault a écrit :
> > Brice Goglin, le Tue 13 Dec 2011 16:41:08 +0100, a écrit :
> >> Le 13/12/2011 16:22, Jeff Squyres a écrit :
> >>> I can't speak for GPUs, but I think the PCI information will be useful to know what devices are close to what PUs / NUMA nodes. That information can be used to make decisions about binding, for example (i.e., you want to be "close" to the specific PCI resources that you'll be using).
> >> In most cases, you don't need PCI support for this, you juste manipulate
> >> a cuda device, an ibv_device, a MX endpoint, ... and use one of the
> >> inline helpers to get the corresponding locality (a cpuset).
> > A problem I see there is that although you can write such helper for
> > Linux, nothing guarantees that it'll be so easy for other OSes.
> And ? :)
> We already have many features that are only supported on Linux. OS
> devices for instance.

Yes, and these are through library functions, not helpers. Whenever
hwloc gets support for other OSes, you don't have to change your
application to benefit from the new support. The current linux-only
helpers can't even be compiled on non-Linux OSes, so applications have
to deal with it themselves, which is a bad thing. linux-libnuma is an
example where it can only be useful on Linux, so that's no problem. But
the glibc helpers should work on all glibc-based systems. I don't know
the portability of myriexpress.h and infiniband/verbs.h, but the CUDA
helpers, for instance, should work on windows too.

> libpci isn't portable to all OS we support, ...

That's no reason. Other OSes have other interfaces, that's fine, that's
what hwloc is there for...

> As long as we don't add something obviously not portable, I am fine.

The current openfabrics-verbs.h and cuda*.h are obviously not portable.