On Aug 6, 2008, at 5:09 AM, Victor Pasko wrote:
> Well, why not for Leopard too? It's Unix.
Unix != Unix != Unix. :-)
Solaris, Linux, and OS X all have completely different processor and
memory affinity mechanisms -- there's no common API.
> BTW, take a look at http://developer.apple.com/releasenotes/Performance/RN-AffinityAPI/index.html
>
> "With Leopard, Apple has released a Thread Affinity API to provide
> thread affinity hints to the kernel scheduler for improving data
> locality in caches, among other uses. While it is still not a full
> implementation like sched_setaffinity() is in Linux, it should make
> parallel programming a little bit more effective and portable"
Yes and no. "Portable" is questionable, since this is a Leopard-
specific API. :-) Perhaps they mean "portable" in a performance
sense, such as across different processor architectures, all running
OS X.
The API described on the URL above provides hints to the kernel about
which threads want to be "together" (e.g., share an L2 cache). It
does not allow specific placement of threads or processes to
processors/cores, like Linux's sched_setaffinity(). There are
benefits and drawbacks to each approach; I'm not saying which is
better -- only that they are different. Specifically: I don't think
that we could emulate their API with PLPA's
plpa_sched_set_affinity(). Additionally, AFAIK, OS X does not export
the same kind of processor topology information that Linux does.
That being said, I think that there is [high] value in trying to come
up with a uniform approach for processor affinity across multiple
OS's, but PLPA is probably not the project to do it in. Processor
affinity is a problem that most app developers don't yet know that
they have. Having a common API that works on POSIX systems would be
great.
My $0.02: as we go to manycore, we need "something better" than the
current API's. I don't know what this API is, but there needs to be a
balance between:
- applications that require discovery/knowledge about resource
location and distance
- applications that require guarantees about resource location and
distance within specific timeframes
- OS's and resource managers that want to control all affinity
behavior and resource allocation
- OS's and resource managers that do not want to give explicit control
of affinity
For example, on a manycore machine, an MPI implementation (http://www.mpi-forum.org/
) wants to nail down all its resource location and distance between
MPI_INIT and MPI_FINALIZE -- the information should never change
between the two. But before you can, you need to know a) how many
peer processes you have, b) what I/O devices exist, c) the length /
contention factor of each route to each I/O device (multipath doesn't
yet exist, but it may someday...?), and d) what other processes in the
same host are doing (e.g., if there are other MPI processes running on
the same host as your MPI processes). Indeed, an MPI process'
decision about which [I/O] resources to use may be influenced by the
location and distance of those resources.
There's probably lots of other things that you'd want to know, too --
this is off the top of my head (admittedly with some prior
conversations with some colleagues on this kind of topic). There's a
fair amount of research that needs to happen in this area. I'd love
to see someone do it. :-)
Once someone comes up with reasonable abstractions, we need a common
POSIX API to effect them. However, such a POSIX API would take a
serious, cross-organization collaboration to come up with such a beast
-- it would be a large project. To make it worthwhile, you'll might
want to get buy-in from Sun, the Linux community, and Apple --
possibly others? For extra bonus points: Microsoft. Perhaps a proof-
of-concept implementation that is only single-OS would be a good
start. Shrug.
--
Jeff Squyres
Cisco Systems
|
![[Flag]](../../../../../community/mirrors/flag-us.gif){kind=small}