Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] release status
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2009-10-02 08:33:18


(we've been having some off-list discussions about when to release
0.9.1 -- we decided that we wanted to add dynamic CPU sets as the last
feature before release. So now we've been talking about how/what to
do for dynamic CPU sets. It only occurred to me mid-thread that there
was no reason to have these discussions off-list, so I'm moving the
discussion to hwloc-devel. Hopefully that's enough context to pick up
this thread mid-stream...)

On Oct 2, 2009, at 8:19 AM, Samuel Thibault wrote:

>> There's an ugly ABI truth to this methodology that we got bitten by
>> in
>> OMPI. :-\ If the struct changes size from version A to B, the ABI
>> changes (meaning that we'll have to bump the "incompatible" flag in
>> the so version). It's completely non-intuitive, but I swear it's
>> true. :-(
>
> Err, because we are using inlines that's true, yes, and so
> recompilation
> is needed, but is it really a problem?

Right -- we already established that we're not interested in ABI -- do
the Right Things with the so version and let that be enough. I just
wanted to point it out because it's a common misconception that just
passing a function pointer as a handle makes you future proof (we
learned the hard way).

> Else we may not inline cpuset functions and in that case the
> application
> never gets in contact with the actual size of the pointed structure, I
> don't see how there can be ABI issues.

1) I vote for not inlining -- do we *really* care about the
performance that much? I don't think it matters enough to care.

2) The linker needs the actual size of the struct (even if it's not
visible in the C language) to link against the .so properly and do
addressing / pointer math properly to find global handle symbols. In
short, the *executable* has the final size of the struct encoded in
it. This is non-intuitive, but I swear it's true. Try this:

private.h:
struct foo {
   int a;
};

private.c:
#include <private.h>
struct foo public_foo_instance;
struct foo *public_handle = &public_foo_instance;

public.h:
struct foo;
typedef struct foo *handle_t;
extern struct foo *public_handle;

Compile, make, and install the above as libmiddleware.so.

my_app.c:
#include <public.h>
handle_t my_handle = public_handle;
int main() { return 0; }

Compile and make my_app, linking against -lmiddleware. It compiles,
links, and runs fine.

Now go change private.h and add another member to the struct.
Recompile and re-install libmiddleware. Now run my_app again --
without re-compiling/re-linking. You'll get warnings from the linker
about how the struct changed size.

This may all be moot if we have no global handle instances (akin to
MPI_COMM_WORLD) that are used outside of the middleware...?

-- 
Jeff Squyres
jsquyres_at_[hidden]