We use lt_dlopen() to open the plugins (Libtool's wrapper for a
portable dlopen). It opens all plugins (DSOs) in a private scope.
That private scope is kept deep in the OPAL MCA base and not exposed
elsewhere in the code base. So if you manually dlopen a plugin again,
I'll bet that the linker realizes that that DSO has already been
loaded into the process space and doesn't actually load it again (but
doesn't fail). So the dlsyms fail because you don't have access to
the private scope from where Libtool originally opened the DSO.
Shouldn't it work to re-dlopen the lib with RTLD_GLOBAL?
I used dlopen("...", RTLD_LAZY | RTLD_GLOBAL). It gave me a non-NULL
handle, but subsequent dlsyms failed.
Also, recent libltdl should allow you to choose which scope you want in
the first place, local or global, through lt_dladvise.
I'm just learning all this dl stuff right now. Jeff's --enable-static
seems to do exactly what I want (namely, make things work in the way
that I'm familiar with!). I did try to figure out what OMPI was doing
and it seemed to me it was using RTLD_LAZY | RTLD_GLOBAL, which is why
I chose that.
For now, --enable-static seems to do exactly what I want. Further
workarounds probably don't make any sense.