On Tue, May 29, 2012 at 9:05 AM, Jeff Squyres <email@example.com> wrote:
We've tossed around ideas such as having the wrappers always assume dynamic linking (e.g., only include a minimum of libraries), and then add another wrapper option like --wrapper:static (or whatever) to know when to add in all the dependent libraries. Or possibly even look for some popular linker options like --static, or some such (which we've tried to avoid, because that can turn into a slippery slope), but such switches aren't always necessary for MPI-only-static (vs. completely-100%-static) linking. It gets even fuzzier when both libmpi.so and libmpi.a are present. Which way should we assume?
Another problem is backwards compatibility -- users who are currently statically linking will assume the old behavior (of not needing to specify anything additional).
Fair enough. But we've never been able to come up with a rational way to do it (note that pkg-config has its own problems -- OMPI provides pkg-config files in addition to wrapper compilers, but they don't fix everything, either).
> Now I'm not saying that Open MPI should commit to pkg-config instead of wrapper compilers, but the concept of linking differently for static versus shared libraries is something that should be observed.
We have users who both --enable-static and --enable-shared (meaning: both libmpi.so and libmpi.a are present). And therefore we've come down on the conservative side of adding in whatever is necessary for static linking.
The pkg-config approach is to use pkg-config --static if you want to link that library statically.
I suppose, but it does depend on how you define "problem". The linker will ignore any unused libraries -- so it's a problem like lint is a problem. It's annoying, but it doesn't do any harm.
> (Over-linking is an ongoing problem with HPC-oriented packages. We are probably all guilty of it, but tools like pkg-config don't handle multiple configurations well and I don't know of a similar system that manages both static/shared and multi-configuration well.)
...or are there cases where it actually does something harmful?
So the problem is almost exclusively one of binary compatibility. If an app or library is only linked to the interface libs, underlying system libraries can be upgraded to different soname without needing to relink the applications. For example, libhwloc could be upgraded to a different ABI, Open MPI rebuilt, and then the user application and intermediate MPI-based libraries would not need to be rebuilt. This is great for distributions and convenient if you work on projects with lots of dependencies.
It's not such an issue for HPC applications because we tend to recompile a lot and don't need binary compatibility for many of the most common use cases. There is also the linker option -Wl,--as-needed that usually does what is desired.