Oh this actually does make a lot of sense. The kicker is that Rmpi doesn't like to use OMPI and really loves to use LAM so I have to use R in batch mode by running mpirun -np 12 and specify the host file. I have a bad feeling that this is loading the library 12 times, once for each R namespace. While this is annoying, not having Rmpi working was far more so and ram is cheap. I do agree whole heatedly that dynamic libraries are far superior to static ones but I wonder if setting the configuration by default to build both would allow those applications which require the static libraries to compile to compile but at the same time getting the applications which use dynamic loading libraries to exist side by side. I don't plan on writing applications which require -libmpi but I know that there are developers not nearly in tune with this line of thinking or know the difference between static and dynamic libraries.
Also this was tripping me up for weeks. I will let the Rmpi developer know though and hopefully they can shift code to the dynamic libraries or perhaps take part in development of the library to make it load dynamic libraries.
Anyway Jeff, thank you for a wonderful explication. I wonder if this should be posted anywhere on OpenMPI as a note for Rmpi developers which desire to compile OMPI themselves. The packages which exist out there are multiple years old at this point, at least for ubuntu and the released version was actually one of your development releases and had several large bug fixes since then.
BTW, I don't see myself not using OMPI for a while, are there any simple projects which I might be able to work on to get to know the code base and maybe move up the chain? Fixing some low hanging fruit bugs and learning how to debug OMPI would be optimal.
Thanks again,
~Ben
On Fri, Apr 6, 2012 at 8:45 AM, Jeffrey Squyres
<jsquyres@cisco.com> wrote:
On Apr 5, 2012, at 9:07 PM, Benedict Holland wrote:
> Oh how interesting and I hope this helps someone. Following another link, I had to use:
>
> ./configure --prefix /usr --enable-shared --enable-static
This makes sense. You were falling victim to the fact that R dlopens libmpi as a dynamic library in a private namespace. Hence, when Open MPI then dlopens its own plugins, it can't find libmpi's symbols. This is a generic problem with any system that opens plugins that, themselves, open plugns. I wish there was a better solution to this -- the OS guys need to give us a better mechanism here. :-(
OMPI's --enable-static option does two things:
- it builds libmpi.a (vs. libmpi.so)
- it slurps all of OMPI's plugins into libmpi.a (so it doesn't need to dlopen anything at run-time)
It's the latter point that is saving you.
Note that you could also just --disable-dlopen (vs. --disable-shared --enable-static), which just does the latter of the above things (meaning: OMPI still builds libmpi.so), and it should also work for you.
> when compiling this for Rmpi. Just curious, why isn't --enable-static a default option?
Among other reasons, shared libraries generally help save memory at run time. This is somewhat important as core counts go up. If you "mpirun -np 32" on a single, 32-core machine, would you rather have 32 independent copies of libmpi.a loaded into RAM, or just one copy that all processes share?
Using libmpi.so enables the latter option.
Make sense?
--
Jeff Squyres
jsquyres@cisco.com
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/