On Mar 22, 2007, at 7:55 AM, Jeff Squyres wrote:
> On Mar 15, 2007, at 12:18 PM, Michael wrote:
>> Situation: I'm compiling my code locally on a machine with just
>> ethernet interfaces and OpenMPI 1.1.2 that I built.
>> When I attempt to run that executable on a HPC machine with OpenMPI
>> 1.1.2 and InfiniBand interfaces I get messages about "can't find
>> libmosal.so.0.0" -- I'm certain this wasn't happening earlier.
>> I can compile on this machine and run on it, even though there is no
>> libmosal.* in my path.
>> mpif90 --showme on this system gives me:
>> /opt/compiler/intel/compiler91/x86_64/bin/ifort -I/opt/mpi/x86_64/
>> intel/9.1/openmpi-1.1.4/include -pthread -I/opt/mpi/x86_64/intel/9.1/
>> openmpi-1.1.4/lib -L/opt/mpi/x86_64/intel/9.1/openmpi-1.1.4/lib -L/
>> opt/gm/lib64 -lmpi_f90 -lmpi -lorte -lopal -lgm -lvapi -lmosal -lrt -
>> lnuma -ldl -Wl,--export-dynamic -lnsl -lutil -ldl
> Based on this output, I assume you have configured OMPI with either --
> enable-static or otherwise including all plugins in libmpi.so, right?
No, I did not configure OpenMPI on this machine. I believe OpenMPI
was configured not static by the installers based on the messages and
the dependency on the missing libraries.
The issue was that some of the 1000+ nodes on this major HPC machine
were missing libraries needed for OpenMPI but because of the low
usage of OpenMPI I'm the first to discover the problem. For whatever
reason these libraries are not on the front-end machines that feed
the main system. It's always nice running OpenMPI on your own
machine but not everyone can always do that.
The way I read my experience is that OpenMPI's libmpi.so depends on
different libraries on different machines, this means that if you
don't compile static you can compile on a machine that does not have
libraries for expensive interfaces and run on another machine with
those expensive interfaces -- that's what I'm am doing now successfully.