Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Passing LD_LIBRARY_PATH to orted
From: Craig Tierney (Craig.Tierney_at_[hidden])
Date: 2008-10-14 17:18:53


Ralph Castain wrote:
> I -think- there is...at least here, it does seem to behave that way on
> our systems. Not sure if there is something done locally to make it work.
>
> Also, though, I have noted that LD_LIBRARY_PATH does seem to be getting
> forwarded on the 1.3 branch in some environments. OMPI isn't doing it
> directly to the best of my knowledge, but I think the base environment
> might be. Specifically, I noticed it on slurm earlier today. I'll check
> the others as far as I can.
>
> Craig: what environment are you using? ssh?
> Ralph
>
>

We are using ssh (we do not use tight integration in SGE).

Craig

>
> On Oct 14, 2008, at 1:18 PM, George Bosilca wrote:
>
>> I use modules too, but they only work locally. Or is there a feature
>> in "module" to automatically load the list of currently loaded local
>> modules remotely ?
>>
>> george.
>>
>> On Oct 14, 2008, at 3:03 PM, Ralph Castain wrote:
>>
>>> You might consider using something like "module" - we use that
>>> system for exactly this reason. Works quite well and solves the
>>> multiple compiler issue.
>>>
>>> Ralph
>>>
>>> On Oct 14, 2008, at 12:56 PM, Craig Tierney wrote:
>>>
>>>> George Bosilca wrote:
>>>>> The option to expand the remote LD_LIBRARY_PATH, in such a way that
>>>>> Open MPI related applications have their dependencies satisfied, is
>>>>> in the trunk. The fact that the compiler requires some
>>>>> LD_LIBRARY_PATH is out of the scope of an MPI implementation, and I
>>>>> don't think we should take care of it.
>>>>> Passing the local LD_LIBRARY_PATH to the remote nodes doesn't make
>>>>> much sense. There are plenty of environment, where the head node
>>>>> have a different configuration than the compute nodes. Again, in
>>>>> this case my original solution seems not that bad. If you copy (or
>>>>> make a link if you prefer) in the Open MPI lib directory to the
>>>>> compiler shared libraries, this will work.
>>>>> george.
>>>>
>>>> This does work. It just increases maintenance for each new version
>>>> of OpenMPI. How often does a head node have a different configuration
>>>> than the compute node? It would see that this would even more
>>>> support the
>>>> passing of LD_LIBRARY_PATH for OpenMPI tools to support a heterogeneous
>>>> configuration as you described.
>>>>
>>>>
>>>> Thanks,
>>>> Craig
>>>>
>>>>
>>>>
>>>>
>>>>> On Oct 14, 2008, at 12:11 PM, Craig Tierney wrote:
>>>>>> George Bosilca wrote:
>>>>>>> Craig,
>>>>>>> This is a problem with the Intel libraries and not the Open MPI
>>>>>>> ones. You have to somehow make these libraries available on the
>>>>>>> compute nodes.
>>>>>>> What I usually do (but it's not the best way to solve this
>>>>>>> problem) is to copy these libraries somewhere on my home area and
>>>>>>> to add the directory to my LD_LIBRARY_PATH.
>>>>>>> george.
>>>>>>
>>>>>> This is ok when you only ever use one compiler, but it isn't very
>>>>>> flexible.
>>>>>> I want to keep it as simple as possible for my users, while having
>>>>>> a maintainable
>>>>>> system.
>>>>>>
>>>>>> The libraries are on the compute nodes, the problem deals with
>>>>>> supporting
>>>>>> multiple versions of compilers. I can't just list all of the lib
>>>>>> paths
>>>>>> in ld.so.conf, because then the user will never get the correct
>>>>>> one. I can't
>>>>>> specify a static LD_LIBRARY_PATH for the same reason. I would
>>>>>> prefer not
>>>>>> to build my system libraries static.
>>>>>>
>>>>>> To the OpenMPI developers, what is your opinion on changing
>>>>>> orterun/mpirun
>>>>>> to pass LD_LIBRARY_PATH to the remote hosts when starting OpenMPI
>>>>>> processes?
>>>>>> By hand, all that would be done is:
>>>>>>
>>>>>> env LD_LIBRARY_PATH=$LD_LIBRARY_PATH $OPMIPATH/orted <args>
>>>>>>
>>>>>> This would ensure that orted is launched correctly.
>>>>>>
>>>>>> Or is it better to just build the OpenMPI tools statically? We also
>>>>>> use other compilers (PGI, Lahey) so I need a solution that works for
>>>>>> all of them.
>>>>>>
>>>>>> Thanks,
>>>>>> Craig
>>>>>>
>>>>>>
>>>>>>
>>>>>>> On Oct 10, 2008, at 6:17 PM, Craig Tierney wrote:
>>>>>>>> I am having problems launching openmpi jobs on my system. I
>>>>>>>> support multiple versions
>>>>>>>> of MPI and compilers using GNU Modules. For the default
>>>>>>>> compiler, everything is fine.
>>>>>>>> For non-default, I am having problems.
>>>>>>>>
>>>>>>>> I built Openmpi-1.2.6 (and 1.2.7) with the following configure
>>>>>>>> options:
>>>>>>>>
>>>>>>>> # module load intel/10.1
>>>>>>>> # ./configure CC=icc CXX=icpc F77=ifort FC=ifort F90=ifort
>>>>>>>> --prefix=/opt/openmpi/1.2.7-intel-10.1 --without-
>>>>>>>> gridengine --enable-io-romio
>>>>>>>> --with-io-romio-flags=--with-file-sys=nfs+ufs
>>>>>>>> --with-openib=/opt/hjet/ofed/1.3.1
>>>>>>>>
>>>>>>>> When I launch a job, I run the module command for the right
>>>>>>>> compiler/MPI version to set the paths
>>>>>>>> correctly. Mpirun passes LD_LIBRARY_PATH to the executable I am
>>>>>>>> launching, but not orted.
>>>>>>>>
>>>>>>>> When orted is launched on the remote system, the LD_LIBRARY_PATH
>>>>>>>> doesn't come with, and the Intel 10.1 libraries can't be found.
>>>>>>>>
>>>>>>>> /opt/openmpi/1.2.7-intel-10.1/bin/orted: error while loading
>>>>>>>> shared libraries: libintlc.so.5: cannot open shared object file:
>>>>>>>> No such file or directory
>>>>>>>>
>>>>>>>> How do others solve this problem?
>>>>>>>>
>>>>>>>> Thanks,
>>>>>>>> Craig
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Craig Tierney (craig.tierney_at_[hidden])
>>>>>>>> _______________________________________________
>>>>>>>> users mailing list
>>>>>>>> users_at_[hidden]
>>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>> _______________________________________________
>>>>>>> users mailing list
>>>>>>> users_at_[hidden]
>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Craig Tierney (craig.tierney_at_[hidden])
>>>>>> _______________________________________________
>>>>>> users mailing list
>>>>>> users_at_[hidden]
>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>> _______________________________________________
>>>>> users mailing list
>>>>> users_at_[hidden]
>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>
>>>>
>>>> --
>>>> Craig Tierney (craig.tierney_at_[hidden])
>>>> _______________________________________________
>>>> users mailing list
>>>> users_at_[hidden]
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>

-- 
Craig Tierney (craig.tierney_at_[hidden])