Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Passing LD_LIBRARY_PATH to orted
From: Ralph Castain (rhc_at_[hidden])
Date: 2008-10-14 15:33:55


I -think- there is...at least here, it does seem to behave that way on
our systems. Not sure if there is something done locally to make it
work.

Also, though, I have noted that LD_LIBRARY_PATH does seem to be
getting forwarded on the 1.3 branch in some environments. OMPI isn't
doing it directly to the best of my knowledge, but I think the base
environment might be. Specifically, I noticed it on slurm earlier
today. I'll check the others as far as I can.

Craig: what environment are you using? ssh?
Ralph

On Oct 14, 2008, at 1:18 PM, George Bosilca wrote:

> I use modules too, but they only work locally. Or is there a feature
> in "module" to automatically load the list of currently loaded local
> modules remotely ?
>
> george.
>
> On Oct 14, 2008, at 3:03 PM, Ralph Castain wrote:
>
>> You might consider using something like "module" - we use that
>> system for exactly this reason. Works quite well and solves the
>> multiple compiler issue.
>>
>> Ralph
>>
>> On Oct 14, 2008, at 12:56 PM, Craig Tierney wrote:
>>
>>> George Bosilca wrote:
>>>> The option to expand the remote LD_LIBRARY_PATH, in such a way
>>>> that Open MPI related applications have their dependencies
>>>> satisfied, is in the trunk. The fact that the compiler requires
>>>> some LD_LIBRARY_PATH is out of the scope of an MPI
>>>> implementation, and I don't think we should take care of it.
>>>> Passing the local LD_LIBRARY_PATH to the remote nodes doesn't
>>>> make much sense. There are plenty of environment, where the head
>>>> node have a different configuration than the compute nodes.
>>>> Again, in this case my original solution seems not that bad. If
>>>> you copy (or make a link if you prefer) in the Open MPI lib
>>>> directory to the compiler shared libraries, this will work.
>>>> george.
>>>
>>> This does work. It just increases maintenance for each new version
>>> of OpenMPI. How often does a head node have a different
>>> configuration
>>> than the compute node? It would see that this would even more
>>> support the
>>> passing of LD_LIBRARY_PATH for OpenMPI tools to support a
>>> heterogeneous
>>> configuration as you described.
>>>
>>>
>>> Thanks,
>>> Craig
>>>
>>>
>>>
>>>
>>>> On Oct 14, 2008, at 12:11 PM, Craig Tierney wrote:
>>>>> George Bosilca wrote:
>>>>>> Craig,
>>>>>> This is a problem with the Intel libraries and not the Open MPI
>>>>>> ones. You have to somehow make these libraries available on the
>>>>>> compute nodes.
>>>>>> What I usually do (but it's not the best way to solve this
>>>>>> problem) is to copy these libraries somewhere on my home area
>>>>>> and to add the directory to my LD_LIBRARY_PATH.
>>>>>> george.
>>>>>
>>>>> This is ok when you only ever use one compiler, but it isn't
>>>>> very flexible.
>>>>> I want to keep it as simple as possible for my users, while
>>>>> having a maintainable
>>>>> system.
>>>>>
>>>>> The libraries are on the compute nodes, the problem deals with
>>>>> supporting
>>>>> multiple versions of compilers. I can't just list all of the
>>>>> lib paths
>>>>> in ld.so.conf, because then the user will never get the correct
>>>>> one. I can't
>>>>> specify a static LD_LIBRARY_PATH for the same reason. I would
>>>>> prefer not
>>>>> to build my system libraries static.
>>>>>
>>>>> To the OpenMPI developers, what is your opinion on changing
>>>>> orterun/mpirun
>>>>> to pass LD_LIBRARY_PATH to the remote hosts when starting
>>>>> OpenMPI processes?
>>>>> By hand, all that would be done is:
>>>>>
>>>>> env LD_LIBRARY_PATH=$LD_LIBRARY_PATH $OPMIPATH/orted <args>
>>>>>
>>>>> This would ensure that orted is launched correctly.
>>>>>
>>>>> Or is it better to just build the OpenMPI tools statically? We
>>>>> also
>>>>> use other compilers (PGI, Lahey) so I need a solution that works
>>>>> for
>>>>> all of them.
>>>>>
>>>>> Thanks,
>>>>> Craig
>>>>>
>>>>>
>>>>>
>>>>>> On Oct 10, 2008, at 6:17 PM, Craig Tierney wrote:
>>>>>>> I am having problems launching openmpi jobs on my system. I
>>>>>>> support multiple versions
>>>>>>> of MPI and compilers using GNU Modules. For the default
>>>>>>> compiler, everything is fine.
>>>>>>> For non-default, I am having problems.
>>>>>>>
>>>>>>> I built Openmpi-1.2.6 (and 1.2.7) with the following configure
>>>>>>> options:
>>>>>>>
>>>>>>> # module load intel/10.1
>>>>>>> # ./configure CC=icc CXX=icpc F77=ifort FC=ifort F90=ifort --
>>>>>>> prefix=/opt/openmpi/1.2.7-intel-10.1 --without-
>>>>>>> gridengine --enable-io-romio --with-io-romio-flags=--with-file-
>>>>>>> sys=nfs+ufs --with-openib=/opt/hjet/ofed/1.3.1
>>>>>>>
>>>>>>> When I launch a job, I run the module command for the right
>>>>>>> compiler/MPI version to set the paths
>>>>>>> correctly. Mpirun passes LD_LIBRARY_PATH to the executable I
>>>>>>> am launching, but not orted.
>>>>>>>
>>>>>>> When orted is launched on the remote system, the LD_LIBRARY_PATH
>>>>>>> doesn't come with, and the Intel 10.1 libraries can't be found.
>>>>>>>
>>>>>>> /opt/openmpi/1.2.7-intel-10.1/bin/orted: error while loading
>>>>>>> shared libraries: libintlc.so.5: cannot open shared object
>>>>>>> file: No such file or directory
>>>>>>>
>>>>>>> How do others solve this problem?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Craig
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Craig Tierney (craig.tierney_at_[hidden])
>>>>>>> _______________________________________________
>>>>>>> users mailing list
>>>>>>> users_at_[hidden]
>>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>> _______________________________________________
>>>>>> users mailing list
>>>>>> users_at_[hidden]
>>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>>>
>>>>>
>>>>> --
>>>>> Craig Tierney (craig.tierney_at_[hidden])
>>>>> _______________________________________________
>>>>> users mailing list
>>>>> users_at_[hidden]
>>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>> _______________________________________________
>>>> users mailing list
>>>> users_at_[hidden]
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>
>>>
>>> --
>>> Craig Tierney (craig.tierney_at_[hidden])
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users