Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] running openmpi with specified lib path
From: Duke Nguyen (duke.lists_at_[hidden])
Date: 2013-05-07 10:39:46


On 5/7/13 9:25 PM, Ralph Castain wrote:
> Look at your ld-library-path. Breaking it down, you have the following elements
>
> /opt/apps/abinit/abinit-7.2.1_composer_xe_2013.3.163_openmpi-1.7.2_intel_fftw3-mkl/usr/local/lib

$ export LD_LIBRARY_PATH=
$ module load mpi/openmpi-1.7.2_composer_xe_2013.3.163
$ module load
apps/abinit-7.2.1_composer_xe_2013.3.163_openmpi-1.7.2_intel_fftw3-mkl
$ echo $LD_LIBRARY_PATH
/opt/apps/abinit/abinit-7.2.1_composer_xe_2013.3.163_openmpi-1.7.2_intel_fftw3-mkl/usr/local/lib:/opt/apps/openmpi/openmpi-1.7.2r28341_composer_xe_2013.3.163/usr/local/lib/openmpi:/opt/apps/openmpi/openmpi-1.7.2r28341_composer_xe_2013.3.163/usr/local/lib

The LD_LIBRARY_PATH above contains three different locations:

/opt/apps/abinit/abinit-7.2.1_composer_xe_2013.3.163_openmpi-1.7.2_intel_fftw3-mkl/usr/local/lib
/opt/apps/openmpi/openmpi-1.7.2r28341_composer_xe_2013.3.163/usr/local/lib/openmpi
/opt/apps/openmpi/openmpi-1.7.2r28341_composer_xe_2013.3.163/usr/local/lib

>
> repeated three times. Did you actually configure with
>
> --prefix=/opt/apps/openmpi/openmpi-1.7.2r28341_composer_xe_2013.3.163/usr/local

No I didnt (and that maybe the reason, I am not really sure I was
correct when installation these things). What I did was:

$ cd /opt/apps/openmpi/openmpi-1.7.2r28341_composer_xe_2013.3.163
$ ./configure CC=/opt/intel/composer_xe_2013.3.163/bin/intel64/icc
CXX=/opt/intel/composer_xe_2013.3.163/bin/intel64/icpc
F77=/opt/intel/composer_xe_2013.3.163/bin/intel64/ifort
FC=/opt/intel/composer_xe_2013.3.163/bin/intel64/ifort
$ make -j8
$ DESTDIR=`pwd` make -j8 install

so all the compiled openmpi components were installed at

$ ls /opt/apps/openmpi/openmpi-1.7.2r28341_composer_xe_2013.3.163/usr/local/
bin etc include lib share

Am I doing wrong??? Please advise.

>
> Note that the prefix path *must* be the first element in your ld-libary-path or you'll pickup one of your other installations. When you did your test, you goofed:
>
>> $ export
>> LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/opt/intel/composer_xe_2013/lib/intel64
>
>
> This put the new path at the end, where it would never be used

What I thought was if the needed lib or exec is not found on the first
path, then the next will be used. Since libimf was not found from all
previous path, added the last one helped. Anyway, try adding to the
beginning:

$ mpirun ./mpihello_intel
[node0111.biobos.grih.org:19243] mca: base: component_find: unable to
open /usr/local/lib/openmpi/mca_ess_hnp:
/usr/local/lib/openmpi/mca_ess_hnp.so: undefined symbol:
orte_local_jobdata (ignored)
[node0111.biobos.grih.org:19243] mca: base: component_find: unable to
open /usr/local/lib/openmpi/mca_ess_slurm:
/usr/local/lib/openmpi/mca_ess_slurm.so: undefined symbol:
orte_orted_exit_with_barrier (ignored)
[node0111.biobos.grih.org:19243] mca: base: component_find: unable to
open /usr/local/lib/openmpi/mca_ess_slurmd:
/usr/local/lib/openmpi/mca_ess_slurmd.so: undefined symbol:
orte_pmap_t_class (ignored)
mpirun: symbol lookup error:
/usr/local/lib/openmpi/mca_ess_singleton.so: undefined symbol:
orte_util_setup_local_nidmap_entries

gave the same error.

>
>
> On May 7, 2013, at 7:11 AM, Duke Nguyen <duke.lists_at_[hidden]> wrote:
>
>> On 5/7/13 8:10 PM, Jeff Squyres (jsquyres) wrote:
>>> On May 7, 2013, at 7:36 AM, Duke Nguyen <duke.lists_at_[hidden]> wrote:
>>>
>>>> So apparently openmpi 1.7.2 looks for the old library at
>>>> /usr/loca/lib/openmpi for 1.6.3 instead of at
>>>> /opt/apps/openmpi/openmpi-1.7.2r28341_composer_xe_2013.3.163/usr/local/lib/openmpi.
>>>> Is there away to force openmpi 1.7.2 look at
>>>> /opt/apps/openmpi/openmpi-1.7.2r28341_composer_xe_2013.3.163/usr/local/lib/openmpi
>>>> first before looking at other locations?
>>>
>>>
>>> This is *usually* a problem of local settings. Make sure you don't have both the 1.6 and 1.7 library directories listed in LD_LIBRARY_PATH, for example.
>>>
>>
>> I also thought of that, but removing /usr/local does not help.
>>
>> $ echo $PATH
>> /usr/lib64/qt-3.3/bin:/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/sbin:/opt/apps/abinit/bin:/opt/torque/sbin:/opt/torque/bin:/usr/local/bin:/opt/apps/bin
>> $ export PATH=/usr/lib64/qt-3.3/bin:/bin:/usr/bin:/usr/sbin:/sbin
>> $ echo $LD_LIBRARY_PATH
>> :/usr/local/lib:/opt/intel/lib/intel64:/opt/intel/mkl/lib/intel64
>> $ export LD_LIBRARY_PATH=
>> $ module load mpi/openmpi-1.7.2_composer_xe_2013.3.163
>> $ module load
>> apps/abinit-7.2.1_composer_xe_2013.3.163_openmpi-1.7.2_intel_fftw3-mkl
>> $ mpirun ./mpihello_intel
>> mpirun: error while loading shared libraries: libimf.so: cannot open
>> shared object file: No such file or directory
>> $ echo $LD_LIBRARY_PATH
>> /opt/apps/abinit/abinit-7.2.1_composer_xe_2013.3.163_openmpi-1.7.2_intel_fftw3-mkl/usr/local/lib:/opt/apps/openmpi/openmpi-1.7.2r28341_composer_xe_2013.3.163/usr/local/lib/openmpi:/opt/apps/openmpi/openmpi-1.7.2r28341_composer_xe_2013.3.163/usr/local/lib
>> $ export
>> LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:/opt/intel/composer_xe_2013/lib/intel64
>> $ mpirun ./mpihello_intel
>> [node0111.biobos.grih.org:18979] mca: base: component_find: unable to
>> open /usr/local/lib/openmpi/mca_ess_hnp:
>> /usr/local/lib/openmpi/mca_ess_hnp.so: undefined symbol:
>> orte_local_jobdata (ignored)
>> [node0111.biobos.grih.org:18979] mca: base: component_find: unable to
>> open /usr/local/lib/openmpi/mca_ess_slurm:
>> /usr/local/lib/openmpi/mca_ess_slurm.so: undefined symbol:
>> orte_orted_exit_with_barrier (ignored)
>> [node0111.biobos.grih.org:18979] mca: base: component_find: unable to
>> open /usr/local/lib/openmpi/mca_ess_slurmd:
>> /usr/local/lib/openmpi/mca_ess_slurmd.so: undefined symbol:
>> orte_pmap_t_class (ignored)
>> mpirun: symbol lookup error:
>> /usr/local/lib/openmpi/mca_ess_singleton.so: undefined symbol:
>> orte_util_setup_local_nidmap_entries
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>