Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI users] OpenMPI 1.6.4 and Intel Composer_xe_2013.4.183: problem with remote runs, orted: error while loading shared libraries: libimf.so
From: Gus Correa (gus_at_[hidden])
Date: 2013-06-21 11:54:42


Hi Stefano

Make sure your Intel compiler's shared libraries
are accessible on all nodes.

Is your /home directory shared across all nodes?
How about /opt (if Intel is installed there)?

By default Intel installs the compilers on /opt, which in typical
clusters (and Linux distributions) is a local directory (to each node),
not shared via NFS.
Although you seem to have installed it somewhere else,
/home/stefano/opt maybe, if /home/stefano/opt
is just a soft link to /opt, not a real directory,
that may not to do the trick across the cluster network.

This error:

>> /home/stefano/opt/mpi/openmpi/1.6.4/intel/bin/orted: error
>> while loading shared libraries: libimf.so: cannot open shared
>> object file: No such file or directory

suggests something like that is going on (libimf.so is an
*Intel shared library*, it is *not an Open MPI libary*).

To have all needed tools (OpenMPI and Intel)
available on all nodes, there are two typical solutions
(by the way, see this FAQ:
http://www.open-mpi.org/faq/?category=building#where-to-install):

1) Install them on all nodes, via RPM, or configure/make/install, or
other mechanism.
This is time consuming and costly to maintain, but scales well
in big or small clusters.

2) Install them on your master/head/adminsitration/storage node,
and and share them via network (typicaly via NFS export/mount).
This is easy to maintain, and scales well in small/medium clusters,
but not so much on big ones.

Make sure the Intel and MPI directories are either shared by
or present/installed on all nodes.

I also wonder if you really need these many environment variables:

>> LD_LIBRARY_PATH=${MPI}/lib/openmpi:${MPI}/lib:$LD_LIBRARY_PATH
>> export LD_RUN_PATH=${MPI}/lib/openmpi:${MPI}/lib:$LD_RUN_PATH

or if that may be actually replaced by the simpler form:

>> LD_LIBRARY_PATH=${MPI}/lib:$LD_LIBRARY_PATH

I hope it helps,
Gus Correa

On 06/21/2013 04:35 AM, Stefano Zaghi wrote:
> Wow... I think you are right... I will am check after the job I have
> just started will finish.
>
> Thank you again.
>
> See you soon
>
> Stefano Zaghi
> Ph.D. Aerospace Engineer,
> Research Scientist, Dept. of Computational Hydrodynamics at *CNR-INSEAN*
> <http://www.insean.cnr.it/en/content/cnr-insean>
> The Italian Ship Model Basin
> (+39) 06.50299297 (Office)
> My codes:
> *OFF* <https://github.com/szaghi/OFF>, Open source Finite volumes Fluid
> dynamics code
> *Lib_VTK_IO* <https://github.com/szaghi/Lib_VTK_IO>, a Fortran library
> to write and read data conforming the VTK standard
> *IR_Precision* <https://github.com/szaghi/IR_Precision>, a Fortran
> (standard 2003) module to develop portable codes
>
>
> 2013/6/21 <thomas.forde_at_[hidden] <mailto:thomas.forde_at_[hidden]>>
>
> hi Stefano
>
> /home/stefano/opt/intel/2013.4.183/lib/intel64/ is also the wrong
> path, as the file is in ..183/lib/ and not ...183/lib/intel64/
>
> is that why?
> ./Thomas
>
>
> Den 21. juni 2013 kl. 10:26 skrev "Stefano Zaghi"
> <stefano.zaghi_at_[hidden] <mailto:stefano.zaghi_at_[hidden]>>:
>
>> Dear Thomas,
>> thank you again.
>>
>> Symlink in /usr/lib64 is not enough, I have symlinked also
>> in /home/stefano/opt/mpi/openmpi/1.6.4/intel/lib and, as expected,
>> not only libimf.so but also ibirng.so and libintlc.so.5 are necessary.
>>
>> Now also remote runs works, but this is only a workaround, I still
>> not understand why mpirun do not find intel library even if
>> LD_LIBRARY_PATH contains also
>> /home/stefano/opt/intel/2013.4.183/lib/intel64. Can you try
>> explain again?
>>
>> Thank you very much.
>>
>> Stefano Zaghi
>> Ph.D. Aerospace Engineer,
>> Research Scientist, Dept. of Computational Hydrodynamics at
>> *CNR-INSEAN* <http://www.insean.cnr.it/en/content/cnr-insean>
>> The Italian Ship Model Basin
>> (+39) 06.50299297 (Office)
>> My codes:
>> *OFF* <https://github.com/szaghi/OFF>, Open source Finite volumes
>> Fluid dynamics code
>> *Lib_VTK_IO* <https://github.com/szaghi/Lib_VTK_IO>, a would
>> Fortran library to write and read data conforming the VTK standard
>> *IR_Precision* <https://github.com/szaghi/IR_Precision>, a Fortran
>> (standard 2003) module to develop portable codes
>>
>>
>> 2013/6/21 <thomas.forde_at_[hidden] <mailto:thomas.forde_at_[hidden]>>
>>
>> your settings are as following:
>> export MPI=/home/stefano/opt/mpi/openmpi/1.6.4/intel
>> export PATH=${MPI}/bin:$PATH
>> export
>> LD_LIBRARY_PATH=${MPI}/lib/openmpi:${MPI}/lib:$LD_LIBRARY_PATH
>> export LD_RUN_PATH=${MPI}/lib/openmpi:${MPI}/lib:$LD_RUN_PATH
>>
>> and your path to libimf.so file is
>> /home/stefano/opt/intel/2013.4.183/lib/libimf.so
>>
>> your export LD_LIbrary_PATH if i can decude it right would be
>> because you use the $MPI first.
>>
>> /home/stefano/opt/mpi/openmpi/1.64./intel/lib/openmpi and
>> /home/stefano/opt/mpi/openmpi/1.64./intel/lib
>>
>> as you can see it doesnt look for the files int he right place.
>>
>> the simplest thing i would try is to symlink the libimf.so
>> file to /usr/lib64 and should give you a workaround.
>>
>>
>>
>>
>>
>>
>> From: Stefano Zaghi <stefano.zaghi_at_[hidden]
>> <mailto:stefano.zaghi_at_[hidden]>>
>> To: Open MPI Users <users_at_[hidden]
>> <mailto:users_at_[hidden]>>,
>> Date: 21.06.2013 09:45
>> Subject: Re: [OMPI users] OpenMPI 1.6.4 and Intel
>> Composer_xe_2013.4.183: problem with remote runs, orted: error
>> while loading shared libraries: libimf.so
>> Sent by: users-bounces_at_[hidden]
>> <mailto:users-bounces_at_[hidden]>
>> ------------------------------------------------------------------------
>>
>>
>>
>> Dear Thomas,
>>
>> thank you very much for your very fast replay.
>>
>> Yes I have that library in the correct place:
>>
>> -rwxr-xr-x 1 stefano users 3.0M May 20 14:22
>> opt/intel/2013.4.183/lib/intel64/libimf.so
>>
>> Stefano Zaghi
>> Ph.D. Aerospace Engineer,
>> Research Scientist, Dept. of Computational Hydrodynamics at
>> *_CNR-INSEAN_* <http://www.insean.cnr.it/en/content/cnr-insean>
>> The Italian Ship Model Basin
>> (+39) 06.50299297 (Office)
>> My codes:
>> *_OFF_* <https://github.com/szaghi/OFF>, Open source Finite
>> volumes Fluid dynamics code
>> *_Lib_VTK_IO_* <https://github.com/szaghi/Lib_VTK_IO>, a
>> Fortran library to write and read data conforming the VTK standard
>> *_IR_Precision_* <https://github.com/szaghi/IR_Precision>, a
>> Fortran (standard 2003) module to develop portable codes
>>
>>
>> 2013/6/21 <_thomas.forde_at_[hidden]_
>> <mailto:thomas.forde_at_[hidden]>>
>> hi Stefano
>>
>> your error message show that you are missing a shared library,
>> not necessary that library path is wrong.
>>
>> do you actually have libimf.so, can you find the file on your
>> system.
>>
>> ./Thomas
>>
>>
>>
>>
>> From: Stefano Zaghi <_stefano.zaghi_at_[hidden]_
>> <mailto:stefano.zaghi_at_[hidden]>>
>> To: _users_at_[hidden]_ <mailto:users_at_[hidden]>,
>> Date: 21.06.2013 09:27
>> Subject: [OMPI users] OpenMPI 1.6.4 and Intel
>> Composer_xe_2013.4.183: problem with remote runs, orted: error
>> while loading shared libraries: libimf.so
>> Sent by: _users-bounces_at_[hidden]_
>> <mailto:users-bounces_at_[hidden]>
>> ------------------------------------------------------------------------
>>
>>
>>
>>
>> Dear All,
>> I have compiled OpenMPI 1.6.4 with Intel Composer_xe_2013.4.183.
>>
>> My configure is:
>>
>> ./configure --prefix=/home/stefano/opt/mpi/openmpi/1.6.4/intel
>> CC=icc CXX=icpc F77=ifort FC=ifort
>>
>> Intel Composer has been installed in:
>>
>> /home/stefano/opt/intel/2013.4.183/composer_xe_2013.4.183
>>
>> Into the .bashrc and .profile in all nodes there is:
>>
>> source /home/stefano/opt/intel/2013.4.183/bin/compilervars.sh
>> intel64
>> export MPI=/home/stefano/opt/mpi/openmpi/1.6.4/intel
>> export PATH=${MPI}/bin:$PATH
>> export
>> LD_LIBRARY_PATH=${MPI}/lib/openmpi:${MPI}/lib:$LD_LIBRARY_PATH
>> export LD_RUN_PATH=${MPI}/lib/openmpi:${MPI}/lib:$LD_RUN_PATH
>>
>> If I run parallel job into each single node (e.g. mpirun -np 8
>> myprog) all works well. However, when I tried to run parallel
>> job in more nodes of the cluster (remote runs) like the
>> following:
>>
>> mpirun -np 16 --bynode --machinefile nodi.txt -x
>> LD_LIBRARY_PATH -x LD_RUN_PATH myprog
>>
>> I got the following error:
>>
>> /home/stefano/opt/mpi/openmpi/1.6.4/intel/bin/orted: error
>> while loading shared libraries: libimf.so: cannot open shared
>> object file: No such file or directory
>>
>> I have read many FAQs and online resources, all indicating
>> LD_LIBRARY_PATH as the possible problem (wrong setting).
>> However I am not able to figure out what is going wrong, the
>> LD_LIBRARY_PATH seems to set right in all nodes.
>>
>> It is worth noting that in the same cluster I have successful
>> installed OpenMPI 1.4.3 with Intel Composer_xe_2011_sp1.6.233
>> following exactly the same procedure.
>>
>> Thank you in advance for all suggestion,
>> sincerely
>>
>> Stefano Zaghi
>> Ph.D. Aerospace Engineer,
>> Research Scientist, Dept. of Computational Hydrodynamics at
>> *_CNR-INSEAN_* <http://www.insean.cnr.it/en/content/cnr-insean>
>> The Italian Ship Model Basin
>> (+39) 06.50299297 (Office)
>> My codes: _
>> _*_OFF_* <https://github.com/szaghi/OFF>, Open source Finite
>> volumes Fluid dynamics code _
>> _*_Lib_VTK_IO_* <https://github.com/szaghi/Lib_VTK_IO>, a
>> Fortran library to write and read data conforming the VTK
>> standard
>> *_IR_Precision_* <https://github.com/szaghi/IR_Precision>, a
>> Fortran (standard 2003) module to develop portable
>> codes_______________________________________________
>> users mailing list_
>> __users_at_[hidden]_ <mailto:users_at_[hidden]>_
>> __http://www.open-mpi.org/mailman/listinfo.cgi/users_
>>
>>
>>
>>
>>
>>
>>
>> Denne e-posten kan innehalde informasjon som er konfidensiell
>> og/eller underlagt lovbestemt teieplikt. Kun den tiltenkte
>> adressat har adgang
>> til å lese eller vidareformidle denne e-posten eller
>> tilhøyrande vedlegg. Dersom De ikkje er den tiltenkte
>> mottakar, vennligst kontakt avsendar pr e-post, slett denne
>> e-posten med vedlegg og makuler samtlige utskrifter og kopiar
>> av den.
>>
>> This e-mail may contain confidential information, or otherwise
>> be protected against unauthorised use. Any disclosure,
>> distribution or other use of the information by anyone but the
>> intended recipient is strictly prohibited.
>> If you have received this e-mail in error, please advise the
>> sender by immediate reply and destroy the received documents
>> and any copies hereof.
>>
>>
>> PBefore
>> printing, think about the environment
>>
>>
>>
>> _______________________________________________
>> users mailing list_
>> __users_at_[hidden]_ <mailto:users_at_[hidden]>_
>> __http://www.open-mpi.org/mailman/listinfo.cgi/users_
>> _______________________________________________
>> users mailing list
>> users_at_[hidden] <mailto:users_at_[hidden]>
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>>
>>
>>
>>
>>
>>
>>
>> Denne e-posten kan innehalde informasjon som er konfidensiell
>> og/eller underlagt lovbestemt teieplikt. Kun den tiltenkte
>> adressat har adgang
>> til å lese eller vidareformidle denne e-posten eller
>> tilhøyrande vedlegg. Dersom De ikkje er den tiltenkte
>> mottakar, vennligst kontakt avsendar pr e-post, slett denne
>> e-posten med vedlegg og makuler samtlige utskrifter og kopiar
>> av den.
>>
>> This e-mail may contain confidential information, or otherwise
>> be protected against unauthorised use. Any disclosure,
>> distribution or other use of the information by anyone but the
>> intended recipient is strictly prohibited.
>> If you have received this e-mail in error, please advise the
>> sender by immediate reply and destroy the received documents
>> and any copies hereof.
>>
>>
>> PBefore
>> printing, think about the environment
>>
>>
>>
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden] <mailto:users_at_[hidden]>
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden] <mailto:users_at_[hidden]>http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> Denne e-posten kan innehalde informasjon som er konfidensiell
> og/eller underlagt lovbestemt teieplikt. Kun den tiltenkte adressat
> har adgang til å lese eller vidareformidle denne e-posten eller
> tilhøyrande vedlegg. Dersom De ikkje er den tiltenkte mottakar,
> vennligst kontakt avsendar pr e-post, slett denne e-posten med
> vedlegg og makuler samtlige utskrifter og kopiar av den.
>
> This e-mail may contain confidential information, or otherwise be
> protected against unauthorised use. Any disclosure, distribution or
> other use of the information by anyone but the intended recipient is
> strictly prohibited. If you have received this e-mail in error,
> please advise the sender by immediate reply and destroy the received
> documents and any copies hereof.
>
> PBefore printing, think about the environment
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden] <mailto:users_at_[hidden]>
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users