Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] OpenMPI 1.7.2-2.1.3 does not work with OpenFOAM 2.2.2 on OpenSUSE 13.1
From: Ralph Castain (rhc_at_[hidden])
Date: 2014-01-24 16:21:33


It is generally a bad idea to install OMPI in system directories like /usr/local as multiple versions can wind up intermingled with each other, thus causing these kind of problems. I would suggest taking a 1.6.5 tarball and building it with a prefix in your own home directory area. You should then put that location at the *front* of your PATH and LD_LIBRARY_PATH to ensure you get that version instead of something else.

Next, you need to ensure that any launch to a remote node gets pointed to the same location - easiest way to do that is configure OMPI --enable-orterun-prefix-by-default.

That should solve the problem.

On Jan 24, 2014, at 8:40 AM, Elisabeth Beer <elisabethbeer_at_[hidden]> wrote:

> Hi,
>
> I've done an operating system up-grade to OpenSUSE 13.1 and I've up-graded OpenFOAM from 2.2.1 to 2.2.2.
>
> Bevor, OpenMPI worked well.
> Now, it does not work at all.
>
> First Step
> ------------------
> After decomposing the domain, I've tried to start parallel computation:
> mpirun -np 8 simpleFoam -parallel
>
> I've received that error message:
> It looks like opal_init failed for some reason; your parallel process is
> likely to abort. There are many reasons that a parallel process can
> fail during opal_init; some of which are due to configuration or
> environment problems. This failure appears to be an internal failure;
> here's some additional information (which may only be relevant to an
> Open MPI developer):
>
> opal_shmem_base_select failed
> --> Returned value -1 instead of OPAL_SUCCESS
> --------------------------------------------------------------------------
> --------------------------------------------------------------------------
> It looks like orte_init failed for some reason; your parallel process is
> likely to abort. There are many reasons that a parallel process can
> fail during orte_init; some of which are due to configuration or
> environment problems. This failure appears to be an internal failure;
> here's some additional information (which may only be relevant to an
> Open MPI developer):
>
> opal_init failed
> --> Returned value Error (-1) instead of ORTE_SUCCESS
>
> I've checked the openmpi version (just to be sure) that was responsible for that error:
> mpirun --version
> mpirun (Open MPI) 1.7.2
>
> bashrc file of OpenFoam allows to switch between the current systems version an other installed versions.
> This version of openmpi (1.7.2) obviously came with OpenSUSE 12.3.
> I've received that error with the current systems version 1.7.2
> OpenFoam /etc/bashrc/
> export WM_MPLIB=SYSTEMOPENMPI
>
> Second Step
> -----------------------
> I've changed OpenFoams bashrc-file
> export WM_MPLIB=OPENMPI
>
> Then I've downloaded ThirdParty2.2.2 package, including openmpi 1.6.3, to see if I could maybe switch and use that (previous) version.
>
> I've received that error message:
> mpirun: error while loading shared libraries: libopen-rte.so.4: cannot open shared object file: No such file or directory
>
> I've add another path to LD_LIBRARY_PATH in my home .bashrc including the new path where the missing file is located:
> export LD_LIBRARY_PATH=/.../..../usr/local/lib64
>
> Now I get another error:
>
> Sorry! You were supposed to get help about:
> opal_init:startup:internal-failure
> But I couldn't open the help file:
> /opt/ThirdParty-2.2.2/platforms/linux64Gcc/openmpi-1.6.3/share/openmpi/help-opal-runtime.txt: No such file or directory. Sorry!
> --------------------------------------------------------------------------
> [linux-0ze2:01786] [[INVALID],INVALID] ORTE_ERROR_LOG: Error in file runtime/orte_init.c at line 79
> [linux-0ze2:01786] [[INVALID],INVALID] ORTE_ERROR_LOG: Error in file orterun.c at line 694
>
> Third Step
> -----------------
> I've openmpi 1.6.3 uninstalled with 'make uninstall' and finally downloaded and unzipped openmpi-1.4.5 in /opt/
>
> shell$ cd openmpi-1.4.5
> shell$ ./configure --prefix=/usr/local <...lots of output...>
> shell$ make all install
>
> I've received following error message:
> [linux-0ze2:03493] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file runtime/orte_init.c at line 125
> --------------------------------------------------------------------------
> Sorry! You were supposed to get help about:
> orte_init:startup:internal-failure
> But I couldn't open the help file:
> /usr/local/lib64/openmpi/share/openmpi/help-orte-runtime: No such file or directory. Sorry!
> --------------------------------------------------------------------------
> [linux-0ze2:03493] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in file orterun.c at line 543
>
> I've changed content of PATH variable with help of export PATH=..... in my users .bashrc file
>
> export PATH=/opt/ThirdParty-2.2.2/platforms/linux64Gcc/gperftools-svn/bin:/opt/OpenFOAM-ParaView-3.12.0/bin:/usr/local/lib64/openmpi:/usr/local:/home/lisa/OpenFOAM/lisa-2.2.2/platforms/linux64GccDPOpt/bin:/opt/site/2.2.2/platforms/linux64GccDPOpt/bin:/opt/OpenFOAM-2.2.2/platforms/linux64GccDPOpt/bin:/opt/OpenFOAM-2.2.2/bin:/opt/OpenFOAM-2.2.2/wmake:/home/lisa/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games:/opt/openmpi-1.4.5
>
>
> The original PATH
> /opt/ThirdParty-2.2.2/platforms/linux64Gcc/gperftools-svn/bin:/opt/OpenFOAM-ParaView-3.12.0/bin:/usr/local/lib64/openmpi/bin:/home/lisa/OpenFOAM/lisa-2.2.2/platforms/linux64GccDPOpt/bin:/opt/site/2.2.2/platforms/linux64GccDPOpt/bin:/opt/OpenFOAM-2.2.2/platforms/linux64GccDPOpt/bin:/opt/OpenFOAM-2.2.2/bin:/opt/OpenFOAM-2.2.2/wmake:/home/lisa/bin:/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/X11R6/bin:/usr/games
>
>
>
> I've changed content of LD_LIBRARY_PATH variable with help of export LD_LIBRARY_PATH=..... in my users .bashrc file
>
> export LD_LIBRARY_PATH=/opt/ThirdParty-2.2.2/platforms/linux64Gcc/gperftools-svn/lib:/opt/OpenFOAM-ParaView-3.12.0/lib/paraview-3.12:/opt/OpenFOAM-scotch-6.0.0/lib:/usr/local/lib64/openmpi:/opt/ThirdParty-2.2.2/platforms/linux64GccDPOpt/lib/openmpi-1.6.3:/opt/ThirdParty-2.2.2/platforms/linux64Gcc/openmpi-1.6.3/lib:/opt/ThirdParty-2.2.2/platforms/linux64Gcc/openmpi-1.6.3/lib64:/home/lisa/OpenFOAM/lisa-2.2.2/platforms/linux64GccDPOpt/lib:/opt/site/2.2.2/platforms/linux64GccDPOpt/lib:/opt/OpenFOAM-2.2.2/platforms/linux64GccDPOpt/lib:/opt/ThirdParty-2.2.2/platforms/linux64GccDPOpt/lib:/opt/OpenFOAM-2.2.2/platforms/linux64GccDPOpt/lib/dummy:/usr/lib64/mpi/gcc/openmpi/lib64:/usr/local/lib64:/opt/openmpi-1.4.5:/usr/local
>
> The original LD_LIBRAR_PATH
>
> /opt/ThirdParty-2.2.2/platforms/linux64Gcc/gperftools-svn/lib:/opt/OpenFOAM-ParaView-3.12.0/lib/paraview-3.12:/opt/OpenFOAM-scotch-6.0.0/lib:/opt/OpenFOAM-2.2.2/platforms/linux64GccDPOpt/lib/openmpi-1.4.5:/opt/ThirdParty-2.2.2/platforms/linux64GccDPOpt/lib/openmpi-1.4.5:/usr/local/lib64/openmpi/lib:/usr/local/lib64/openmpi/lib64:/home/lisa/OpenFOAM/lisa-2.2.2/platforms/linux64GccDPOpt/lib:/opt/site/2.2.2/platforms/linux64GccDPOpt/lib:/opt/OpenFOAM-2.2.2/platforms/linux64GccDPOpt/lib:/opt/ThirdParty-2.2.2/platforms/linux64GccDPOpt/lib:/opt/OpenFOAM-2.2.2/platforms/linux64GccDPOpt/lib/dummy:/usr/lib64/mpi/gcc/openmpi/lib64:/usr/local/lib64
>
>
> I've changed the file /opt/OpenFOAM-2.2.2/etc/config/settings.sh as well! --> OPENMPI
>
> since I've adjusted .bashrc of OpenFoam :
>
> #- MPI implementation:
> # WM_MPLIB = SYSTEMOPENMPI | OPENMPI | MPICH | MPICH-GM | HPMPI
> # | GAMMA | MPI | QSMPI | SGIMPI
> export WM_MPLIB=OPENMPI
>
> # Communications library
> # ~~~~~~~~~~~~~~~~~~~~~~
>
>
> /opt/OpenFOAM-2.2.2/etc/config/settings.sh:
>
> unset MPI_ARCH_PATH MPI_HOME FOAM_MPI_LIBBIN
>
> case "$WM_MPLIB" in
> SYSTEMOPENMPI)
> # Use the system installed openmpi, get library directory via mpicc
> export FOAM_MPI=openmpi-system
>
> libDir=`mpicc --showme:link | sed -e 's/.*-L\([^ ]*\).*/\1/'`
>
> # Bit of a hack: strip off 'lib' and hope this is the path to openmpi
> # include files and libraries.
> export MPI_ARCH_PATH="${libDir%/*}"
>
> _foamAddLib $libDir
> unset libDir
> ;;
>
> OPENMPI)
> #export FOAM_MPI=openmpi-1.6.3
> export FOAM_MPI=openmpi-1.4.5
> # optional configuration tweaks:
> _foamSource `$WM_PROJECT_DIR/bin/foamEtcFile config/openmpi.sh`
>
> #export MPI_ARCH_PATH=$WM_THIRD_PARTY_DIR/platforms/$WM_ARCH$WM_COMPILER/$FOAM_MPI
> export MPI_ARCH_PATH=/usr/local/lib64/openmpi
>
> # Tell OpenMPI where to find its install directory
> export OPAL_PREFIX=$MPI_ARCH_PATH
>
> _foamAddPath $MPI_ARCH_PATH/bin
>
> # 64-bit on OpenSuSE 12.1 uses lib64 others use lib
> _foamAddLib $MPI_ARCH_PATH/lib$WM_COMPILER_LIB_ARCH
> _foamAddLib $MPI_ARCH_PATH/lib
>
> _foamAddMan $MPI_ARCH_PATH/share/man
> ;;
>
>
> I am quite new and don't have experience neither with Linux nor with MPI. Please let me know if I should provide further information.
> Please help me to get rid of that error message and make mpi running.
>
> I am working on a single workstation with 16 nodes.
>
>
>
> Best regards,
> Elisabeth
>
>
>
>
>
>
>
>
> <config.log.bz2><ompi_info.txt><ompi_info_1.7>_______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users