Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Relocating an Open MPI installation using OPAL_PREFIX
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2008-12-25 08:12:49


It's quite possible that we don't handle this situation properly.
Won't you need to libdir's (one for the 32 bit OMPI executables, and
one for the 64 bit MPI apps)?

On Dec 23, 2008, at 3:58 PM, Ethan Mallove wrote:

> I think the problem is that I am doing a multi-lib build. I have
> 32-bit libraries in lib/, and 64-bit libraries in lib/64. I assume I
> do not see the issue for 32-bit tests, because all the dependencies
> are where Open MPI expects them to be. For the 64-bit case, I tried
> setting OPAL_LIBDIR to /opt/openmpi-relocated/lib/lib64, but no luck.
> Given the below configure arguments, what do my OPAL_* env vars need
> to be? (Also, could using --enable-orterun-prefix-by-default interfere
> with OPAL_PREFIX?)
>
> $ ./configure CC=cc CXX=CC F77=f77 FC=f90 --with-openib --
> without-udapl --disable-openib-ibcm --enable-heterogeneous --enable-
> cxx-exceptions --enable-shared --enable-orterun-prefix-by-default --
> with-sge --enable-mpi-f90 --with-mpi-f90-size=small --disable-mpi-
> threads --disable-progress-threads --disable-debug CFLAGS="-m32 -
> xO5" CXXFLAGS="-m32 -xO5" FFLAGS="-m32 -xO5" FCFLAGS="-m32 -xO5" --
> prefix=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-
> tarball-testing/installs/DGQx/install --mandir=/workspace/em162155/
> hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/
> install/man --libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-
> v20z-12/ompi-tarball-testing/installs/DGQx/install/lib --includedir=/
> workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-
> testing/installs/DGQx/install/include --without-mx --with-tm=/ws/
> ompi-tools/orte/torque/current/shared-install32 --with-contrib-vt-
> flags="--prefix=/workspace/em162155/hpc/mtt-scratch/burl-ct-v!
> 20z-12/ompi-tarball-testing/installs/DGQx/install --mandir=/
> workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-
> testing/installs/DGQx/install/man --libdir=/workspace/em162155/hpc/
> mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/
> install/lib --includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-
> v20z-12/ompi-tarball-testing/installs/DGQx/install/include LDFLAGS=-
> R/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-
> testing/installs/DGQx/install/lib"
>
> $ ./confgiure CC=cc CXX=CC F77=f77 FC=f90 --with-openib --
> without-udapl --disable-openib-ibcm --enable-heterogeneous --enable-
> cxx-exceptions --enable-shared --enable-orterun-prefix-by-default --
> with-sge --enable-mpi-f90 --with-mpi-f90-size=small --disable-mpi-
> threads --disable-progress-threads --disable-debug CFLAGS="-m64 -
> xO5" CXXFLAGS="-m64 -xO5" FFLAGS="-m64 -xO5" FCFLAGS="-m64 -xO5" --
> prefix=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-
> tarball-testing/installs/DGQx/install --mandir=/workspace/em162155/
> hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/
> install/man --libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-
> v20z-12/ompi-tarball-testing/installs/DGQx/install/lib/lib64 --
> includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-
> tarball-testing/installs/DGQx/install/include/64 --without-mx --with-
> tm=/ws/ompi-tools/orte/torque/current/shared-install64 --with-
> contrib-vt-flags="--prefix=/workspace/em162155/hpc/mtt-scratch/!
> burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install --mandir=/
> workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-
> testing/installs/DGQx/install/man --libdir=/workspace/em162155/hpc/
> mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/
> install/lib/lib64 --includedir=/workspace/em162155/hpc/mtt-scratch/
> burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/include/
> 64 LDFLAGS=-R/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/
> ompi-tarball-testing/installs/DGQx/install/lib" --disable-binaries
>
> -Ethan
>
>>
>>
>> On Dec 22, 2008, at 12:42 PM, Ethan Mallove wrote:
>>
>>> Can anyone get OPAL_PREFIX to work on Linux? A simple test is to see
>>> if the following works for any mpicc/mpirun:
>>>
>>> $ mv <openmpi-installation> /tmp/foo
>>> $ set OPAL_PREFIX /tmp/foo
>>> $ mpicc ...
>>> $ mpirun ...
>>>
>>> If you are able to get the above to run successfully, I'm interested
>>> in your config.log file.
>>>
>>> Thanks,
>>> Ethan
>>>
>>>
>>> On Thu, Dec/18/2008 11:03:25AM, Ethan Mallove wrote:
>>>> Hello,
>>>>
>>>> The below FAQ lists instructions on how to use a relocated Open MPI
>>>> installation:
>>>>
>>>> http://www.open-mpi.org/faq/?category=building#installdirs
>>>>
>>>> On Solaris, OPAL_PREFIX and friends (documented in the FAQ) work
>>>> for
>>>> me with both MPI (hello_c) and non-MPI (hostname) programs. On
>>>> Linux,
>>>> I can only get the non-MPI case to work. Here are the environment
>>>> variables I am setting:
>>>>
>>>> $ cat setenv_opal_prefix.csh
>>>> set opal_prefix = "/opt/openmpi-relocated"
>>>>
>>>> setenv OPAL_PREFIX $opal_prefix
>>>> setenv OPAL_BINDIR $opal_prefix/bin
>>>> setenv OPAL_SBINDIR $opal_prefix/sbin
>>>> setenv OPAL_DATAROOTDIR $opal_prefix/share
>>>> setenv OPAL_SYSCONFDIR $opal_prefix/etc
>>>> setenv OPAL_SHAREDSTATEDIR $opal_prefix/com
>>>> setenv OPAL_LOCALSTATEDIR $opal_prefix/var
>>>> setenv OPAL_LIBDIR $opal_prefix/lib
>>>> setenv OPAL_INCLUDEDIR $opal_prefix/include
>>>> setenv OPAL_INFODIR $opal_prefix/info
>>>> setenv OPAL_MANDIR $opal_prefix/man
>>>>
>>>> setenv PATH $opal_prefix/bin:$PATH
>>>> setenv LD_LIBRARY_PATH $opal_prefix/lib:$opal_prefix/lib/64
>>>>
>>>> Here is the error I get:
>>>>
>>>> $ mpirun -np 2 hello_c
>>>>
>>>> --------------------------------------------------------------------------
>>>> It looks like opal_init failed for some reason; your parallel
>>>> process is
>>>> likely to abort. There are many reasons that a parallel process
>>>> can
>>>> fail during opal_init; some of which are due to configuration or
>>>> environment problems. This failure appears to be an internal
>>>> failure;
>>>> here's some additional information (which may only be relevant to
>>>> an
>>>> Open MPI developer):
>>>>
>>>> opal_carto_base_select failed
>>>> --> Returned value -13 instead of OPAL_SUCCESS
>>>>
>>>> --------------------------------------------------------------------------
>>>> [burl-ct-v20z-0:27737] [[INVALID],INVALID] ORTE_ERROR_LOG: Not
>>>> found in
>>>> file runtime/orte_init.c at line 77
>>>>
>>>> Any ideas on what's going on?
>>>>
>>>> Thanks,
>>>> Ethan
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>>
>> --
>> Jeff Squyres
>> Cisco Systems
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users

-- 
Jeff Squyres
Cisco Systems