Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI users] Relocating an Open MPI installation using OPAL_PREFIX
From: Ethan Mallove (ethan.mallove_at_[hidden])
Date: 2009-01-05 13:35:15


On Thu, Dec/25/2008 08:12:49AM, Jeff Squyres wrote:
> It's quite possible that we don't handle this situation properly. Won't
> you need to libdir's (one for the 32 bit OMPI executables, and one for the
> 64 bit MPI apps)?

I don't need an OPAL environment variable for the executables, just a
single OPAL_LIBDIR var for the libraries. (One set of 32-bit
executables runs with both 32-bit and 64-bit libraries.) I'm guessing
OPAL_LIBDIR will not work for you if you configure with a non-standard
--libdir option.

-Ethan

>
> On Dec 23, 2008, at 3:58 PM, Ethan Mallove wrote:
>
>> I think the problem is that I am doing a multi-lib build. I have
>> 32-bit libraries in lib/, and 64-bit libraries in lib/64. I assume I
>> do not see the issue for 32-bit tests, because all the dependencies
>> are where Open MPI expects them to be. For the 64-bit case, I tried
>> setting OPAL_LIBDIR to /opt/openmpi-relocated/lib/lib64, but no luck.
>> Given the below configure arguments, what do my OPAL_* env vars need
>> to be? (Also, could using --enable-orterun-prefix-by-default interfere
>> with OPAL_PREFIX?)
>>
>> $ ./configure CC=cc CXX=CC F77=f77 FC=f90 --with-openib
>> --without-udapl --disable-openib-ibcm --enable-heterogeneous
>> --enable-cxx-exceptions --enable-shared --enable-orterun-prefix-by-default
>> --with-sge --enable-mpi-f90 --with-mpi-f90-size=small
>> --disable-mpi-threads --disable-progress-threads --disable-debug
>> CFLAGS="-m32 -xO5" CXXFLAGS="-m32 -xO5" FFLAGS="-m32 -xO5" FCFLAGS="-m32
>> -xO5"
>> --prefix=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install
>> --mandir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/man
>> --libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib
>> --includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/include
>> --without-mx --with-tm=/ws/ompi-tools/orte/torque/current/shared-install32
>> --with-contrib-vt-flags="--prefix=/workspace/em162155/hpc/mtt-scratch/burl-ct-v!
>> 20z-12/ompi-tarball-testing/installs/DGQx/install
>> --mandir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/man
>> --libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib
>> --includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/include
>> LDFLAGS=-R/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib"
>>
>> $ ./confgiure CC=cc CXX=CC F77=f77 FC=f90 --with-openib
>> --without-udapl --disable-openib-ibcm --enable-heterogeneous
>> --enable-cxx-exceptions --enable-shared --enable-orterun-prefix-by-default
>> --with-sge --enable-mpi-f90 --with-mpi-f90-size=small
>> --disable-mpi-threads --disable-progress-threads --disable-debug
>> CFLAGS="-m64 -xO5" CXXFLAGS="-m64 -xO5" FFLAGS="-m64 -xO5" FCFLAGS="-m64
>> -xO5"
>> --prefix=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install
>> --mandir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/man
>> --libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib/lib64
>> --includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/include/64
>> --without-mx --with-tm=/ws/ompi-tools/orte/torque/current/shared-install64
>> --with-contrib-vt-flags="--prefix=/workspace/em162155/hpc/mtt-scratch/!
>> burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install
>> --mandir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/man
>> --libdir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib/lib64
>> --includedir=/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/include/64
>> LDFLAGS=-R/workspace/em162155/hpc/mtt-scratch/burl-ct-v20z-12/ompi-tarball-testing/installs/DGQx/install/lib"
>> --disable-binaries
>>
>> -Ethan
>>
>>>
>>>
>>> On Dec 22, 2008, at 12:42 PM, Ethan Mallove wrote:
>>>
>>>> Can anyone get OPAL_PREFIX to work on Linux? A simple test is to see
>>>> if the following works for any mpicc/mpirun:
>>>>
>>>> $ mv <openmpi-installation> /tmp/foo
>>>> $ set OPAL_PREFIX /tmp/foo
>>>> $ mpicc ...
>>>> $ mpirun ...
>>>>
>>>> If you are able to get the above to run successfully, I'm interested
>>>> in your config.log file.
>>>>
>>>> Thanks,
>>>> Ethan
>>>>
>>>>
>>>> On Thu, Dec/18/2008 11:03:25AM, Ethan Mallove wrote:
>>>>> Hello,
>>>>>
>>>>> The below FAQ lists instructions on how to use a relocated Open MPI
>>>>> installation:
>>>>>
>>>>> http://www.open-mpi.org/faq/?category=building#installdirs
>>>>>
>>>>> On Solaris, OPAL_PREFIX and friends (documented in the FAQ) work for
>>>>> me with both MPI (hello_c) and non-MPI (hostname) programs. On Linux,
>>>>> I can only get the non-MPI case to work. Here are the environment
>>>>> variables I am setting:
>>>>>
>>>>> $ cat setenv_opal_prefix.csh
>>>>> set opal_prefix = "/opt/openmpi-relocated"
>>>>>
>>>>> setenv OPAL_PREFIX $opal_prefix
>>>>> setenv OPAL_BINDIR $opal_prefix/bin
>>>>> setenv OPAL_SBINDIR $opal_prefix/sbin
>>>>> setenv OPAL_DATAROOTDIR $opal_prefix/share
>>>>> setenv OPAL_SYSCONFDIR $opal_prefix/etc
>>>>> setenv OPAL_SHAREDSTATEDIR $opal_prefix/com
>>>>> setenv OPAL_LOCALSTATEDIR $opal_prefix/var
>>>>> setenv OPAL_LIBDIR $opal_prefix/lib
>>>>> setenv OPAL_INCLUDEDIR $opal_prefix/include
>>>>> setenv OPAL_INFODIR $opal_prefix/info
>>>>> setenv OPAL_MANDIR $opal_prefix/man
>>>>>
>>>>> setenv PATH $opal_prefix/bin:$PATH
>>>>> setenv LD_LIBRARY_PATH $opal_prefix/lib:$opal_prefix/lib/64
>>>>>
>>>>> Here is the error I get:
>>>>>
>>>>> $ mpirun -np 2 hello_c
>>>>>
>>>>> --------------------------------------------------------------------------
>>>>> It looks like opal_init failed for some reason; your parallel process
>>>>> is
>>>>> likely to abort. There are many reasons that a parallel process can
>>>>> fail during opal_init; some of which are due to configuration or
>>>>> environment problems. This failure appears to be an internal failure;
>>>>> here's some additional information (which may only be relevant to an
>>>>> Open MPI developer):
>>>>>
>>>>> opal_carto_base_select failed
>>>>> --> Returned value -13 instead of OPAL_SUCCESS
>>>>>
>>>>> --------------------------------------------------------------------------
>>>>> [burl-ct-v20z-0:27737] [[INVALID],INVALID] ORTE_ERROR_LOG: Not found in
>>>>> file runtime/orte_init.c at line 77
>>>>>
>>>>> Any ideas on what's going on?
>>>>>
>>>>> Thanks,
>>>>> Ethan
>>>> _______________________________________________
>>>> users mailing list
>>>> users_at_[hidden]
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>
>>>
>>> --
>>> Jeff Squyres
>>> Cisco Systems
>>>
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> --
> Jeff Squyres
> Cisco Systems
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users