Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Joe Griffin (joe.griffin_at_[hidden])
Date: 2007-04-10 11:39:46


Hi Jeff

Comments after your text....

Jeff Squyres wrote 2/5/07:

>Greetings Joe.
>
>What we did was to make 2 sets of environment variables that you can
>use to modify OMPI's internal path behavior:
>
>1. OPAL_DESTDIR: If you want to use staged builds (a la RPM builds),
>you can set the environment variable OPAL_DESTDIR. For example:
>
> ./configure --prefix=/opt/ompi
> make install DESTDIR=/tmp
> # at this point, OMPI is installed into /tmp/opt/ompi
> setenv OPAL_DESTDIR /tmp
> # put /tmp/opt/ompi in your path, then run ompi_info
> ompi_info
>
>Hence, OPAL_DESTDIR simply prefixes every internal path with the
>destdir.
>
>2. OPAL_PREFIX: If you want to wholly override where the OMPI
>installation is, set OPAL_PREFIX to the new prefix value (quite
>similar to the LAMHOME environment variable, but slightly better).
>For example:
>
> ./configure --prefix=/opt/ompi
> make install
> mv /opt/ompi /home/ompi
> setenv OPAL_PREFIX /home/ompi
> ompi_info
>
>This will re-base all of OMPI's internal directories to key off the
>new prefix value. Similarly, OPAL_* environment variables exist for
>all other GNU-standard directories (OPAL_BINDIR,
>OPAL_INCLUEDIR, ...etc.). Note that the default values of these
>variables are their GNU counterparts, so OMPI_BINDIR defaults to $
>{prefix}/bin -- which requires resolving ${prefix} first (i.e.,
>OPAL_PREFIX). Hence, setting OPAL_PREFIX will effectively change all
>the others. We anticipate that most people will likely only ever use
>OPAL_PREFIX.
>
>
I finally got a chance to try V1.2. I am having the same issue:

I installed with:
./configure --prefix=/usr/local/openmpi-1.2-1
make prefix=/usr/local/openmpi-1.2-1
make install prefix=/usr/local/openmpi-1.2-1

I can run with:
em64t6 <155> /usr/local/openmpi-1.2-1/bin/mpirun -n 1 --host em64t6 hostname
em64t6
em64t6 <156> cp -r /usr/local/openmpi-1.2-1 /tmp
em64t6 <157> /tmp/openmpi-1.2-1/bin/mpirun -n 1 --host em64t6 hostname
em64t6

Then I move the directory:
[root_at_em64t6 openmpi-1.2]# cd /usr/local/
[root_at_em64t6 local]# mv openmpi-1.2-1/ openmpiHIDE

Then I try to run:

em64t6 <158> /tmp/openmpi-1.2-1/bin/mpirun -n 1 --host em64t6 hostname
/tmp/openmpi-1.2-1/bin/mpirun: error while loading shared libraries:
libopen-rte.so.0: cannot open shared object file: No such file or directory

If I set the env:
em64t6 <160> setenv OPAL_PREFIX /tmp/openmpi-1.2-1
em64t6 <164> setenv OPAL_DESTDIR /tmp/openmpi-1.2-1/
em64t6 <166> set path=( /tmp/openmpi-1.2-1/bin $path)
em64t6 <168> setenv LD_LIBRARY_PATH /tmp/openmpi-1.2-1/lib

The error becomes:
em64t6 <169> /tmp/openmpi-1.2-1/bin/mpirun -n 1 --host em64t6 hostname
[em64t6:30103] [NO-NAME] ORTE_ERROR_LOG: Not found in file
runtime/orte_init_stage1.c at line 214
--------------------------------------------------------------------------
Sorry! You were supposed to get help about:
    orte_init:startup:internal-failure
from the file:
    help-orte-runtime
But I couldn't find any file matching that name. Sorry!
--------------------------------------------------------------------------
[em64t6:30103] [NO-NAME] ORTE_ERROR_LOG: Not found in file
runtime/orte_system_init.c at line 42
[em64t6:30103] [NO-NAME] ORTE_ERROR_LOG: Not found in file
runtime/orte_init.c at line 52
--------------------------------------------------------------------------
Sorry! You were supposed to get help about:
    orterun:init-failure
from the file:
    help-orterun.txt
But I couldn't find any file matching that name. Sorry!
--------------------------------------------------------------------------

The output from ompi_info is:

em64t6 <170> ompi_info
                Open MPI: 1.2
   Open MPI SVN revision: r14027
                Open RTE: 1.2
   Open RTE SVN revision: r14027
                    OPAL: 1.2
       OPAL SVN revision: r14027
                  Prefix: /usr/local/openmpi-1.2-1
 Configured architecture: x86_64-unknown-linux-gnu
           Configured by: root
           Configured on: Tue Apr 10 09:39:28 PDT 2007
          Configure host: em64t6
                Built by: root
                Built on: Tue Apr 10 09:51:27 PDT 2007
              Built host: em64t6
              C bindings: yes
            C++ bindings: yes
      Fortran77 bindings: yes (all)
      Fortran90 bindings: no
 Fortran90 bindings size: na
              C compiler: gcc
     C compiler absolute: /usr/bin/gcc
            C++ compiler: g++
   C++ compiler absolute: /usr/bin/g++
      Fortran77 compiler: g77
  Fortran77 compiler abs: /usr/bin/g77
      Fortran90 compiler: none
  Fortran90 compiler abs: none
             C profiling: yes
           C++ profiling: yes
     Fortran77 profiling: yes
     Fortran90 profiling: no
          C++ exceptions: no
          Thread support: posix (mpi: no, progress: no)
  Internal debug support: no
     MPI parameter check: runtime
Memory profiling support: no
Memory debugging support: no
         libltdl support: yes
   Heterogeneous support: yes
 mpirun default --prefix: no
           MCA backtrace: execinfo (MCA v1.0, API v1.0, Component v1.2)
              MCA memory: ptmalloc2 (MCA v1.0, API v1.0, Component v1.2)
               MCA timer: linux (MCA v1.0, API v1.0, Component v1.2)

So the Prefix is not getting changed. Can someone please
tell me what I am doing wrong? I searched the archives,
but haven't found anything related.

I would like to:
1. install into one directory
2. Move the files
3. Run from the new directory.

The above is important if we redistribute and clients want to
install in a different directory.

Thanks,
Joe

>-----
>
>We've been calling this work "the installdirs patch" internally. It
>got finalized on a private branch literally just before we closed the
>1.2 tree for release, so it's not going to make it into the final
>v1.2 -- it was just too late in the release process. It's not even
>on the OMPI trunk yet; we wanted to get OMPI v1.2 out before putting
>this patch on the trunk (to keep the trunk and v1.2 "somewhat
>similar"). We haven't yet discussed 1.2.1 as a group yet -- it's
>possible that the installdirs patch will make it into there (but I
>can't guarantee that).
>
>That being said, I will be releasing OMPI v1.2ofed as part of the
>OpenFabrics Enterprise Distribution (OFED -- the open source
>InfiniBand stack) that will be the final community OMPI v1.2 release
>plus the installdirs patch. We need this installdirs stuff in the
>OFED installer for DESTDIR reasons.
>
>I'd be happy to share the installdirs patch with anyone who wants it;
>it just won't make it into OMPI v1.2.
>
>
>
>On Feb 2, 2007, at 6:28 PM, Joe Griffin wrote:
>
>
>
>>Hello,
>>
>>I can see that Patrick posted my same error on:
>>
>>http://www.open-mpi.org/community/lists/devel/2006/12/1230.php
>>
>>Can I please find out the status? I have a simple test case which
>>demonstrates the --prefix is not working on mpirun. I build into
>>/usr/local/openmpi-1.1.3
>>and it works. However, if I copy the files, remove the
>>/usr/local/openmpi-1.1.3/lib
>>and try to point to my new directory of /tmp/openmpi-1.1.3, it fails.
>>
>>
>>prompt <154> /usr/local/openmpi-1.1.3/bin/mpirun -n 1 --host em64t
>>hostname
>>em64t
>>prompt <155> cp -r /usr/local/openmpi-1.1.3 /tmp
>>prompt <156> /tmp/openmpi-1.1.3/bin/mpirun -n 1 --host em64t hostname
>>em64t
>>
>># As ROOT:
>>[root_at_em64t openmpi-1.1.3]# mv /usr/local/openmpi-1.1.3/lib
>>/usr/local/openmpi-1.1.3/LLL
>>
>>
>>prompt <157> /tmp/openmpi-1.1.3/bin/mpirun -n 1 --host em64t --prefix
>>/tmp/openmpi-1.1.3 hostname
>>[em64t:26023] [NO-NAME] ORTE_ERROR_LOG: Not found in file
>>runtime/orte_init_stage1.c at line 214
>>----------------------------------------------------------------------
>>----
>>It looks like orte_init failed for some reason; your parallel
>>process is
>>likely to abort. There are many reasons that a parallel process can
>>fail during orte_init; some of which are due to configuration or
>>environment problems. This failure appears to be an internal failure;
>>here's some additional information (which may only be relevant to an
>>Open MPI developer):
>>
>> orte_sds_base_select failed
>> --> Returned value -13 instead of ORTE_SUCCESS
>>
>>----------------------------------------------------------------------
>>----
>>[em64t:26023] [NO-NAME] ORTE_ERROR_LOG: Not found in file
>>runtime/orte_system_init.c at line 42
>>[em64t:26023] [NO-NAME] ORTE_ERROR_LOG: Not found in file
>>runtime/orte_init.c at line 49
>>----------------------------------------------------------------------
>>----
>>Open RTE was unable to initialize properly. The error occured while
>>attempting to orte_init(). Returned value -13 instead of
>>ORTE_SUCCESS.
>>----------------------------------------------------------------------
>>----
>>
>>
>>The man page implies that --prefix would work. I know my example is
>>silly. The real case
>>involves going over a cluster, but the above is the simpliest case I
>>could create.
>> From the man page:
>>
>> --prefix <dir>
>> Prefix directory that will be used to set the
>>PATH and
>> LD_LIBRARY_PATH on the remote node before invoking
>>Open MPI
>> or the target process. See the "Remote Execution"
>>section,
>> below.
>>
>>BTW - I am trying to migrate from LAM to OpenMPI. The switch was
>>very painless. Thanks for the good work.
>>
>>Regards,
>>Joe
>>
>>
>>_______________________________________________
>>users mailing list
>>users_at_[hidden]
>>http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>>
>
>
>
>