Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

From: Joe Griffin (joe.griffin_at_[hidden])
Date: 2007-04-10 11:39:46


Hi Jeff

Comments after your text....

Jeff Squyres wrote 2/5/07:

>Greetings Joe.
>
>What we did was to make 2 sets of environment variables that you can
>use to modify OMPI's internal path behavior:
>
>1. OPAL_DESTDIR: If you want to use staged builds (a la RPM builds),
>you can set the environment variable OPAL_DESTDIR. For example:
>
> ./configure --prefix=/opt/ompi
> make install DESTDIR=/tmp
> # at this point, OMPI is installed into /tmp/opt/ompi
> setenv OPAL_DESTDIR /tmp
> # put /tmp/opt/ompi in your path, then run ompi_info
> ompi_info
>
>Hence, OPAL_DESTDIR simply prefixes every internal path with the
>destdir.
>
>2. OPAL_PREFIX: If you want to wholly override where the OMPI
>installation is, set OPAL_PREFIX to the new prefix value (quite
>similar to the LAMHOME environment variable, but slightly better).
>For example:
>
> ./configure --prefix=/opt/ompi
> make install
> mv /opt/ompi /home/ompi
> setenv OPAL_PREFIX /home/ompi
> ompi_info
>
>This will re-base all of OMPI's internal directories to key off the
>new prefix value. Similarly, OPAL_* environment variables exist for
>all other GNU-standard directories (OPAL_BINDIR,
>OPAL_INCLUEDIR, ...etc.). Note that the default values of these
>variables are their GNU counterparts, so OMPI_BINDIR defaults to $
>{prefix}/bin -- which requires resolving ${prefix} first (i.e.,
>OPAL_PREFIX). Hence, setting OPAL_PREFIX will effectively change all
>the others. We anticipate that most people will likely only ever use
>OPAL_PREFIX.
>
>
I finally got a chance to try V1.2. I am having the same issue:

I installed with:
./configure --prefix=/usr/local/openmpi-1.2-1
make prefix=/usr/local/openmpi-1.2-1
make install prefix=/usr/local/openmpi-1.2-1

I can run with:
em64t6 <155> /usr/local/openmpi-1.2-1/bin/mpirun -n 1 --host em64t6 hostname
em64t6
em64t6 <156> cp -r /usr/local/openmpi-1.2-1 /tmp
em64t6 <157> /tmp/openmpi-1.2-1/bin/mpirun -n 1 --host em64t6 hostname
em64t6

Then I move the directory:
[root_at_em64t6 openmpi-1.2]# cd /usr/local/
[root_at_em64t6 local]# mv openmpi-1.2-1/ openmpiHIDE

Then I try to run:

em64t6 <158> /tmp/openmpi-1.2-1/bin/mpirun -n 1 --host em64t6 hostname
/tmp/openmpi-1.2-1/bin/mpirun: error while loading shared libraries:
libopen-rte.so.0: cannot open shared object file: No such file or directory

If I set the env:
em64t6 <160> setenv OPAL_PREFIX /tmp/openmpi-1.2-1
em64t6 <164> setenv OPAL_DESTDIR /tmp/openmpi-1.2-1/
em64t6 <166> set path=( /tmp/openmpi-1.2-1/bin $path)
em64t6 <168> setenv LD_LIBRARY_PATH /tmp/openmpi-1.2-1/lib

The error becomes:
em64t6 <169> /tmp/openmpi-1.2-1/bin/mpirun -n 1 --host em64t6 hostname
[em64t6:30103] [NO-NAME] ORTE_ERROR_LOG: Not found in file
runtime/orte_init_stage1.c at line 214
--------------------------------------------------------------------------
Sorry! You were supposed to get help about:
    orte_init:startup:internal-failure
from the file:
    help-orte-runtime
But I couldn't find any file matching that name. Sorry!
--------------------------------------------------------------------------
[em64t6:30103] [NO-NAME] ORTE_ERROR_LOG: Not found in file
runtime/orte_system_init.c at line 42
[em64t6:30103] [NO-NAME] ORTE_ERROR_LOG: Not found in file
runtime/orte_init.c at line 52
--------------------------------------------------------------------------
Sorry! You were supposed to get help about:
    orterun:init-failure
from the file:
    help-orterun.txt
But I couldn't find any file matching that name. Sorry!
--------------------------------------------------------------------------

The output from ompi_info is:

em64t6 <170> ompi_info
                Open MPI: 1.2
   Open MPI SVN revision: r14027
                Open RTE: 1.2
   Open RTE SVN revision: r14027
                    OPAL: 1.2
       OPAL SVN revision: r14027
                  Prefix: /usr/local/openmpi-1.2-1
 Configured architecture: x86_64-unknown-linux-gnu
           Configured by: root
           Configured on: Tue Apr 10 09:39:28 PDT 2007
          Configure host: em64t6
                Built by: root
                Built on: Tue Apr 10 09:51:27 PDT 2007
              Built host: em64t6
              C bindings: yes
            C++ bindings: yes
      Fortran77 bindings: yes (all)
      Fortran90 bindings: no
 Fortran90 bindings size: na
              C compiler: gcc
     C compiler absolute: /usr/bin/gcc
            C++ compiler: g++
   C++ compiler absolute: /usr/bin/g++
      Fortran77 compiler: g77
  Fortran77 compiler abs: /usr/bin/g77
      Fortran90 compiler: none
  Fortran90 compiler abs: none
             C profiling: yes
           C++ profiling: yes
     Fortran77 profiling: yes
     Fortran90 profiling: no
          C++ exceptions: no
          Thread support: posix (mpi: no, progress: no)
  Internal debug support: no
     MPI parameter check: runtime
Memory profiling support: no
Memory debugging support: no
         libltdl support: yes
   Heterogeneous support: yes
 mpirun default --prefix: no
           MCA backtrace: execinfo (MCA v1.0, API v1.0, Component v1.2)
              MCA memory: ptmalloc2 (MCA v1.0, API v1.0, Component v1.2)
               MCA timer: linux (MCA v1.0, API v1.0, Component v1.2)

So the Prefix is not getting changed. Can someone please
tell me what I am doing wrong? I searched the archives,
but haven't found anything related.

I would like to:
1. install into one directory
2. Move the files
3. Run from the new directory.

The above is important if we redistribute and clients want to
install in a different directory.

Thanks,
Joe

>-----
>
>We've been calling this work "the installdirs patch" internally. It
>got finalized on a private branch literally just before we closed the
>1.2 tree for release, so it's not going to make it into the final
>v1.2 -- it was just too late in the release process. It's not even
>on the OMPI trunk yet; we wanted to get OMPI v1.2 out before putting
>this patch on the trunk (to keep the trunk and v1.2 "somewhat
>similar"). We haven't yet discussed 1.2.1 as a group yet -- it's
>possible that the installdirs patch will make it into there (but I
>can't guarantee that).
>
>That being said, I will be releasing OMPI v1.2ofed as part of the
>OpenFabrics Enterprise Distribution (OFED -- the open source
>InfiniBand stack) that will be the final community OMPI v1.2 release
>plus the installdirs patch. We need this installdirs stuff in the
>OFED installer for DESTDIR reasons.
>
>I'd be happy to share the installdirs patch with anyone who wants it;
>it just won't make it into OMPI v1.2.
>
>
>
>On Feb 2, 2007, at 6:28 PM, Joe Griffin wrote:
>
>
>
>>Hello,
>>
>>I can see that Patrick posted my same error on:
>>
>>http://www.open-mpi.org/community/lists/devel/2006/12/1230.php
>>
>>Can I please find out the status? I have a simple test case which
>>demonstrates the --prefix is not working on mpirun. I build into
>>/usr/local/openmpi-1.1.3
>>and it works. However, if I copy the files, remove the
>>/usr/local/openmpi-1.1.3/lib
>>and try to point to my new directory of /tmp/openmpi-1.1.3, it fails.
>>
>>
>>prompt <154> /usr/local/openmpi-1.1.3/bin/mpirun -n 1 --host em64t
>>hostname
>>em64t
>>prompt <155> cp -r /usr/local/openmpi-1.1.3 /tmp
>>prompt <156> /tmp/openmpi-1.1.3/bin/mpirun -n 1 --host em64t hostname
>>em64t
>>
>># As ROOT:
>>[root_at_em64t openmpi-1.1.3]# mv /usr/local/openmpi-1.1.3/lib
>>/usr/local/openmpi-1.1.3/LLL
>>
>>
>>prompt <157> /tmp/openmpi-1.1.3/bin/mpirun -n 1 --host em64t --prefix
>>/tmp/openmpi-1.1.3 hostname
>>[em64t:26023] [NO-NAME] ORTE_ERROR_LOG: Not found in file
>>runtime/orte_init_stage1.c at line 214
>>----------------------------------------------------------------------
>>----
>>It looks like orte_init failed for some reason; your parallel
>>process is
>>likely to abort. There are many reasons that a parallel process can
>>fail during orte_init; some of which are due to configuration or
>>environment problems. This failure appears to be an internal failure;
>>here's some additional information (which may only be relevant to an
>>Open MPI developer):
>>
>> orte_sds_base_select failed
>> --> Returned value -13 instead of ORTE_SUCCESS
>>
>>----------------------------------------------------------------------
>>----
>>[em64t:26023] [NO-NAME] ORTE_ERROR_LOG: Not found in file
>>runtime/orte_system_init.c at line 42
>>[em64t:26023] [NO-NAME] ORTE_ERROR_LOG: Not found in file
>>runtime/orte_init.c at line 49
>>----------------------------------------------------------------------
>>----
>>Open RTE was unable to initialize properly. The error occured while
>>attempting to orte_init(). Returned value -13 instead of
>>ORTE_SUCCESS.
>>----------------------------------------------------------------------
>>----
>>
>>
>>The man page implies that --prefix would work. I know my example is
>>silly. The real case
>>involves going over a cluster, but the above is the simpliest case I
>>could create.
>> From the man page:
>>
>> --prefix <dir>
>> Prefix directory that will be used to set the
>>PATH and
>> LD_LIBRARY_PATH on the remote node before invoking
>>Open MPI
>> or the target process. See the "Remote Execution"
>>section,
>> below.
>>
>>BTW - I am trying to migrate from LAM to OpenMPI. The switch was
>>very painless. Thanks for the good work.
>>
>>Regards,
>>Joe
>>
>>
>>_______________________________________________
>>users mailing list
>>users_at_[hidden]
>>http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>>
>
>
>
>