Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2007-02-05 11:58:41

Greetings Joe.

What we did was to make 2 sets of environment variables that you can
use to modify OMPI's internal path behavior:

1. OPAL_DESTDIR: If you want to use staged builds (a la RPM builds),
you can set the environment variable OPAL_DESTDIR. For example:

     ./configure --prefix=/opt/ompi
     make install DESTDIR=/tmp
     # at this point, OMPI is installed into /tmp/opt/ompi
     setenv OPAL_DESTDIR /tmp
     # put /tmp/opt/ompi in your path, then run ompi_info

Hence, OPAL_DESTDIR simply prefixes every internal path with the

2. OPAL_PREFIX: If you want to wholly override where the OMPI
installation is, set OPAL_PREFIX to the new prefix value (quite
similar to the LAMHOME environment variable, but slightly better).
For example:

     ./configure --prefix=/opt/ompi
     make install
     mv /opt/ompi /home/ompi
     setenv OPAL_PREFIX /home/ompi

This will re-base all of OMPI's internal directories to key off the
new prefix value. Similarly, OPAL_* environment variables exist for
all other GNU-standard directories (OPAL_BINDIR,
OPAL_INCLUEDIR, ...etc.). Note that the default values of these
variables are their GNU counterparts, so OMPI_BINDIR defaults to $
{prefix}/bin -- which requires resolving ${prefix} first (i.e.,
OPAL_PREFIX). Hence, setting OPAL_PREFIX will effectively change all
the others. We anticipate that most people will likely only ever use


We've been calling this work "the installdirs patch" internally. It
got finalized on a private branch literally just before we closed the
1.2 tree for release, so it's not going to make it into the final
v1.2 -- it was just too late in the release process. It's not even
on the OMPI trunk yet; we wanted to get OMPI v1.2 out before putting
this patch on the trunk (to keep the trunk and v1.2 "somewhat
similar"). We haven't yet discussed 1.2.1 as a group yet -- it's
possible that the installdirs patch will make it into there (but I
can't guarantee that).

That being said, I will be releasing OMPI v1.2ofed as part of the
OpenFabrics Enterprise Distribution (OFED -- the open source
InfiniBand stack) that will be the final community OMPI v1.2 release
plus the installdirs patch. We need this installdirs stuff in the
OFED installer for DESTDIR reasons.

I'd be happy to share the installdirs patch with anyone who wants it;
it just won't make it into OMPI v1.2.

On Feb 2, 2007, at 6:28 PM, Joe Griffin wrote:

> Hello,
> I can see that Patrick posted my same error on:
> Can I please find out the status? I have a simple test case which
> demonstrates the --prefix is not working on mpirun. I build into
> /usr/local/openmpi-1.1.3
> and it works. However, if I copy the files, remove the
> /usr/local/openmpi-1.1.3/lib
> and try to point to my new directory of /tmp/openmpi-1.1.3, it fails.
> prompt <154> /usr/local/openmpi-1.1.3/bin/mpirun -n 1 --host em64t
> hostname
> em64t
> prompt <155> cp -r /usr/local/openmpi-1.1.3 /tmp
> prompt <156> /tmp/openmpi-1.1.3/bin/mpirun -n 1 --host em64t hostname
> em64t
> # As ROOT:
> [root_at_em64t openmpi-1.1.3]# mv /usr/local/openmpi-1.1.3/lib
> /usr/local/openmpi-1.1.3/LLL
> prompt <157> /tmp/openmpi-1.1.3/bin/mpirun -n 1 --host em64t --prefix
> /tmp/openmpi-1.1.3 hostname
> [em64t:26023] [NO-NAME] ORTE_ERROR_LOG: Not found in file
> runtime/orte_init_stage1.c at line 214
> ----------------------------------------------------------------------
> ----
> It looks like orte_init failed for some reason; your parallel
> process is
> likely to abort. There are many reasons that a parallel process can
> fail during orte_init; some of which are due to configuration or
> environment problems. This failure appears to be an internal failure;
> here's some additional information (which may only be relevant to an
> Open MPI developer):
> orte_sds_base_select failed
> --> Returned value -13 instead of ORTE_SUCCESS
> ----------------------------------------------------------------------
> ----
> [em64t:26023] [NO-NAME] ORTE_ERROR_LOG: Not found in file
> runtime/orte_system_init.c at line 42
> [em64t:26023] [NO-NAME] ORTE_ERROR_LOG: Not found in file
> runtime/orte_init.c at line 49
> ----------------------------------------------------------------------
> ----
> Open RTE was unable to initialize properly. The error occured while
> attempting to orte_init(). Returned value -13 instead of
> ----------------------------------------------------------------------
> ----
> The man page implies that --prefix would work. I know my example is
> silly. The real case
> involves going over a cluster, but the above is the simpliest case I
> could create.
> From the man page:
> --prefix <dir>
> Prefix directory that will be used to set the
> PATH and
> LD_LIBRARY_PATH on the remote node before invoking
> Open MPI
> or the target process. See the "Remote Execution"
> section,
> below.
> BTW - I am trying to migrate from LAM to OpenMPI. The switch was
> very painless. Thanks for the good work.
> Regards,
> Joe
> _______________________________________________
> users mailing list
> users_at_[hidden]

Jeff Squyres
Server Virtualization Business Unit
Cisco Systems