Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2007-02-05 12:31:06


On Feb 5, 2007, at 12:19 PM, Joe Griffin wrote:

> Thanks. As always .. you are a perl.

FWIW -- credit Brian on this one. He did 98% of the work; I only did
the final validation of it. :-)

> I will try that with 1.2.

Are you saying you'd like me to ship you the patch? Or you'll wait
for v1.2ofed? (one clarification: the v1.2ofed release will be a
full OMPI SRPM -- not InfiniBand specific at all)

> Joe
>
> Jeff Squyres wrote:
>
>> Greetings Joe.
>>
>> What we did was to make 2 sets of environment variables that you can
>> use to modify OMPI's internal path behavior:
>>
>> 1. OPAL_DESTDIR: If you want to use staged builds (a la RPM builds),
>> you can set the environment variable OPAL_DESTDIR. For example:
>>
>> ./configure --prefix=/opt/ompi
>> make install DESTDIR=/tmp
>> # at this point, OMPI is installed into /tmp/opt/ompi
>> setenv OPAL_DESTDIR /tmp
>> # put /tmp/opt/ompi in your path, then run ompi_info
>> ompi_info
>>
>> Hence, OPAL_DESTDIR simply prefixes every internal path with the
>> destdir.
>>
>> 2. OPAL_PREFIX: If you want to wholly override where the OMPI
>> installation is, set OPAL_PREFIX to the new prefix value (quite
>> similar to the LAMHOME environment variable, but slightly better).
>> For example:
>>
>> ./configure --prefix=/opt/ompi
>> make install
>> mv /opt/ompi /home/ompi
>> setenv OPAL_PREFIX /home/ompi
>> ompi_info
>>
>> This will re-base all of OMPI's internal directories to key off the
>> new prefix value. Similarly, OPAL_* environment variables exist for
>> all other GNU-standard directories (OPAL_BINDIR,
>> OPAL_INCLUEDIR, ...etc.). Note that the default values of these
>> variables are their GNU counterparts, so OMPI_BINDIR defaults to $
>> {prefix}/bin -- which requires resolving ${prefix} first (i.e.,
>> OPAL_PREFIX). Hence, setting OPAL_PREFIX will effectively change all
>> the others. We anticipate that most people will likely only ever use
>> OPAL_PREFIX.
>>
>> -----
>>
>> We've been calling this work "the installdirs patch" internally. It
>> got finalized on a private branch literally just before we closed the
>> 1.2 tree for release, so it's not going to make it into the final
>> v1.2 -- it was just too late in the release process. It's not even
>> on the OMPI trunk yet; we wanted to get OMPI v1.2 out before putting
>> this patch on the trunk (to keep the trunk and v1.2 "somewhat
>> similar"). We haven't yet discussed 1.2.1 as a group yet -- it's
>> possible that the installdirs patch will make it into there (but I
>> can't guarantee that).
>>
>> That being said, I will be releasing OMPI v1.2ofed as part of the
>> OpenFabrics Enterprise Distribution (OFED -- the open source
>> InfiniBand stack) that will be the final community OMPI v1.2 release
>> plus the installdirs patch. We need this installdirs stuff in the
>> OFED installer for DESTDIR reasons.
>>
>> I'd be happy to share the installdirs patch with anyone who wants it;
>> it just won't make it into OMPI v1.2.
>>
>>
>>
>> On Feb 2, 2007, at 6:28 PM, Joe Griffin wrote:
>>
>>
>>
>>> Hello,
>>>
>>> I can see that Patrick posted my same error on:
>>>
>>> http://www.open-mpi.org/community/lists/devel/2006/12/1230.php
>>>
>>> Can I please find out the status? I have a simple test case which
>>> demonstrates the --prefix is not working on mpirun. I build into
>>> /usr/local/openmpi-1.1.3
>>> and it works. However, if I copy the files, remove the
>>> /usr/local/openmpi-1.1.3/lib
>>> and try to point to my new directory of /tmp/openmpi-1.1.3, it
>>> fails.
>>>
>>>
>>> prompt <154> /usr/local/openmpi-1.1.3/bin/mpirun -n 1 --host em64t
>>> hostname
>>> em64t
>>> prompt <155> cp -r /usr/local/openmpi-1.1.3 /tmp
>>> prompt <156> /tmp/openmpi-1.1.3/bin/mpirun -n 1 --host em64t
>>> hostname
>>> em64t
>>>
>>> # As ROOT:
>>> [root_at_em64t openmpi-1.1.3]# mv /usr/local/openmpi-1.1.3/lib
>>> /usr/local/openmpi-1.1.3/LLL
>>>
>>>
>>> prompt <157> /tmp/openmpi-1.1.3/bin/mpirun -n 1 --host em64t --
>>> prefix
>>> /tmp/openmpi-1.1.3 hostname
>>> [em64t:26023] [NO-NAME] ORTE_ERROR_LOG: Not found in file
>>> runtime/orte_init_stage1.c at line 214
>>> --------------------------------------------------------------------
>>> --
>>> ----
>>> It looks like orte_init failed for some reason; your parallel
>>> process is
>>> likely to abort. There are many reasons that a parallel process can
>>> fail during orte_init; some of which are due to configuration or
>>> environment problems. This failure appears to be an internal
>>> failure;
>>> here's some additional information (which may only be relevant to an
>>> Open MPI developer):
>>>
>>> orte_sds_base_select failed
>>> --> Returned value -13 instead of ORTE_SUCCESS
>>>
>>> --------------------------------------------------------------------
>>> --
>>> ----
>>> [em64t:26023] [NO-NAME] ORTE_ERROR_LOG: Not found in file
>>> runtime/orte_system_init.c at line 42
>>> [em64t:26023] [NO-NAME] ORTE_ERROR_LOG: Not found in file
>>> runtime/orte_init.c at line 49
>>> --------------------------------------------------------------------
>>> --
>>> ----
>>> Open RTE was unable to initialize properly. The error occured while
>>> attempting to orte_init(). Returned value -13 instead of
>>> ORTE_SUCCESS.
>>> --------------------------------------------------------------------
>>> --
>>> ----
>>>
>>>
>>> The man page implies that --prefix would work. I know my
>>> example is
>>> silly. The real case
>>> involves going over a cluster, but the above is the simpliest case I
>>> could create.
>>> From the man page:
>>>
>>> --prefix <dir>
>>> Prefix directory that will be used to set the
>>> PATH and
>>> LD_LIBRARY_PATH on the remote node before invoking
>>> Open MPI
>>> or the target process. See the "Remote Execution"
>>> section,
>>> below.
>>>
>>> BTW - I am trying to migrate from LAM to OpenMPI. The switch was
>>> very painless. Thanks for the good work.
>>>
>>> Regards,
>>> Joe
>>>
>>>
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>
>>>
>>
>>
>>
>>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users

-- 
Jeff Squyres
Server Virtualization Business Unit
Cisco Systems