Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Suggestions on multi-compiler/multi-mpi build?
From: Katherine Holcomb (kholcomb_at_[hidden])
Date: 2007-12-03 08:42:36


On Fri, 2007-11-30 at 19:05 -0800, Jim Kusznir wrote:
> Thank you for your response!
>
> Just to clarify some things for my understanding:
>
> Do users load a single module that specifies both compiler and mpi
> version (as opposed to loading two different modules, one for
> complier, and one for mpi)?

We combine compiler and MPI because it's necessary that the
mpifcc/cxx/f90 be compatible with the compiler, so the correct compiler
needs to be in the user's path, and it's just easier on the users to
have them load a single module. A module can load another module, so if
you create a module for, say, the PGI compilers, the appropriate OpenMPI
module can load that module itself.

Here's an example, my ompi-pgi script (note: all my scripts were cribbed
from old ROCKS scripts; ROCKS doesn't include modules by default anymore
but they used to provide it):

#%
Module3.0#####################################################################
##
## CRAYfw mpich
##
proc ModulesHelp { } {
        global version MPIhome

        puts stderr "\tCRAY freeware: OpenMPI\n"
        puts stderr "\tThis module loads the lastest versions of OpenMPI.\n"
        puts stderr "\n\tThis adds $OpenMPI/* to several of the"
        puts stderr "\tenvironment variables."
        puts stderr "\n\tVersion $version\n"
}

module-whatis "loads the OpenMPI environment for PGI over Ethernet"

# for Tcl script use only
set version 1.2
set MPIhome /opt/openmpi-1.2/ethernet/pgi

prepend-path PATH $MPIhome/bin
prepend-path INCLUDE $MPIhome/include
prepend-path INFOPATH $MPIhome/share
prepend-path LD_LIBRARY_PATH $MPIhome/lib

setenv OMPI_MPIBIN $MPIhome
setenv OMPI_MPIHOME $MPIhome
setenv OMPI_MPIF77 "pgf77"
setenv OMPI_MPIF90 "pgf90"
setenv OMPI_MPICC "pgcc"
setenv OMPI_MPICXX "pgCC"

if [ module-info mode load ] {

    if {! [ is-loaded pgi ] } { module load pgi }

    if { [ is-loaded ompi-pgi ] } {
        puts stderr "\tModule ompi-pgi already loaded\n"
    }
    if { [ is-loaded ompi-gnu ] } {
        puts stderr "\tModule ompi-gnu superceded\n"
    }
    if { [ is-loaded ompi-intel ] } {
        puts stderr "\tModule ompi-intel superceded\n"
    }
    if { [ is-loaded mpich-eth-pgi ] } {
        puts stderr "\tModule mpich-eth-pgi environment superceded.\n"
    }

    if { [ is-loaded mpich-eth-gnu ] } {
        puts stderr "\tModule mpich-eth-gnu environment superceded.\n"
    }
    if { [ is-loaded mpich-eth-intel ] } {
        puts stderr "\tModule mpich-eth-intel environment superceded.\n"
    }
}

>
> So, if the MPI implementations need to be built separately for each
> compiler, what about other software packages that users' link against?
> For example, as far as I know, there are not seperate versions of
> libc for each compiler (but I may be ignorant here....please correct
> me if I'm wrong). How about other packages that code #includes? How
> do I know when something needs to be compiled separately vs one
> version for everyone?

We build a distinct version at least for every library that has F90/95
bindings. Probably a good idea for C++ as well.

>
> Another thing that I'm still not quite clear on: Can I have all
> versions of the various MPI libraries in the ld path at once (so the
> user doesn't have to "select" the correct one somehow prior to code
> execution)?

I'll let somebody who is more of a real system administrator answer
that.

>
> Are there any tricks to building/installing all this stuff in a fairly
> automated manner?
>

We use ROCKS. You have to create an rpm for everything, but once you do
then propagating it is automated.

-- 
Katherine Holcomb, Ph.D.                kholcomb_at_[hidden]
Research Computing Support Group - ITC  Office Phone: (434) 982-5948
I046 BSEL, Clark Hall                    Center Phone: (434) 243-8799
University of Virginia 22904