Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

From: Glendenning, Lisa (lisa.glendenning_at_[hidden])
Date: 2007-07-11 18:47:27


Some supplementary information to the wiki at
https://svn.open-mpi.org/trac/ompi/wiki/CrayXT3.

I. Accessing the Open MPI source:

  * Subversion is installed on redstorm in /projects/unsupported/bin

  * Reddish has subversion in the default path (you don't have to load a
module)

  * The proxy information in the wiki is accurate, and works on both
redstorm and reddish

II. Building Open MPI on reddish:

  * 'module load PrgEnv-pgi-xc' to cross compile for redstorm

  * Reddish and redstorm do not have recent enough version of autotools,
so you must build your own (currently available in
/project/openmpi/rbbrigh/tools)

  * Suggested configuration: 'configure CC=qk-gcc CXX=qk-pgCC
F77=qk-pgf77 FC=qk-pgf90 --disable-mpi-profile --with-platform=redstorm
--host=x86_64-cray-linux-gnu --build=x86_64-unknown-linux-gnu
--disable-mpi-f90 --disable-mem-debug --disable-mem-profile
--disable-debug build_alias=x86_64-unknown-linux-gnu
host_alias=x86_64-cray-linux-gnu'

III. Building with Open MPI:

  * No executables will be installed in $PREFIX/bin, so must compile
without mpicc, e.g. 'qk-gcc -I$PREFIX/include *.c -L$PREFIX/lib -lmpi
-lopen-rte -lopen-pal'

  * When linking with libopen-pal, the following warning is normal: 'In
function `checkpoint_response': warning: mkfifo is not implemented and
will always fail'

IV. Running on Redstorm:

  * scp your executable from reddish to redstorm

  * Use 'qsub' to submit job and 'yod' to launch job (if you do an
interactive session, you can bypass PBS)

  * qsub expects project/task information - you can either provide this
with -A option or set it in $XT_ACCOUNT environmental variable

  * Sample job script for qsub with 8 nodes/processes and 10 minute
runtime:

    #!/bin/sh
    #PBS -lsize=8
    #PBS -lwalltime=10:00
    cd $PBS_O_WORKDIR
    yod a.out

  * Use 'xtshowmesh' and 'qstat' to query job status and cluster
configuration