Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] custom modules per job (PBS/OpenMPI/environment-modules)
From: Michael Sternberg (sternberg_at_[hidden])
Date: 2009-11-17 15:39:32


On Nov 17, 2009, at 10:17 , Michael Sternberg wrote:
> On Nov 17, 2009, at 9:10 , Ralph Castain wrote:
>> Not exactly. It completely depends on how Torque was setup - OMPI
>> isn't forwarding the environment. Torque is.
>
> I actually tried compiling OMPI with the tm interface a couple of
> versions back for both packages but ran into memory trouble, which
> is why I didn't pursue this. With torque-2.4.x out and OpenMPI
> getting close to 1.3.4 I'll try again.

Follow-up: I recompiled OpenMPI-1.3.2 "--with-tm" (from torque-2.3.6)
and, lo and behold, environment variables and modules now are passed
across nodes, which thus includes custom modules loaded in the job
file. Darn, that was an old hang-up!

The variables passed do include (unsurprisingly) $HOSTNAME, but I can
live with that:

        login4 $ qsub -l nodes=2:ppn=1 -I
        qsub: waiting for job 34717.mds01 to start
        qsub: job 34717.mds01 ready

        n102 $ mpirun hostname
        n102
        n091
        n102 $ mpirun env | grep HOSTNAME
        HOSTNAME=n102
        HOSTNAME=n102

Ralph, David - thank you for the pointers!

Michael