Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Suggestions on multi-compiler/multi-mpi build?
From: Brock Palen (brockp_at_[hidden])
Date: 2007-11-15 13:09:25


Really modules is the only way to go, we use it to maintain no less
than 12 versions of openmpi compiled with pgi, gcc, nag, and intel.

Yes as the other reply said just set LD_LIBRARY_PATH you can use

ldd executable

to see which libraries the executable is going to use under the
current environment. We also have uses submit to torque with
#PBS -V

which copies the current environment with the job, this allows users
to submit and have multiple jobs running each with their own
environment. If your using PBS or torque look at the qsub man page.

Another addition we came up with was after we load a default set a
modules (pbs, a compiler , mpi library) we look for a file in

$HOME/privatemodules/default

which just has a list of module commands. This allows users to
change their default modules for every login.
For example i have in my default file

module load git

Because i use git for managing files.

We have done other hacks, our webpage of software is built out of
cron from the module files:
http://cac.engin.umich.edu/resources/systems/nyxV2/software.html

So we dont maintain software lists online, we just generate it
dynamically.
Modules is the admins best friend.

Brock Palen
Center for Advanced Computing
brockp_at_[hidden]
(734)936-1985

On Nov 15, 2007, at 8:54 AM, Jim Kusznir wrote:

> Hi all:
>
> I'm trying to set up a cluster for a group of users with very
> different needs. So far, it looks like I need gcc, pgi, and intel to
> work with openmpi and mpich, with each user able to control what
> combination they get. This is turning out to be much more difficult
> than I expected.
>
> Someone has pointed me to enviornment-modules ("Modules"), which looks
> like it will be a critical part of the solution. I even noticed that
> the provided openmpi.spec file has some direct support for modules,
> which makes me quite happy!
>
> However, I still have many questions about how to set things up.
>
> First, I get the impression that openmpi will need to be compiled with
> each compiler that will use it. If this is true, I'm not quite sure
> how to go about it. I could install in different directories for the
> user commands, but what about the libraries? I don't think I have a
> feesable way of selecting which library to use on the fly on the
> entire cluster for each user, so it seems like it would be better to
> have all the libraries available. In addition, I will need RPMs to
> deploy efficiently on the cluster. I suspect I can change the
> versioning info and build with each compiler, but at this point, I
> don't even know how to reliably select what compiler rpmbuild will use
> (I've only succeeded in using gcc).
>
> Finally, using modules, how do I set it up so that if a user changes
> compilers, but stays with openmpi, it will load the correct openmpi
> paths? I know I can set up the openmpi module file to load after the
> compiler module and based on that select different paths based on the
> currently-loaded compiler module. If the user changes the compiler
> module, will that cause the mpi module to also be reloaded so the new
> settings will be loaded? Or do I need this at all?
>
> Thanks for all your help!
> --Jim
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>