Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Suggestions on multi-compiler/multi-mpi build?
From: Jim Kusznir (jkusznir_at_[hidden])
Date: 2007-11-27 11:46:10

So another question on this topic:

I'm running on a 64-bit cluster. Is it important/needed/useful to
maintain 32-bit and 64-bit versions of openmpi and such? At the
moment, I'm using Rocks' default openmpi, which includes both 32 and
64-bit versions of the libraries. Yet, so far, my attempt to build a
32-bit on the 64-bit system has proven very difficult, and I finally
resolved to just install a 32-bit build host to build the 32-bit
versions if required.


Oh, and I'd be very grateful for any samples and automation scripts to
help with the whole setup. I'm pretty new to "real cluster
management" :)

Thanks again!


On 11/15/07, Brock Palen <brockp_at_[hidden]> wrote:
> Really modules is the only way to go, we use it to maintain no less
> than 12 versions of openmpi compiled with pgi, gcc, nag, and intel.
> Yes as the other reply said just set LD_LIBRARY_PATH you can use
> ldd executable
> to see which libraries the executable is going to use under the
> current environment. We also have uses submit to torque with
> #PBS -V
> which copies the current environment with the job, this allows users
> to submit and have multiple jobs running each with their own
> environment. If your using PBS or torque look at the qsub man page.
> Another addition we came up with was after we load a default set a
> modules (pbs, a compiler , mpi library) we look for a file in
> $HOME/privatemodules/default
> which just has a list of module commands. This allows users to
> change their default modules for every login.
> For example i have in my default file
> module load git
> Because i use git for managing files.
> We have done other hacks, our webpage of software is built out of
> cron from the module files:
> So we dont maintain software lists online, we just generate it
> dynamically.
> Modules is the admins best friend.
> Brock Palen
> Center for Advanced Computing
> brockp_at_[hidden]
> (734)936-1985
> On Nov 15, 2007, at 8:54 AM, Jim Kusznir wrote:
> > Hi all:
> >
> > I'm trying to set up a cluster for a group of users with very
> > different needs. So far, it looks like I need gcc, pgi, and intel to
> > work with openmpi and mpich, with each user able to control what
> > combination they get. This is turning out to be much more difficult
> > than I expected.
> >
> > Someone has pointed me to enviornment-modules ("Modules"), which looks
> > like it will be a critical part of the solution. I even noticed that
> > the provided openmpi.spec file has some direct support for modules,
> > which makes me quite happy!
> >
> > However, I still have many questions about how to set things up.
> >
> > First, I get the impression that openmpi will need to be compiled with
> > each compiler that will use it. If this is true, I'm not quite sure
> > how to go about it. I could install in different directories for the
> > user commands, but what about the libraries? I don't think I have a
> > feesable way of selecting which library to use on the fly on the
> > entire cluster for each user, so it seems like it would be better to
> > have all the libraries available. In addition, I will need RPMs to
> > deploy efficiently on the cluster. I suspect I can change the
> > versioning info and build with each compiler, but at this point, I
> > don't even know how to reliably select what compiler rpmbuild will use
> > (I've only succeeded in using gcc).
> >
> > Finally, using modules, how do I set it up so that if a user changes
> > compilers, but stays with openmpi, it will load the correct openmpi
> > paths? I know I can set up the openmpi module file to load after the
> > compiler module and based on that select different paths based on the
> > currently-loaded compiler module. If the user changes the compiler
> > module, will that cause the mpi module to also be reloaded so the new
> > settings will be loaded? Or do I need this at all?
> >
> > Thanks for all your help!
> > --Jim
> > _______________________________________________
> > users mailing list
> > users_at_[hidden]
> >
> >
> >
> _______________________________________________
> users mailing list
> users_at_[hidden]