On 9/13/06, imran shaik <sk.imran_at_[hidden]> wrote:
> I need to run parallel jobs on a cluster typically of size 600 nodes and
> running SGE, but the programmers are good at perl but not C or C++. So i
> thought of MPI, but i dont know whether it has perl support?
SGE will dispatch process among the nodes of your cluster but it does
not support interprocess communication, which MPI does. If your
problem is easily splittable (like parse a large apache log, read a
large xml list of things) you might be able to split the data and
spawn as many process as you can.
I do it using LSF (another dispatcher) and a Makefile that controls
the dependencies and spawn the processes (using make's -j flag) and it
works quite well. But if your job need the communication (like
processing big matrices, collecting and distributing data among
processes etc) you'll need an interprocess communication and that's
what MPI is best at.
In a nutshell, you'll need the runtime environment to run MPI programs
as well as you need SGE's runtime environments on every node to
dispatch jobs and collect information.
About MPI bindings for Perl, there's this module:
but it's far too young to be trustworthy, IMHO, and you'll probably
need the MPI runtime on all nodes as well...