Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Open MPI task scheduler
From: Matthieu Brucher (matthieu.brucher_at_[hidden])
Date: 2010-06-20 14:13:14


Hi Jack,

What you are seeking is the client/server pattern. Have one node act
as a server. It will create a list of tasks or even a graph of tasks
if you have dependencies, and then create clients that will connect to
the server with an RPC protocol (I've done this with a SOAP+TCP
protocol, the severance of the TCP connection meaning that the client
is dead and that its task should be recycled, ités easy to do with
Boost.ASIO and some Python/GCCXML scripts to automatically generate
your code, I've written a skeletton on my blog). You may even have
clients with different sizes or capabilities and tell the server what
each client can do, and then the server may dispatch appropriate
tickets to the clients.

Each client and server can be a MPI process, you don't have to create
all clients inside one MPI process (you may use several if the
smallest resource your batch scheduler allocates is bigger that one of
your tasks). With a batch scheduler, it's better to allocate your
tasks as small as possible so that you can balance the resources you
need.

Matthieu

2010/6/20 Jack Bryan <dtustudy68_at_[hidden]>:
> Hi, all:
> I need to design a task scheduler (not PBS job scheduler) on Open MPI
> cluster.
> I need to parallelize an algorithm so that a big problem is decomposed into
> small tasks, which can be distributed
> to other worker nodes by the Scheduler and after being solved, the results
> of these tasks are returned to the manager node with the Scheduler, which
> will distribute more tasks on the base of the collected results.
> I need to use C++ to design the scheduler.
> I have searched online and I cannot find any scheduler available
> for this purpose.
> Any help is appreciated.
> thanks
> Jack
> June 19  2010
> ________________________________
> Hotmail has tools for the New Busy. Search, chat and e-mail from your inbox.
> Learn more.
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>

-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher