Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Open MPI task scheduler
From: Matthieu Brucher (matthieu.brucher_at_[hidden])
Date: 2010-06-20 15:00:06


2010/6/20 Jack Bryan <dtustudy68_at_[hidden]>:
> Hi, Matthieu:
> Thanks for your help.
> Most of your ideas show that what I want to do.
> My scheduler should be able to be called from any C++ program, which can
> put
> a list of tasks to the scheduler and then the scheduler distributes the
> tasks to other client nodes.
> It may work like in this way:
> while(still tasks available) {
> myScheduler.push(tasks);
> myScheduler.get(tasks results from client nodes);
> }

Exactly. In your case, you want only one server, so you must find a
system so that every task can be serialized in the same form. The
easiest way to do so is to serialize your parameter set as an XML
fragment and add the type of task as another field.

> My cluster has 400 nodes with Open MPI. The tasks should be transferred b y
> MPI protocol.

No, they should not ;) MPI can be used, but it is not the easiest way
to do so. You still have to serialize your ticket, and you have to use
some functions that are from MPI2 (so perhaps not as portable as MPI1
functions). Besides, it cannot be used from programs that do not know
of using MPI protocols.

> I am not familiar with  RPC Protocol.

RPC is not a protocol per se. SOAP is. RPC stands for Remote Procedure
Call. It is basically your scheduler that has several functions
clients can call:
- add tickets
- retrieve ticket
- ticket is done

> If I use Boost.ASIO and some Python/GCCXML script to generate the code, it
> can be
> called from C++ program on Open MPI cluster ?

Yes, SOAP is just an XML way of representing the fact that you call a
function on the server. You can use it with C++, Java, ... I use it
with Python to monitor how many tasks are remaining, for instance.

> I cannot find the skeletton on your blog.
> Would you please tell me where to find it ?

It's not complete as some of the work is property of my employer. This
is how I use GCCXML to generate the calling code:
http://matt.eifelle.com/2009/07/21/using-gccxml-to-automate-c-wrappers-creation/
You have some additional code to write, but this is the main idea.

> I really appreciate your help.

No sweat, I hope I can give you correct hints!

Matthieu

-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher