Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Thread as MPI process
From: Reuti (reuti_at_[hidden])
Date: 2010-09-21 05:24:30


Hi,

Am 21.09.2010 um 10:54 schrieb Mikael Lavoie:

> Sorry, but i get lost in what i wanna do, i have build a small home cluster with Pelican_HPC, that user openMPI, and i was trying to find a way to get a multithreaded program work in a multiprocess way without taking the time to learn MPI. And my vision was a sort of wrapper that take C posix app src code, and convert it from pthread to a multiprocessMPI app. But the problem is the remote memory access, that will only be implemented in MPI 3.0(for what i've read of it).
>
> So, after 12 hour of intensive reading about MPI and POSIX, the best way to deal with my problem(running a C pthreaded app in my cluster) is to convert the src in a SPMD way.
> I didn't mentionned that basicly, my prog open huge text file, take each string and process it through lot's of cryptographic iteration and then save the result in an output.out like file.
> So i will need to make the master process split the input file and then send them as input for the worker process.
>
> But if you or someone else know a kind of interpretor like program to run a multithreaded C program and convert it logically to a master/worker multiprocess MPI that will be sended by ssh to the interpreter on the worker side and then lunched.

what about taking a step back and use PVM? Of course you have no shared memory access, but it will allow you to transfer information between nodes and start worker processes:

http://www.netlib.org/pvm3/book/node17.html

Looks like PVM is no longer included in Pelican_HPC by default, but you can compile it on your own.

-- Reuti

> This is what i've tried to explain in the last msg. A dream for the hobyist that want to get the full power of a night-time cluster, without having to learn all the MPI syntax and structure.
>
> If it doesn't exist, this would be a really great tool i think.
>
> Thank you for your reply, but i think i have answered my question alone... No Pain, No Gain...
>
> On 20 Sep 2010, at 22:24, Mikael Lavoie wrote:
> > I wanna know if it exist a implementation that permit to run a single host process on the master of the cluster, that will then spawn 1 process per -np X defined thread at the host specified in the host list. The host will then act as a syncronized sender/collecter of the work done.
>
> I don't fully understand you explanation either but I may be able to help clear up what you are asking for:
>
> If you mean "pthreads" or "linux threads" then no, you cannot have different threads on different nodes under any programming paradigm.
>
> However if you mean "execution threads" or in MPI parlance "ranks" then yes, under OpenMPI each "rank" will be a separate process on one of the nodes in the host list, as Jody says look at MPI_Comm_Spawn for this.
>
> Ashley,
>
> --
>
> Ashley Pittman, Bath, UK.
>
> Padb - A parallel job inspection tool for cluster computing
> http://padb.pittman.org.uk
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users