Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] sharing memory between processes
From: Shaun Jackman (sjackman_at_[hidden])
Date: 2009-04-28 15:11:15


For what it's worth, the genome assembly software ABySS uses exactly
this system that Jody is describing to represent a directed graph.
http://www.bcgsc.ca/platform/bioinfo/software/abyss

Cheers,
Shaun

jody wrote:
> Hi Barnabas
>
> As far as i know, Open-MPI is not a shared memory system.
>
> Using Open-MPI to attack your problem on N processors, i would poceed
> as follows:
> - processor 0 reads the table and then splits it into N parts
> (Table_0,...Table_N)
> - processor 0 sends Table_i to processor i (for all i > 0) using MPI functions
> - if processor k needs info from Table_j, processor k sends a request
> to processor j
> - processor j sends requested information to processor k
>
> Of course you'd need some scheme to find out on which Table_i a particular
> entry can be found.
>
> This solution will work as well if your processors sit on several hosts.
>
> Jody
>
> On Tue, Apr 28, 2009 at 3:28 PM, Barnabas Debreczeni <keo_at_[hidden]> wrote:
>> Hi!
>>
>> I am new to this list and to parallel programming in general. I am
>> writing a trading simulator for the forex market and I am using
>> genetic algorithms to breed trading parameters.
>>
>> I am using PGAPack as a GA library, and it uses MPI to parallelize
>> optimization runs. This is how I got to Open MPI.
>>
>> I am stuck at some point mainly because my lack of parallel
>> programming knowledge. What I'd like to achieve is: (I am doing it the
>> serial way right now)
>>
>> - Load price data from files, and compute a few tables (right now this
>> takes up ~4 GB of memory)
>> - Repeat...
>> -- Create new offsprings in the master process for the GA
>> -- Evaluate them in parallel (on 4 local CPUs but maybe more on LAN if
>> i need it)
>> - Until I get a satisfactory result.
>>
>>
>> My problem is, I'd like to share that 2 GB table (computed once at the
>> beginning, and is read-only after) between processes so I don't have
>> to use up 16 gigs of memory.
>>
>> How do you share data between processes locally?
>>
>> Later I will need to use other hosts too in the calculation. Will the
>> slaves on other hosts need to calculate their own tables on go on from
>> there and share them locally, or can I share these tables on the
>> master host with them?
>>
>> How do you usually solve these kinds of problems?
>>
>> Can you point me to some docs or keywords what should I learn about?
>>
>> Thank you very much.
>>
>> Regards,
>>
>> Barnabas
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>