I am new to this list and to parallel programming in general. I am
writing a trading simulator for the forex market and I am using
genetic algorithms to breed trading parameters.
I am using PGAPack as a GA library, and it uses MPI to parallelize
optimization runs. This is how I got to Open MPI.
I am stuck at some point mainly because my lack of parallel
programming knowledge. What I'd like to achieve is: (I am doing it the
serial way right now)
- Load price data from files, and compute a few tables (right now this
takes up ~4 GB of memory)
-- Create new offsprings in the master process for the GA
-- Evaluate them in parallel (on 4 local CPUs but maybe more on LAN if
i need it)
- Until I get a satisfactory result.
My problem is, I'd like to share that 2 GB table (computed once at the
beginning, and is read-only after) between processes so I don't have
to use up 16 gigs of memory.
How do you share data between processes locally?
Later I will need to use other hosts too in the calculation. Will the
slaves on other hosts need to calculate their own tables on go on from
there and share them locally, or can I share these tables on the
master host with them?
How do you usually solve these kinds of problems?
Can you point me to some docs or keywords what should I learn about?
Thank you very much.