Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Problem with MPI_File_read() (2)
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2009-04-15 06:33:47


On Apr 15, 2009, at 5:06 AM, Jovana Knezevic wrote:

> Yes, sure, what you say makes sense. On the other hand, it seems I
> will have to "traditionaly"-open the input file for n times - each one
> for one process, since anyway all of my processes have to collect
> their data from it (each parsing it from the beginning to the end),
> don't you think so? I wanted to take an advantage of MPI to read (in
> each process) the data from one file... Or have I misunderstood
> something?
>

The idea behind MPI I/O is that it can be done in parallel. It
usually works best when you have an underlying parallel filesystem.
In such cases (typically paired with very large input data), you can
exploit the parallelism of the underlying filesystem to efficiently
get just the necessary data to each MPI process.

If you input data isn't that large, or if you don't have a parallel
filesystem (e.g., you're just using NFS), such schemes can actually be
less efficient / slower. It may even be better to have something like
MPI_COMM_WORLD rank 0 read in the entire file and MPI_BCAST /
MPI_SCATTER / etc. the relevant data to each MPI process as necessary.

It's up to you to decide which is best for your application; it really
depends on the requirements of what you are doing, your local setup,
etc.

-- 
Jeff Squyres
Cisco Systems