Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI users] IO performance
From: John Hearns (hearnsj_at_[hidden])
Date: 2012-02-04 03:28:06


On 03/02/2012, Tom Rosmond <rosmond_at_[hidden]> wrote:
> Recently the organization I work for bought a modest sized Linux cluster
> for running large atmospheric data assimilation systems. In my
> experience a glaring problem with systems of this kind is poor IO
> performance. Typically they have 2 types of network: 1) A high speed,
> low latency, e.g. Infiniband, network dedicated to MPI communications,
> and, 2) A lower speed network, e.g 1Gb or 10Gb ethernet, for IO. On
> clusters this second network is usually the basis for a global parallel
> file system (GPFS), through which nearly all IO traffic must pass.

Not necessarily.
You can use the Infiniband network for storage very easily, and this
is commonly done.
Indeed, you can increase your bandwidth for storage by using the two
ports of the Infiniband adapters, allocating even and odd compute
nodes to different storage subnets.

All you need do is have an Infiniband card (or cards) in your storage server,
or to put in dedicated storage routers (such as Panasas do).

GPFS is not the only kid on the block. There are other filesystems out
there - such as Panasas and Lustre.
And indeed what is wrong with having a properly balanced RAID array
and an NFS server with an Infiniband interface (or interfaces).

The 'glaring problem' is not inherent in clusters.