Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: [OMPI devel] New Romio for OpenMPI available in bitbucket
From: Pascal Deveze (Pascal.Deveze_at_[hidden])
Date: 2010-09-17 06:36:00


Hi all,

In charge of ticket 1888 (see at
https://svn.open-mpi.org/trac/ompi/ticket/1888) ,
I have put the resulting code in bitbucket at:
http://bitbucket.org/devezep/new-romio-for-openmpi/

The work in this repo consisted in refreshing ROMIO to a newer
version: the one from the very last MPICH2 release (mpich2-1.3b1).

Testing:
  1. runs fine except one minor error (see the explanation below) on
various FS.
  2. runs fine with Lustre, but:
     . had to add a small patch in romio/adio/ad_lustre_open.c
  3. see below how to efficiently run with Lustre.

You are invited to test and send comments

Enjoy !

Pascal

======== The minor error ===================
The test error.c fails because OpenMPI does not handle correctly the
"two level" error functions of ROMIO:
    error_code = MPIO_Err_create_code(MPI_SUCCESS, MPIR_ERR_RECOVERABLE,
                                        myname, __LINE__, MPI_ERR_ARG,
                                        "**iobaddisp", 0);
OpenMPI limits its view to MPI_ERR_ARG, but the real error is
"**iobaddisp".

========= How to test performances with Lustre ===================
1) Compile with Lustre ADIO driver. For this, add the flag
        --with-io-romio-flags="--with-file-system=ufs+nfs+lustre" to
your configure command.

2) Of course, you should have a Lustre file system mounted on all the
nodes you will run on.

3) Take an application like coll_perf.c (in the test directory). In this
application, change the
    three dimensions to 1000, that will create a file of 4 GB (big
files are required in order
    to reach good performances with Lustre).

4) Put the highest possible striping_factor in the hint. For this, one
solution is :
 - If your Lustre file system have 16 OST, create a hint file with the
following line:
    striping_factor 16
- Export the path to this file in the variable ROMIO_HINTS:
   export ROMIO_HINTS=my_directory/my_hints
 If you do not specify the striping_factor, Lustre will set the default
value (often 2 only).
 You can verify the striping_factor set by Lustre with the following
command:
     lfs getstripe <file> (look at the value of lmm_stripe_count)
  Note: The striping_factor is set once at file creation and cannot be
changed after.

5) Run your test, specifying a file located in the Lustre file system.