Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Bernd Schubert (bernd-schubert_at_[hidden])
Date: 2007-08-31 17:00:08


Robert Latham wrote:

> On Sun, Aug 26, 2007 at 06:44:18PM +0200, Bernd Schubert wrote:
>> I'm presently trying to add lustre support to open-mpi's romio using this
>> patch http://ft.ornl.gov/projects/io/src/adio-lustre-mpich2-v02.patch.
>>
>> It basically applies, only a few C files have been renamed in open-mpi,
>> but the autotools build system gives me headaches.
>
> Boy tell me about it (says the guy who's job it is to work on it).

I don't know much about it myself, I only read a discussion about it on the
lustre mailing list.

>
>> Fine, adding a similar entry for lustre is easy, but now we need to
>> define BUILD_XFS. So where does this come from?
>> There is romio/romio/Makefile.in
>
> I don't know where you got BUILD_XFS. In MPICH2, we don't use that
> symbol anywhere.

This was supposed to be BUILD_LUSTRE, taken as example from BUILD_XFS. I was
a bit overworked that weekend.

>
>> BUILD_UFS_FALSE = @BUILD_UFS_FALSE@
>> BUILD_UFS_TRUE = @BUILD_UFS_TRUE@
>> BUILD_XFS_FALSE = @BUILD_XFS_FALSE@
>> BUILD_XFS_TRUE = @BUILD_XFS_TRUE@
>> CC = @CC@
>
> These look like OpenMPI additions. Perhaps they can chime in.
>
> In MPICH2-land we build up FILE_SYS_DIRS in configure.in, and then do
> a "for dir in $FILE_SYS_DIRS", with each directory contributing some
> .o files to the io library. Hey, it works.

Yes, the patch from ft.ornl.gov also does it that way, well its for mpich2.

>
>> Not a single line about filesystems, *grumble*.
>
> Again speaking about MPICH2 land, all fileystem logic (such as it is)
> lives in configure.in. it's regrettably scattered amongst some
> architecture-specific sections.

To make it short, I could get everything to compile and to work (though more
or less a brute force way), but I'm still not pleased with the performance.
So far I didn't have the time yet to investigate the reason why the
performance is still low, though, I suspect openmpi/ofed problems, because
as long as I run the computations on a single system without
networking/infiniband the i/o performance is rather good.
I first need to figure out why writing to nfs-exported lustre is slow and
then I will care about MPI. Well, I already know the reason for the nfs
problem, now I need to write a proper patch. I guess I will start to work
on the MPI problem on Wednesday.

>
>> Now lets assume I eventually find the proper autotools files to
>> patch lustre support in, how can I distribute that patch?
>
> Distribute changes to the primary sources. i.e. fix configure.in or
> Makefile.am and let autotools regenerate the necessary files.
> ROMIO's got an AC_PREREQ in there, so things can't go horribly bad

So I need to write an explanation what to do after applying the patch? Who
actually reads documentation ;) ?

>
> (like in the old days when people would try to autoconf ROMIO's
> ancient autoconf-1.7 era configure with autoconf-2.13 -- you heard me.
> You think it's bad now? Why, back in '00 you were lucky if you ... ok
> you get the idea. i'll quit with the old man bit.)
>
>> In principle I would have to distribute a patch that also patches
>> the auto-generated configure, automake.in, etc files.
>
> As you understand already, that's untennable.

Well, the original patch does it that way.

>
>> Any plans for a sane build system?
>
> I'm not proud of how ROMIO's configure.in has evolved over the years.
> not proud at all. But I can gaurantee you that the alternatives are
> worse. Don't speak to me of scons and cmake!

I would have suggested cmake. Whats so bad about it?

>
> I hope the MPICH2 perspective on ROMIO's build system helps a bit, but
> I think the integration with OpenMPI may have changed things somewhat.
>
> Good luck

Thanks!

Cheers,
Bernd

PS: The patch is here:
http://www.pci.uni-heidelberg.de/tc/usr/bernd/downloads/lustre/openmpi/

Apply it and then run in ompi/mca/io/romio/romio/

aclocal
autoheader
autoconf
libtoolize --force
automake -a -c