Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] OpenMPI-ROMIO-OrangeFS
From: Edgar Gabriel (gabriel_at_[hidden])
Date: 2014-02-26 16:27:05


ok, then this must be a difference between OrangeFS and PVFs2. It turns
out that trunk and 1.7 does actually have the patch, but 1.6 series does
not have it. The actual commit was done in

https://svn.open-mpi.org/trac/ompi/changeset/24768

and based on the line numbers, I think it should apply cleanly.
Using that patch, 1.6 compiles and executes without complaining about
PVFS2 support, but running my testsuite brings up a ton of data errors.

I am still digging a bit into that, but that's where I am right now.

Thanks
Edgar

On 2/26/2014 3:17 PM, vithanousek wrote:
> At first Thank you very much for your time.
>
> "--with-file-system=pvfs2+ufs+nfs" didnt help.
>
> But if find (by google) some part of orangefs test. I dont know what is
> this exactly doing, but when I edited source code of OpenMPI like doing
> this line, all seems that it is working now. (changing
> ADIOI_PVFS2_IReadContig and ADIOI_PVFS2_IWriteContig to NULL in file
> ad_pvfs2.c)
>
> http://www.orangefs.org/trac/orangefs/browser/branches/OFSTest-dev/OFSTest/OFSTestNode.py?rev=10645#L1328
>
> Other tests I will do tomorrow.
>
> Thanks
> Hanousek Vít
>
>
>
> ---------- Původní zpráva ----------
> Od: Edgar Gabriel <gabriel_at_[hidden]>
> Komu: users_at_[hidden]
> Datum: 26. 2. 2014 21:08:07
> Předmět: Re: [OMPI users] OpenMPI-ROMIO-OrangeFS
>
>
> not sure whether its the problem or not, but usually have an additional
> flag set :
>
> --with-io-romio-flags="--with-file-system=pvfs2+ufs+nfs
> --with-pvfs2=/opt/pvfs-2.8.2"
>
> compilation is a bit slow for me today...
>
> Edgar
>
>
> On 2/26/2014 2:05 PM, vithanousek wrote:
> > Now I compiled by doing this:
> > OrangeFS (original, withou editing):
> >
> > ./configure --prefix=/usr/local/orangefs
> > --with-kernel=/usr/src/kernels/2.6.32-431.5.1.el6.x86_64
> > --with-openib=/usr --without-bmi-tcp --enable-shared
> > make
> > make kmod
> > make install
> > make kmod_install
> >
> > Without error.
> > OpenMPI (with edited switch to ifs):
> >
> > ./configure --prefix=/usr/local/openmpi_1.6.5_romio
> > --with-io-romio-flags='--with-pvfs2=/usr/local/orangefs'
> > make
> > make install
> >
> > Without error.
> > parallel FS mount work. But I still cant use MPIIO.
> > I compiled simple MPIIO program and run it by this:
> >
> > mpicc -o mpiio mpiio.c
> > mpirun -np 1 -host node18 mpiio
> > [node18:02334] mca: base: component_find: unable to open
> > /usr/local/openmpi_1.6.5_romio/lib/openmpi/mca_io_romio:
> > /usr/local/openmpi_1.6.5_romio/lib/openmpi/mca_io_romio.so: undefined
> > symbol: ADIOI_PVFS2_IReadContig (ignored)
> >
> > And no file is created.
> > I tried compile it with:
> > mpicc -o mpiio mpiio.c -lpvfs2 -L/usr/local/orangefs/lib
> >
> > but i got the same results, have You any idea?
> >
> > Thank for reply
> > Hanousek Vít
> >
> >
> >
> >
> >
> > ---------- Původní zpráva ----------
> > Od: vithanousek <vithanousek_at_[hidden]>
> > Komu: Open MPI Users <users_at_[hidden]>
> > Datum: 26. 2. 2014 20:30:17
> > Předmět: Re: [OMPI users] OpenMPI-ROMIO-OrangeFS
> >
> >
> > Thanks for your Time,
> >
> > I'm little bit confused, what is diferent between pvfs2 and
> > orangefs. I was thinking, that only project changes name.
> >
> > I get hint from OrangeFS maillist, to compile OrangeFs with
> > --enable-shared. This produce a some shared library (.so) in
> > /usr/local/orangefs/lib and I can compile OpenMPI 1.6.5 now (with
> > fixed "switch =>ifs" in ROMIO).
> >
> > I will test if it is working in next hour (some configuration steps
> > is needed).
> >
> > Thanks.
> > Hanousek Vít
> >
> > ---------- Původní zpráva ----------
> > Od: Edgar Gabriel <gabriel_at_[hidden]>
> > Komu: Open MPI Users <users_at_[hidden]>
> > Datum: 26. 2. 2014 20:18:03
> > Předmět: Re: [OMPI users] OpenMPI-ROMIO-OrangeFS
> >
> >
> > so we had ROMIO working with PVFS2 (not OrangeFS, which is however
> > registered as PVFS2 internally). We have one cluster which uses
> > OrangeFS, on that machine however we used OMPIO, not ROMIO. I am
> > currently compiling the 1.6 version of Open MPI to see whether I can
> > reproduce your problem.
> >
> > Thanks
> > Edgar
> >
> > On 2/26/2014 12:23 PM, vithanousek wrote:
> > > Thanks for reply,
> > >
> > > Is it possible that the patch solvs all this problems, not
> > only "switch
> > > => ifs" problem?
> > > I realy dont know, wher the problem is now (OpenMPI, ROMIO,
> > OrangeFS).
> > >
> > > Thanks
> > > Hanousek Vít
> > >
> > > ---------- Původní zpráva ----------
> > > Od: Ralph Castain <rhc_at_[hidden]>
> > > Komu: Open MPI Users <users_at_[hidden]>
> > > Datum: 26. 2. 2014 19:16:36
> > > Předmět: Re: [OMPI users] OpenMPI-ROMIO-OrangeFS
> > >
> > >
> > > Edgar hasn't had a chance to find the necessary patch - he was on
> > > travel, returning soon.
> > >
> > >
> > > On Feb 26, 2014, at 9:27 AM, vithanousek
> > <vithanousek_at_[hidden]> wrote:
> > >
> > > > Hello,
> > > >
> > > > I have still problems with compiling OpenMPI 1.6.5 with OrangeFS
> > > 2.8.7 support.
> > > >
> > > > I compiled OrangeFS by this:
> > > >
> > > > ./configure --prefix=/usr/local/orangefs2
> > > --with-kernel=/usr/src/kernels/2.6.32-431.5.1.el6.x86_64
> > > --with-openib=/usr --without-bmi-tcp
> > > > make -j 32
> > > > make -j 32 kmod
> > > > make install
> > > > make kmod_install
> > > >
> > > > this works.
> > > > than I tried to compile OpenMPI (with fixed convert_named
> > function
> > > in ad_pvfs2_io_dtype.c) by this:
> > > >
> > > > ./configure --prefix=/usr/local/openmpi_1.6.5_romio2
> > > --with-io-romio-flags='--with-pvfs2=/usr/local/orangefs2'
> > > > (...)
> > > > make -j32
> > > > (...)
> > > > CCLD mca_io_romio.la
> > > > /usr/bin/ld:
> > /usr/local/orangefs2/lib/libpvfs2.a(errno-mapping.o):
> > > relocation R_X86_64_32S against `PINT_errno_mapping' can not
> > be used
> > > when making a shared object; recompile with -fPIC
> > > > /usr/local/orangefs2/lib/libpvfs2.a: could not read symbols:
> > Bad value
> > > > collect2: ld returned 1 exit status
> > > > make[3]: *** [mca_io_romio.la] Error 1
> > > >
> > > > So I tried recompile OrangeFS by this:
> > > >
> > > > export CFLAGS="-fPIC"
> > > > ./configure --prefix=/usr/local/orangefs2
> > > --with-kernel=/usr/src/kernels/2.6.32-431.5.1.el6.x86_64
> > > --with-openib=/usr --without-bmi-tcp
> > > > make -j 32
> > > > make -j 32 kmod
> > > > make install
> > > > make kmod_install
> > > >
> > > > (there was errors with current->fsuid =>
> > current->cred->fsuid, in
> > > multiple files. I hardcoded this in files, bad idea I know )
> > > > Then compilation of OpenMPI works.
> > > >
> > > > ./configure --prefix=/usr/local/openmpi_1.6.5_romio2
> > > --with-io-romio-flags='--with-pvfs2=/usr/local/orangefs2'
> > > > make -j32
> > > > make install
> > > >
> > > > but when i created simple program which is using MPIIO, it
> > failed
> > > when i run it:
> > > >
> > > > mpirun -np 1 -host node18 mpiio
> > > > [node18:01696] mca: base: component_find: unable to open
> > > /usr/local/openmpi_1.6.5_romio/lib/openmpi/mca_io_romio:
> > > /usr/local/openmpi_1.6.5_romio/lib/openmpi/mca_io_romio.so:
> > > undefined symbol: ADIOI_PVFS2_IReadContig (ignored)
> > > >
> > > > Because I got message form OrangeFS mailing list about -fPIC
> > > errors, i tryed to recompile OrangeFS withou this flag and compile
> > > OpenMPI (static linked) by this:
> > > >
> > > > ./congure --prefix=/usr/local/openmpi_1.6.5_romio2
> > > --with-io-romio-flags='--with-pvfs2=/usr/local/orangefs2'
> > > --enable-static --disable-shared
> > > > (...)
> > > > make -j 32
> > > > (...)
> > > > CCLD otfmerge-mpi
> > > >
> > >
> >
> /root/openmpi-1.6.5/ompi/contrib/vt/vt/../../../.libs/libmpi.a(ad_pvfs2.o):(.data+0x60):
> > > undefined reference to `ADIOI_PVFS2_IReadContig'
> > > >
> > >
> >
> /root/openmpi-1.6.5/ompi/contrib/vt/vt/../../../.libs/libmpi.a(ad_pvfs2.o):(.data+0x68):
> > > undefined reference to `ADIOI_PVFS2_IWriteContig'
> > > > collect2: ld returned 1 exit status
> > > > make[10]: *** [otfmerge-mpi] Error 1
> > > > (...)
> > > >
> > > > Now I realy dont know, what is wrong.
> > > > Is there Anybody ho has OpenMPI working with OrangeFS?
> > > >
> > > > Thanks for replies
> > > > HanousekVít
> > > > _______________________________________________
> > > > users mailing list
> > > > users_at_[hidden]
> > > > http://www.open-mpi.org/mailman/listinfo.cgi/users
> > >
> > > _______________________________________________
> > > users mailing list
> > > users_at_[hidden]
> > > http://www.open-mpi.org/mailman/listinfo.cgi/users
> > >
> > >
> > >
> > > _______________________________________________
> > > users mailing list
> > > users_at_[hidden]
> > > http://www.open-mpi.org/mailman/listinfo.cgi/users
> > >
> >
> > --
> > Edgar Gabriel
> > Associate Professor
> > Parallel Software Technologies Lab http://pstl.cs.uh.edu
> > Department of Computer Science University of Houston
> > Philip G. Hoffman Hall, Room 524 Houston, TX-77204, USA
> > Tel: +1 (713) 743-3857 Fax: +1 (713)
> > 743-3335_______________________________________________
> > users mailing list
> > users_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
> >
> > _______________________________________________
> > users mailing list
> > users_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
> >
> >
> >
> > _______________________________________________
> > users mailing list
> > users_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
> >
>
> --
> Edgar Gabriel
> Associate Professor
> Parallel Software Technologies Lab http://pstl.cs.uh.edu
> Department of Computer Science University of Houston
> Philip G. Hoffman Hall, Room 524 Houston, TX-77204, USA
> Tel: +1 (713) 743-3857 Fax: +1 (713)
> 743-3335_______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>

-- 
Edgar Gabriel
Associate Professor
Parallel Software Technologies Lab      http://pstl.cs.uh.edu
Department of Computer Science          University of Houston
Philip G. Hoffman Hall, Room 524        Houston, TX-77204, USA
Tel: +1 (713) 743-3857                  Fax: +1 (713) 743-3335