I'm compiling source units and linking them into an executable. Or perhaps you are talking about how OpenMPI itself is built? Excuse my ignorance...
The source code units are compiled like this:
/usr/mpi/intel/openmpi-1.4.3/bin/mpif90 -D_GNU_SOURCE -traceback -align -pad -xHost -falign-functions -fpconstant -O2 -I. -I/usr/mpi/intel/openmpi-1.4.3/include -c ../code/src/main/main.f90
The link step is like this:
/usr/mpi/intel/openmpi-1.4.3/bin/mpif90 -D_GNU_SOURCE -traceback -align -pad -xHost -falign-functions -fpconstant -static-intel -o ../bin/<name> <some libraries> -lstdc++
OpenMPI itself was configured like this:
./configure --prefix=/release/cfd/openmpi-intel --without-tm --without-sge --without-lsf --without-psm --without-portals --without-gm --without-elan --without-mx --without-slurm --without-loadleveler --enable-mpirun-prefix-by-default --enable-contrib-no-build=vt --enable-mca-no-build=maffinity --disable-per-user-config-files --disable-io-romio --with-mpi-f90-size=small --enable-static --disable-shared CXX=/appserv/intel/Compiler/11.1/072/bin/intel64/icpc CC=/appserv/intel/Compiler/11.1/072/bin/intel64/icc 'CFLAGS= -O2' 'CXXFLAGS= -O2' F77=/appserv/intel/Compiler/11.1/072/bin/intel64/ifort 'FFLAGS=-D_GNU_SOURCE -traceback -O2' FC=/appserv/intel/Compiler/11.1/072/bin/intel64/ifort 'FCFLAGS=-D_GNU_SOURCE -traceback -O2' 'LDFLAGS= -static-intel'
ldd output on the final executable gives:
linux-vdso.so.1 => (0x00007fffb77e7000)
libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00002b2e2b652000)
libibverbs.so.1 => /usr/lib64/libibverbs.so.1 (0x00002b2e2b95e000)
libdl.so.2 => /lib64/libdl.so.2 (0x00002b2e2bb6d000)
libnsl.so.1 => /lib64/libnsl.so.1 (0x00002b2e2bd72000)
libutil.so.1 => /lib64/libutil.so.1 (0x00002b2e2bf8a000)
libm.so.6 => /lib64/libm.so.6 (0x00002b2e2c18d000)
libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b2e2c3e4000)
libc.so.6 => /lib64/libc.so.6 (0x00002b2e2c600000)
libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002b2e2c959000)
Do you see anything that suggests I should have been compiling the application and/or OpenMPI with -fPIC?
From: users-bounces_at_[hidden] [mailto:users-bounces_at_[hidden]] On Behalf Of Tim Prince
Sent: Wednesday, September 21, 2011 10:53 AM
Subject: EXTERNAL: Re: [OMPI users] Question about compilng with fPIC
On 9/21/2011 11:44 AM, Blosch, Edwin L wrote:
> Follow-up to a mislabeled thread: "How could OpenMPI (or MVAPICH) affect floating-point results?"
> I have found a solution to my problem, but I would like to understand the underlying issue better.
> To rehash: An Intel-compiled executable linked with MVAPICH runs fine; linked with OpenMPI fails. The earliest symptom I could see was some strange difference in numerical values of quantities that should be unaffected by MPI calls. Tim's advice guided me to assume memory corruption. Eugene's advice guided me to explore the detailed differences in compilation.
> I observed that the MVAPICH mpif90 wrapper adds -fPIC.
> I tried adding -fPIC and -mcmodel=medium to the compilation of the OpenMPI-linked executable. Now it works fine. I haven't tried without -mcmodel=medium, but my guess is -fPIC did the trick.
> Does anyone know why compiling with -fPIC has helped? Does it suggest an application problem or an OpenMPI problem?
> To note: This is an Infiniband-based cluster. The application does pretty basic MPI-1 operations: send, recv, bcast, reduce, allreduce, gather, gather, isend, irecv, waitall. There is one task that uses iprobe with MPI_ANY_TAG, but this task is only involved in certain cases (including this one). Conversely, cases that do not call iprobe have not yet been observed to crash. I am deducing that this function is the problem.
If you are making a .so, the included .o files should be built with
-fPIC or similar. Ideally, the configure and build tools would enforce this.
users mailing list