Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: [OMPI users] How could OpenMPI (or MVAPICH) affect floating-point results?
From: Blosch, Edwin L (edwin.l.blosch_at_[hidden])
Date: 2011-09-19 18:41:08


I am observing differences in floating-point results from an application program that appear to be related to whether I link with OpenMPI 1.4.3 or MVAPICH 1.2.0. Both packages were built with the same installation of Intel 11.1, as well as the application program; identical flags passed to the compiler in each case.

I've tracked down some differences in a compute-only routine where I've printed out the inputs to the routine (to 18 digits) ; the inputs are identical. The output numbers are different in the 16th place (perhaps a few in the 15th place). These differences only show up for optimized code, not for -O0.

My assumption is that some optimized math intrinsic is being replaced dynamically, but I do not know how to confirm this. Anyone have guidance to offer? Or similar experience?

Thanks very much

Ed

Just for what it's worth, here's the output of ldd:

% ldd application_mvapich
        linux-vdso.so.1 => (0x00007fffe3746000)
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00002b5b45fc1000)
        libmpich.so.1.0 => /usr/mpi/intel/mvapich-1.2.0/lib/shared/libmpich.so.1.0 (0x00002b5b462cd000)
        libibverbs.so.1 => /usr/lib64/libibverbs.so.1 (0x00002b5b465ed000)
        libibumad.so.3 => /usr/lib64/libibumad.so.3 (0x00002b5b467fc000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b5b46a04000)
        librt.so.1 => /lib64/librt.so.1 (0x00002b5b46c21000)
        libm.so.6 => /lib64/libm.so.6 (0x00002b5b46e2a000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00002b5b47081000)
        libc.so.6 => /lib64/libc.so.6 (0x00002b5b47285000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002b5b475e3000)
        /lib64/ld-linux-x86-64.so.2 (0x00002b5b45da0000)
        libimf.so => /opt/intel/Compiler/11.1/072/lib/intel64/libimf.so (0x00002b5b477fb000)
        libsvml.so => /opt/intel/Compiler/11.1/072/lib/intel64/libsvml.so (0x00002b5b47b8f000)
        libintlc.so.5 => /opt/intel/Compiler/11.1/072/lib/intel64/libintlc.so.5 (0x00002b5b47da5000)

% ldd application_openmpi
       linux-vdso.so.1 => (0x00007fff6ebff000)
        libstdc++.so.6 => /usr/lib64/libstdc++.so.6 (0x00002b6e7c17d000)
        libmpi_f90.so.0 => /usr/mpi/intel/openmpi-1.4.3/lib64/libmpi_f90.so.0 (0x00002b6e7c489000)
        libmpi_f77.so.0 => /usr/mpi/intel/openmpi-1.4.3/lib64/libmpi_f77.so.0 (0x00002b6e7c68d000)
        libmpi.so.0 => /usr/mpi/intel/openmpi-1.4.3/lib64/libmpi.so.0 (0x00002b6e7c8ca000)
        libopen-rte.so.0 => /usr/mpi/intel/openmpi-1.4.3/lib64/libopen-rte.so.0 (0x00002b6e7cb9c000)
        libopen-pal.so.0 => /usr/mpi/intel/openmpi-1.4.3/lib64/libopen-pal.so.0 (0x00002b6e7ce01000)
        libdl.so.2 => /lib64/libdl.so.2 (0x00002b6e7d077000)
        libnsl.so.1 => /lib64/libnsl.so.1 (0x00002b6e7d27c000)
        libutil.so.1 => /lib64/libutil.so.1 (0x00002b6e7d494000)
        libm.so.6 => /lib64/libm.so.6 (0x00002b6e7d697000)
        libpthread.so.0 => /lib64/libpthread.so.0 (0x00002b6e7d8ee000)
        libc.so.6 => /lib64/libc.so.6 (0x00002b6e7db0b000)
        libgcc_s.so.1 => /lib64/libgcc_s.so.1 (0x00002b6e7de69000)
        /lib64/ld-linux-x86-64.so.2 (0x00002b6e7bf5c000)
        libifport.so.5 => /opt/intel/Compiler/11.1/072/lib/intel64/libifport.so.5 (0x00002b6e7e081000)
        libifcoremt.so.5 => /opt/intel/Compiler/11.1/072/lib/intel64/libifcoremt.so.5 (0x00002b6e7e1ba000)
        libimf.so => /opt/intel/Compiler/11.1/072/lib/intel64/libimf.so (0x00002b6e7e45f000)
        libsvml.so => /opt/intel/Compiler/11.1/072/lib/intel64/libsvml.so (0x00002b6e7e7f4000)
        libintlc.so.5 => /opt/intel/Compiler/11.1/072/lib/intel64/libintlc.so.5 (0x00002b6e7ea0a000)