Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Hybrid MPI/Pthreads program behaves differently on two different machines with same hardware
From: Ralph Castain (rhc_at_[hidden])
Date: 2011-10-24 09:14:21


Does the difference persist if you run the single process using mpirun? In other words, does "mpirun -np 1 ./my_hybrid_app..." behave the same as "mpirun -np 2 ./..."?

There is a slight difference in the way procs start when run as singletons. It shouldn't make a difference here, but worth testing.

On Oct 24, 2011, at 12:37 AM, 吕慧伟 wrote:

> Dear List,
>
> I have a hybrid MPI/Pthreads program named "my_hybrid_app", this program is memory-intensive and take advantage of multi-threading to improve memory throughput. I run "my_hybrid_app" on two machines, which have same hardware configuration but different OS and GCC. The problem is: when I run "my_hybrid_app" with one process, two machines behaves the same: the more number of threads, the better the performance; however, when I run "my_hybrid_app" with two or more processes. The first machine still increase performance with more threads, the second machine degrades in performance with more threads.
>
> Since running "my_hybrid_app" with one process behaves correctly, I suspect my linking to MPI library has some problem. Would somebody point me in the right direction? Thanks in advance.
>
> Attached are the commandline used, my machine informantion and link informantion.
> p.s. 1: Commandline
> single process: ./my_hybrid_app <number of threads>
> multiple process: mpirun -np 2 ./my_hybrid_app <number of threads>
>
> p.s. 2: Machine Informantion
> The first machine is CentOS 5.3 with GCC 4.1.2:
> Target: x86_64-redhat-linux
> Configured with: ../configure --prefix=/usr --mandir=/usr/share/man --infodir=/usr/share/info --enable-shared --enable-threads=posix --enable-checking=release --with-system-zlib --enable-__cxa_atexit --disable-libunwind-exceptions --enable-libgcj-multifile --enable-languages=c,c++,objc,obj-c++,java,fortran,ada --enable-java-awt=gtk --disable-dssi --enable-plugin --with-java-home=/usr/lib/jvm/java-1.4.2-gcj-1.4.2.0/jre --with-cpu=generic --host=x86_64-redhat-linux
> Thread model: posix
> gcc version 4.1.2 20080704 (Red Hat 4.1.2-44)
> The second machine is SUSE Enterprise Server 11 with GCC 4.3.4:
> Target: x86_64-suse-linux
> Configured with: ../configure --prefix=/usr --infodir=/usr/share/info --mandir=/usr/share/man --libdir=/usr/lib64 --libexecdir=/usr/lib64 --enable-languages=c,c++,objc,fortran,obj-c++,java,ada --enable-checking=release --with-gxx-include-dir=/usr/include/c++/4.3 --enable-ssp --disable-libssp --with-bugurl=http://bugs.opensuse.org/ --with-pkgversion='SUSE Linux' --disable-libgcj --disable-libmudflap --with-slibdir=/lib64 --with-system-zlib --enable-__cxa_atexit --enable-libstdcxx-allocator=new --disable-libstdcxx-pch --enable-version-specific-runtime-libs --program-suffix=-4.3 --enable-linux-futex --without-system-libunwind --with-cpu=generic --build=x86_64-suse-linux
> Thread model: posix
> gcc version 4.3.4 [gcc-4_3-branch revision 152973] (SUSE Linux)
>
> p.s. 3: ldd Informantion
> The first machine:
> $ ldd my_hybrid_app
> libm.so.6 => /lib64/libm.so.6 (0x000000358d400000)
> libmpi.so.0 => /usr/local/openmpi/lib/libmpi.so.0 (0x00002af0d53a7000)
> libopen-rte.so.0 => /usr/local/openmpi/lib/libopen-rte.so.0 (0x00002af0d564a000)
> libopen-pal.so.0 => /usr/local/openmpi/lib/libopen-pal.so.0 (0x00002af0d5895000)
> libdl.so.2 => /lib64/libdl.so.2 (0x000000358d000000)
> libnsl.so.1 => /lib64/libnsl.so.1 (0x000000358f000000)
> libutil.so.1 => /lib64/libutil.so.1 (0x000000359a600000)
> libgomp.so.1 => /usr/lib64/libgomp.so.1 (0x00002af0d5b07000)
> libpthread.so.0 => /lib64/libpthread.so.0 (0x000000358d800000)
> libc.so.6 => /lib64/libc.so.6 (0x000000358cc00000)
> /lib64/ld-linux-x86-64.so.2 (0x000000358c800000)
> librt.so.1 => /lib64/librt.so.1 (0x000000358dc00000)
> The second machine:
> $ ldd my_hybrid_app
> linux-vdso.so.1 => (0x00007fff3eb5f000)
> libmpi.so.0 => /root/opt/openmpi/lib/libmpi.so.0 (0x00007f68627a1000)
> libm.so.6 => /lib64/libm.so.6 (0x00007f686254b000)
> libopen-rte.so.0 => /root/opt/openmpi/lib/libopen-rte.so.0 (0x00007f68622fc000)
> libopen-pal.so.0 => /root/opt/openmpi/lib/libopen-pal.so.0 (0x00007f68620a5000)
> libdl.so.2 => /lib64/libdl.so.2 (0x00007f6861ea1000)
> libnsl.so.1 => /lib64/libnsl.so.1 (0x00007f6861c89000)
> libutil.so.1 => /lib64/libutil.so.1 (0x00007f6861a86000)
> libgomp.so.1 => /usr/lib64/libgomp.so.1 (0x00007f686187d000)
> libpthread.so.0 => /lib64/libpthread.so.0 (0x00007f6861660000)
> libc.so.6 => /lib64/libc.so.6 (0x00007f6861302000)
> /lib64/ld-linux-x86-64.so.2 (0x00007f6862a58000)
> librt.so.1 => /lib64/librt.so.1 (0x00007f68610f9000)
> I installed openmpi-1.4.2 to a user directory /root/opt/openmpi and use "-L/root/opt/openmpi -Wl,-rpath,/root/opt/openmpi" when linking.
> --
> Huiwei Lv
> PhD. student at Institute of Computing Technology,
> Beijing, China
> http://asg.ict.ac.cn/lhw
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users