Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Performance: MPICH2 vs OpenMPI
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2008-10-08 10:51:23

On Oct 8, 2008, at 10:26 AM, Sangamesh B wrote:

> - What version of Open MPI are you using? Please send the
> information listed here:
> 1.2.7
> - Did you specify to use mpi_leave_pinned?
> No
> Use "--mca mpi_leave_pinned 1" on your mpirun command line (I don't
> know if leave pinned behavior benefits Gromacs or not, but it likely
> won't hurt)

I see from your other mail that you are not using IB. If you're only
using TCP, then mpi_leave_pinned will have little/no effect.

> - Did you enable processor affinity?
> No
> Use "--mca mpi_paffinity_alone 1" on your mpirun command line.
> Will use these options in the next benchmark
> - Are you sure that Open MPI didn't fall back to ethernet (and not
> use IB)? Use "--mca btl openib,self" on your mpirun command line.
> I'm using TCP. There is no infiniband support. But eventhough the
> results can be compared?

Yes, they should be comparable. We've always known that our TCP
support is "ok" but not "great" (truthfully: we've not tuned it nearly
as extensively as we've tuned our other transports). But such a huge
performance difference is surprising.

It this one 1 or more nodes? It might be useful to delineate between
TCP and shared memory performance difference. I believe that MPICH2's
shmem performance is likely to be better than OMPI v1.2's, but like
TCP, it shouldn't be *that* huge.

> - Have you tried compiling Open MPI with something other than GCC?
> No.
> Just this week, we've gotten some reports from an OMPI member that
> they are sometimes seeing *huge* performance differences with OMPI
> compiled with GCC vs. any other compiler (Intel, PGI, Pathscale).
> We are working to figure out why; no root cause has been identified
> yet.
> I'll try for other than gcc and comeback to you

That would be most useful; thanks.

Jeff Squyres
Cisco Systems