On Oct 8, 2008, at 10:26 AM, Sangamesh B wrote:
> - What version of Open MPI are you using? Please send the
> information listed here:
> - Did you specify to use mpi_leave_pinned?
> Use "--mca mpi_leave_pinned 1" on your mpirun command line (I don't
> know if leave pinned behavior benefits Gromacs or not, but it likely
> won't hurt)
I see from your other mail that you are not using IB. If you're only
using TCP, then mpi_leave_pinned will have little/no effect.
> - Did you enable processor affinity?
> Use "--mca mpi_paffinity_alone 1" on your mpirun command line.
> Will use these options in the next benchmark
> - Are you sure that Open MPI didn't fall back to ethernet (and not
> use IB)? Use "--mca btl openib,self" on your mpirun command line.
> I'm using TCP. There is no infiniband support. But eventhough the
> results can be compared?
Yes, they should be comparable. We've always known that our TCP
support is "ok" but not "great" (truthfully: we've not tuned it nearly
as extensively as we've tuned our other transports). But such a huge
performance difference is surprising.
It this one 1 or more nodes? It might be useful to delineate between
TCP and shared memory performance difference. I believe that MPICH2's
shmem performance is likely to be better than OMPI v1.2's, but like
TCP, it shouldn't be *that* huge.
> - Have you tried compiling Open MPI with something other than GCC?
> Just this week, we've gotten some reports from an OMPI member that
> they are sometimes seeing *huge* performance differences with OMPI
> compiled with GCC vs. any other compiler (Intel, PGI, Pathscale).
> We are working to figure out why; no root cause has been identified
> I'll try for other than gcc and comeback to you
That would be most useful; thanks.