Performance benchmarks are always problematic and a source of contention -
it is very hard to obtain a meaningful comparison between MPI
implementations without taking great care that they are being wholly
optimized for the testbed environment. As you can imagine, all the MPIs
watch each other pretty carefully, and so results showing large performance
differences usually mean that someone didn't optimize one of the MPIs under
examination. OMPI can be particularly difficult given its wide range of
controls, many of which are intended to support academic research as
opposed to general users.
The OMPI developers don't generally spend much time on benchmarks as we
have found them to be poor predictors of actual application performance.
However, we recognize that others may wish to use them, and that users can
definitely benefit from tuning their applications to their environment.
There is an FAQ section dedicated to tuning that focuses on getting
improved performance for your application, but we have now added an
additional section specifically addressing benchmarks to the web site:
The intent of this section is mostly to provide some baseline benchmarks
between some of the MPIs we typically track on our systems, and to provide
the configuration and runtime parameter settings used to "tune" OMPI for
that testbed environment. Hopefully, this will help both users and those
performing benchmarks in their work.
We'll be adding data to this section periodically to track the relative
performance of the OMPI releases as we roll forward. Feedback on the
usefulness of the data, and/or questions about benchmarking OMPI
performance, are of course welcome.