> You sound like our vendors, "what is your app"
;-) I used to be one.
Ideally OMPI should do the switch between MXM/RC/XRC internally in the transport layer. Unfortunately,
we don't have such smart selection logic. Hopefully IB vendors will fix some day.
> Note most of our users run just fine with the standard Peer-Peer queues, default out the box OpenMPI.
The P2P queue is fine, but most like using XRC your users will observe better performance. This is not just scalability.