I've been looking into OpenMPI's support for RoCE (Mellanox's recent
Infiniband-over-Ethernet) lately. While it's promising, I've hit a
snag: RoCE requires lossless ethernet, and on my switches the only way
to guarantee this is with CoS. RoCE adapters cannot emit CoS priority
tags unless the client program selects an IB service level and uses a
There's a command-line option in OpenMPI to pick an IB SL, but I can't
find one for picking a different GID. Does this exist for the openib
btl? Or am I going about this the wrong way?