Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] can't understand the mean of "ofud" in openmpi
From: Gus Correa (gus_at_[hidden])
Date: 2010-12-02 12:10:11


Hi Jeff

I am glad this question was asked.
Thanks to whoever did it.
Acronyms are always a pain, particularly if you don't know them,
and they are in no dictionary.
OFUD, OFED, OPENIB, MCA, BTL, SM, OOB, ... the list goes on and on.

Your answer makes a great start for another FAQ entry,
called, say, "The OpenMPI Hacker's Dictionary".

Cheers,
Gus Correa

Jeff Squyres wrote:
> On Dec 2, 2010, at 3:59 AM, 阚圣哲 wrote:
>
>> When I use openmpi mpirun --mca btl <arg1>, I find arg1 can be ofud, self, sm, openib, but www.open-mpi.org desn't explain those args.
>
> "BTL" stands for "byte transfer layer" -- is the lowest networking software layer for the "ob1" MPI transport in Open MPI (ob1 is usually the default transport in Open MPI).
>
> Each BTL supports a different kind of network:
>
> - ofud: experimental UD-based OpenFabrics transport. I would not use this; it was developed as part of research and was never really finished.
>
> - self: send-to-self (i.e., loopback to the same MPI process)
>
> - sm: shared memory
>
> - openib: generalized OpenFabrics transport.
>
> Open MPI will automatically pick which BTL to use on a per-communication basis, based on which MPI process peer you are communicating with.
>
> The "--mca btl ..." argument to mpirun restricts which BTLs Open MPI will use at run-time.
>
>> I can't understand the mean of "ofud", what different between "ofud" and "openib",
>> I also can't understand the different between "ibcm" and "rdmacm", when I use mpirun --mca btl_openib_cpc_include <arg2>.
>
> There are 4 different ways for openib BTL to make connections across Open Fabrics networks:
>
> - oob: the default ("out of band", meaning that it uses TCP sockets)
> - xoob: the default when using Mellanox XRC ("out of band with XRC support")
> - rdmacm: the default when using iWARP (because iWARP doesn't support OOB or XOOB)
> - ibcm: not currently used; it's an IB-specific method that was never really finished
>
> Usually, the right CM is just automatically picked -- you shouldn't need to manually select anything.
>
>> maybe www.open-mpi.org can publish a openmpi's document to explain those args and principle.
>
> We are lacking in the documentation department; contributions would be greatly appreciated...
>
> The README file has a bunch about BTLs; that may be helpful reading.
>