On Wed, 2007-05-09 at 08:37 +0300, Or Gerlitz wrote:
> Andrew Friedley wrote:
> > Jeff Squyres wrote:
> >>>> FWIW, yes, adding RDMA CM support has actually been on my to-do list
> >>>> for a while, but it keeps getting bumped by higher priority items.
> >>>> It would be *much* better if some iWARP companies got involved in
> >>>> Open MPI...
> > Hmm I'm interested. I've already done some work switching over to RDMA
> > CM for some research stuff I've been doing; it's not publicly accessible
> > w/o the 3rd party agreement. I can help answer questions on what
> > exactly needs to change, and do some testing.
> Doing a bit of zoom out from the "how to make ofed's udapl work for
> ompi" thread, my thinking is that the ompi udapl btl enablement is
> actually only the first step, where for production/longterm/etc you want
> to have an rdmacm btl. Reasoning here is made of many arguments, among
> them the quickest i can make are:
> A) it seems that ompi would want to use not only RC but rather also UD
> multicast and unicast, which are not covered by udapl
> B) there's actually no real justification to maintain two APIs (namely
> udapl vs libibvers/librdmacm), so down the road, only one of them would
> survive (udapl is implemented ***over*** libibverbs/librdmacm so if the
> latteres dies same does udapl). Specifically, I hear here and there that
> the OFED stack is now on its way to be deployed all over the place,
> specifically in commercial Unix OSs (which want modern! code that
> supports IPoIB-CM,RDS,SRP,iSER, etc you named it) so eventually the
> rdmacm btl can be used also over Solaris et al.
Agreed. enabling udapl will get OMPI over iwarp immediately (and
hopefully in ofed-1.2). Post ofed-1.2, I think OMPI _should_ create a
rdma-cm btl. That's the plan...