Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] OMPI OpenIB Credit Schema breaks Chelsio HW
From: Steve Wise (swise_at_[hidden])
Date: 2008-03-10 14:51:24


Gleb Natapov wrote:
> On Mon, Mar 10, 2008 at 09:50:13AM -0500, Steve Wise wrote:
>
>>> I personally don't like the idea to add another layer of complexity to openib
>>> BTL code just to work around HW that doesn't follow spec. If work around
>>> is simple that is OK, but in this case it is not so simple and will add
>>> code path that is rarely tested. A simple workaround for the problem may
>>> be to not configure multiple QPs if HW has a bug (and we can extend ini
>>> file to contain this info).
>>>
>>>
>>>
>> It doesn't sound too complex to implement the above design. In fact,
>> that's the way this btl used to work, no? There are large customers
>> requesting OMPI over cxgb3 and we're ready to provide the effort to get
>> this done. So I request we come to an agreement on how to support this
>> device efficiently (and for ompi-1.3).
>>
> Yes. The btl used to work like that before. But the current way of doing
> credit management requires much less memory, so I don't think reverting
> to the old way is a right thing. And having two different ways of
> sending credit updates seems like additional complexity. The problem is
> not just with writing code, but this code will have to be maintained for
> unknown period of time (will this problem be solved in your next gen HW?).
>
Yes.
> I am OK with adding old fc in addition to current fc if the code is trivial
> and easy to maintain. Do you think it is possible to add what you want
> and meet these requirements?
>
I hope so! :)

But I think we're going to end up using just a single PP QP for this
version of the chelsio HW. We're exploring how that works now. The next
gen rnic from chelsio will support SRQs and fix this post_recv issue, so
we can then plug in properly with OMPI.

Steve.

> --
> Gleb.
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>