Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

From: Gleb Natapov (glebn_at_[hidden])
Date: 2007-05-10 09:44:05


On Thu, May 10, 2007 at 04:30:27PM +0300, Or Gerlitz wrote:
> Jeff Squyres wrote:
> >On May 10, 2007, at 9:02 AM, Or Gerlitz wrote:
>
> >>To start with, my hope here is at least to be able play defensive
> >>here, that is convince you that the disadvantages are minor, where
> >>only if this fails, would schedule myself some reading into the
> >>ipoib-cm rfc to dig the advantages.
>
> >I ask about the advantages because OMPI currently treats QP's as
> >bi-directional. Having OMPI treat them at unidirectional would be a
> >change. I'm not against such a change, but I think we'd need to be
> >convinced that there are good reasons to do so. For example, on the
> >surface, it seems like this scheme would simply consume more QPs and
> >potentially more registered memory (and is therefore unattractive).
>
> Indeed you would need two QPs per btl connection, however, for each
> direction you can make the relevant QP consume ~zero resources per the
> other direction, ie on side A:
>
> for the A --> B QP : RX WR num = 0, RX SG size = 0
> for the B --> A QP : TX WR num = 0, TX SG size = 0
>
> and on side B the other way. I think that IB disallows to have zero len
> WR num so you set it actually to 1. Note that since you use SRQ for
> large jobs you have zero overhead for RX resources and this one TX WR
> overhead for the "RX" connection on each side. This is the only memory
> related overhead since you don't have to allocate any extra buffers over
> what you do now.
>
QP is a limited resource and we already have 2 per connection (and much
more if LMC is in used), so I don't see any reason to use this scheme only
to overcome brain damaged design of iWarp.

--
			Gleb.