Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] RFC: sm Latency
From: Patrick Geoffray (patrick_at_[hidden])
Date: 2009-01-20 23:43:11


Eugene,

All my remarks are related to the receive side. I think the send side
optimizations are fine, but don't take my word for it.

Eugene Loh wrote:
> To recap:
> 1) The work is already done.

How do you do "directed polling" with ANY_TAG ? How do you ensure you
check all incoming queues from time to time to prevent flow control
(specially if the queues are small for scaling) ? What about the
one-sided that Brian mentioned where there is no corresponding receive
to tell you which queue to poll ?

If you want to handle all the constraints, a single-queue model is much
less work in the end, IMHO.

> 2) The single-queue model addresses only one of the RFC's issues.

The single-queue model addresses not only the latency overhead when
scaling, but also the exploding memory footprint. In many ways, these
problems are the same that plagued the RDMA QP model, and the only
solution was using shared receive queues.

By experience, the linear overhead of polling N queues very quickly
become greater than all the optimizations you can do on the send side.

> 3) I'm a fan of the single-queue model, but it's just a separate discussion.

No problem. You are the one doing the real work here, the rest is
armchair quarterbacking :-)

Patrick