Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] RTE Issue II: Interaction between the ROUTED and GRPCOMM frameworks
From: Brian W. Barrett (brbarret_at_[hidden])
Date: 2007-12-05 11:29:46

To me, (a) is dumb and (c) isn't a non-starter.

The whole point of the component system is to seperate concerns. Routing
topology and collectives operations are two difference concerns. While
there's some overlap (a topology-aware collective doesn't make sense when
using the unity routing structure), it's not overlap in the one implies
you need the other. I can think of a couple of different ways of
implementing the group communication framework, all of which are totally
independent of the particulars of how routing is tracked.

(b) has a very reasonable track record of working well on the OMPI side
(the mpool / btl thing figures itself out fairly well). Bringing such a
setup over to ORTE wouldn't be bad, but a bit hackish.

Of course, there's at most two routed components built at any time, and
the defaults are all most non-debugging people will ever need, so I guess
I"m not convinced (c) isn't a non-starter.


On Wed, 5 Dec 2007, Tim Prins wrote:

> To me, (c) is a non-starter. I think whenever possible we should be
> automatically doing the right thing. The user should not need to have
> any idea how things work inside the library.
> Between options (a) and (b), I don't really care.
> (b) would be great if we had a mca component dependency system which has
> been much talked about. But without such a system it gets messy.
> (a) has the advantage of making sure there is no problems and allowing
> the 2 systems to interact very nicely together, but it also might add a
> large burden to a component writer.
> On a related, but slightly different topic, one thing that has always
> bothered me about the grpcomm/routed implementation is that it is not
> self contained. There is logic for routing algorithms outside of the
> components (for example, in orte/orted/orted_comm.c). So, if there are
> any overhauls planned I definitely think this needs to be cleaned up.
> Thanks,
> Tim
> Ralph H Castain wrote:
>> II. Interaction between the ROUTED and GRPCOMM frameworks
>> When we initially developed these two frameworks within the RTE, we
>> envisioned them to operate totally independently of each other. Thus, the
>> grpcomm collectives provide algorithms such as a binomial "xcast" that uses
>> the daemons to scalably send messages across the system.
>> However, we recently realized that the efficacy of the current grpcomm
>> algorithms directly hinge on the daemons being fully connected - which we
>> were recently told may not be the case as other people introduce different
>> ROUTED components. For example, using the binomial algorithm in grpcomm's
>> xcast while having a ring topology selected in ROUTED would likely result in
>> terrible performance.
>> This raises the following questions:
>> (a) should the GRPCOMM and ROUTED frameworks be consolidated to ensure that
>> the group collectives algorithms properly "match" the communication
>> topology?
>> (b) should we automatically select the grpcomm/routed pairings based on some
>> internal logic?
>> (c) should we leave this "as-is" and the user is responsible for making
>> intelligent choices (and for detecting when the performance is bad due to
>> this mismatch)?
>> (d) other suggestions?
>> Ralph
>> _______________________________________________
>> devel mailing list
>> devel_at_[hidden]
> _______________________________________________
> devel mailing list
> devel_at_[hidden]