Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] PML selection logic
From: Ralph H Castain (rhc_at_[hidden])
Date: 2008-06-23 14:35:49

Since my goal is to eliminate the modex completely for managed
installations, could you give me a brief understanding of this eventual PML
selection logic? It would help to hear an example of how and why different
procs could get different answers - and why we would want to allow them to
do so.


On 6/23/08 11:59 AM, "Aurélien Bouteiller" <bouteill_at_[hidden]> wrote:

> The first approach sounds fair enough to me. We should avoid 2 and 3
> as the pml selection mechanism used to be
> more complex before we reduced it to accommodate a major design bug in
> the BTL selection process. When using the complete PML selection, BTL
> would be initialized several times, leading to a variety of bugs.
> Eventually the PML selection should return to its old self, when the
> BTL bug gets fixed.
> Aurelien
> Le 23 juin 08 à 12:36, Ralph H Castain a écrit :
>> Yo all
>> I've been doing further research into the modex and came across
>> something I
>> don't fully understand. It seems we have each process insert into
>> the modex
>> the name of the PML module that it selected. Once the modex has
>> exchanged
>> that info, it then loops across all procs in the job to check their
>> selection, and aborts if any proc picked a different PML module.
>> All well and good...assuming that procs actually -can- choose
>> different PML
>> modules and hence create an "abort" scenario. However, if I look
>> inside the
>> PML's at their selection logic, I find that a proc can ONLY pick a
>> module
>> other than ob1 if:
>> 1. the user specifies the module to use via -mca pml xyz or by using a
>> module specific mca param to adjust its priority. In this case,
>> since the
>> mca param is propagated, ALL procs have no choice but to pick that
>> same
>> module, so that can't cause us to abort (we will have already
>> returned an
>> error and aborted if the specified module can't run).
>> 2. the pml/cm module detects that an MTL module was selected, and
>> that it is
>> other than "psm". In this case, the CM module will be selected
>> because its
>> default priority is higher than that of OB1.
>> In looking deeper into the MTL selection logic, it appears to me
>> that you
>> either have the required capability or you don't. I can see that in
>> some
>> environments (e.g., rsh across unmanaged collections of machines),
>> it might
>> be possible for someone to launch across a set of machines where
>> some do and
>> some don't have the required support. However, in all other cases,
>> this will
>> be homogeneous across the system.
>> Given this analysis (and someone more familiar with the PML should
>> feel free
>> to confirm or correct it), it seems to me that this could be
>> streamlined via
>> one or more means:
>> 1. at the most, we could have rank=0 add the PML module name to the
>> modex,
>> and other procs simply check it against their own and return an
>> error if
>> they differ. This accomplishes the identical functionality to what
>> we have
>> today, but with much less info in the modex.
>> 2. we could eliminate this info from the modex altogether by
>> requiring the
>> user to specify the PML module if they want something other than the
>> default
>> OB1. In this case, there can be no confusion over what each proc is
>> to use.
>> The CM module will attempt to init the MTL - if it cannot do so,
>> then the
>> job will return the correct error and tell the user that CM/MTL
>> support is
>> unavailable.
>> 3. we could again eliminate the info by not inserting it into the
>> modex if
>> (a) the default PML module is selected, or (b) the user specified
>> the PML
>> module to be used. In the first case, each proc can simply check to
>> see if
>> they picked the default - if not, then we can insert the info to
>> indicate
>> the difference. Thus, in the "standard" case, no info will be
>> inserted.
>> In the second case, we will already get an error if the specified
>> PML module
>> could not be used. Hence, the modex check provides no additional
>> info or
>> value.
>> I understand the motivation to support automation. However, in this
>> case,
>> the automation actually doesn't seem to buy us very much, and it isn't
>> coming "free". So perhaps some change in how this is done would be
>> in order?
>> Ralph
>> _______________________________________________
>> devel mailing list
>> devel_at_[hidden]
> _______________________________________________
> devel mailing list
> devel_at_[hidden]