On Jul 24, 2009, at 9:39 AM, Sylvain Jeaugey wrote:
> > Right. But the problem is that at least some subset of MPI users
> today use
> > taskset/plpa_taskset to bind their MPI processes because many
> launchers do
> > *not* already bind for them.
> I must miss something. If the launcher did *not* do the binding, then
> their affinity mask is full, right? So getting affinity shouldn't
> change
> anything. If they use taskset or PLPA taskset, then they know what
> they're
> doing, hence Open MPI shouldn't do anything ...
>
My point is that at least currently, PLPA is not reporting the
difference between "here's the processors you can bind to" vs. "here's
the processors that you are bound to". Hence, OMPI can't tell the
difference between "you're bound to a cpuset; you can move around
within it" vs. "someone used plpa_taskset; you shouldn't do anything".
> > I guess my point is that there really are 2 different pieces of
> information:
> >
> > - what processors you *can* bind to
> > - what processors you *are* bound to
> I understand that, but this difference is only a difference of
> "rights" in
> the kernel. One of the reason why we did cpusets in the kernel (as
> opposed
> to a userland library using affinity) was that affinity was not
> preventing
> broken programs to use cpus they were not entitled to.
>
Right -- and I agree that that's a Good Thing. :-)
> However, that's
> just for broken programs. If a process affinity has been restricted,
> this
> means that some component did an explicit action to restrict the
> affinity,
> and it should (I think) be taken into account by default. Having the
> possibility to try to escape may be useful as a workaround, but
> shouldn't
> be the default case.
>
> I mean, when I launch taskset -c 0 ./myprog, I don't want myprog to go
> elsewhere, otherwise I wouldn't have used taskset, right ?
>
Right. But how can something like mpirun know the difference?
I'm saying that if the (set of allowed processors) == (set of bound
processors), then something like OMPI can know that it effectively has
permission to bind to whatever it wants within that set. If those two
sets are not equal, then something like OMPI can infer that someone
set binding manually (e.g., via plpa_taskset or taskset or whatever)
and therefore it shouldn't do anything. But the only way to be able
to know that is to distinguish between the two different sets of
information (what you're allowed vs. what is currently set).
> > Even if these two pieces of information are sometimes the same,
> sometimes
> > they're not. Hence, I'm suggesting that PLPA should differentiate
> between
> > these two when reporting information upward.
> To sum up my opinion, I think these two pieces of information
> _should_ be
> the same. If they're not, one component's action is being overriden by
> some other and this should be noted/warned/forced/...
>
We agree. :-) I'm just saying that PLPA should be able to report
these two different pieces of information to a caller.
--
Jeff Squyres
jsquyres_at_[hidden]
|
![[Flag]](../../../../../community/mirrors/flag-us.gif){kind=small}