Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] btl/openib: get_ib_dev_distance doesn't see processes as bound if the job has been launched by srun
From: Ralph Castain (rhc_at_[hidden])
Date: 2012-02-09 08:00:16


There is another aspect, though - I had missed it in the thread, but the question Nadia was addressing is: how to tell I am bound? The way we currently do it is to compare our cpuset against the local cpuset - if we are on a subset, then we know we are bound.

So if all hwloc returns to us is our cpuset, then we cannot make that determination. Yet I do see a utility as well in only showing our own cpus.

Would it make sense to add a field to the hwloc_obj_t that contains the "accessible" cpus? Or a flag indicating "you are bound to a subset of all available cpus"?

Really, all we need is the flag - but we could compute it ourselves if we had the larger scope info.

On Feb 9, 2012, at 5:53 AM, Jeff Squyres wrote:

> On Feb 9, 2012, at 7:50 AM, Chris Samuel wrote:
>
>>> Just so that I understand this better -- if a process is bound in a
>>> cpuset, will tools like hwloc's lstopo only show the Linux
>>> processors *in that cpuset*? I.e., does it not have any
>>> visibility of the processors outside of its cpuset?
>>
>> I believe that was the intention - there's no real benefit to knowing
>> about resources that you can't access or use (and will likely get an
>> error if you do) to my mind.
>
> The real question, however, is how are IO devices represented if you don't do WHOLE_SUBSYSTEM? I.e., what about IO devices that are not local to the socket of your cpuset, for example?
>
> Take this sample image, for example:
>
> http://www.open-mpi.org/projects/hwloc/devel09-pci.png
>
> What if my cpuset is only on Socket P#0? What exactly will be reported via (WHOLE_SUBSYSTEM | HWLOC_TOPOLOGY_FLAG_WHOLE_IO)?
>
> IO devices is something that we do have an interest in reporting so that we can tell the "distance" to them, for example.
>
> --
> Jeff Squyres
> jsquyres_at_[hidden]
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
>
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel