Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] btl/openib: get_ib_dev_distance doesn't see processes as bound if the job has been launched by srun
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2012-02-09 07:53:39


On Feb 9, 2012, at 7:50 AM, Chris Samuel wrote:

>> Just so that I understand this better -- if a process is bound in a
>> cpuset, will tools like hwloc's lstopo only show the Linux
>> processors *in that cpuset*? I.e., does it not have any
>> visibility of the processors outside of its cpuset?
>
> I believe that was the intention - there's no real benefit to knowing
> about resources that you can't access or use (and will likely get an
> error if you do) to my mind.

The real question, however, is how are IO devices represented if you don't do WHOLE_SUBSYSTEM? I.e., what about IO devices that are not local to the socket of your cpuset, for example?

Take this sample image, for example:

    http://www.open-mpi.org/projects/hwloc/devel09-pci.png

What if my cpuset is only on Socket P#0? What exactly will be reported via (WHOLE_SUBSYSTEM | HWLOC_TOPOLOGY_FLAG_WHOLE_IO)?

IO devices is something that we do have an interest in reporting so that we can tell the "distance" to them, for example.

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/