On Dec 4, 2013, at 4:31 AM, Paul Kapinos <kapinos_at_[hidden]> wrote:
> Argh - what a shame not to see "btl:usnic" :-|
What a shame you don't have Cisco hardware to use the usnic BTL! :-p
>> Look for the openib messages, not the usnic messages.
> Well, as said there were *no messages* form the patch you provided in
Ah, I see.
> I've attached of a run with single process per node on nodes with 2 NICs, maybe you can see what goes wrong..
What I'm guessing is happening here is that hwloc was built without PCI device detection, and therefore you're not getting the benefit of the near/far detection.
I don't think we currently export whether hwloc was built with PCI device detection support or not, so look for the section in your configure output labeled:
--- MCA component hwloc:hwloc152 (m4 configuration macro, priority 75)
Send the output of that section here. There should be tests for PCI libraries in there; that should tell us whether you have PCI detection support enabled.
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/