Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] IBCM error
From: Don Kerr (Don.Kerr_at_[hidden])
Date: 2008-07-16 11:07:30


Jeff Squyres wrote:
> On Jul 15, 2008, at 7:30 AM, Ralph Castain wrote:
>
>>> Minor clarification: we did not test RDMACM on RoadRunner.
>>
>> Just for further clarification - I did, and it wasn't a particularly
>> good
>> experience. Encountered several problems, none of them overwhelming,
>> hence
>> my comments.
>
> Ah -- I didn't know this. What went wrong? We need to fix it if
> there are problems.
>
>>> RDMACM, on the other hand, is *necessary* for iWARP connections. We
>>> know it won't scale well because of ARP issues, to which the iWARP
>>> vendors are publishing their own solutions (pre-populating ARP caches,
>>> etc.). Even when built and installed, RDMACM will not be used by
>>> default for IB hardware (you have to specifically ask for it). Since
>>> it's necessary for iWARP, I think we need to build and install it by
>>> default. Most importantly: production IB users won't be disturbed.
>>
>> If it is necessary for iWARP, then fine - so long as it is only used if
>> specifically requested.
>>
>> However, I would also ask that we be able to -not- build it upon
>> request so
>> we can be certain a user doesn't attempt to use it by mistake ("gee,
>> that
>> looks interesting - let Mikey try it!"). Ditto for ibcm support.
>
> Pasha added configure switches for this about a week ago:
>
> --en|disable-openib-ibcm
> --en|disable-openib-rdmacm
I like these flags but I thought there was going to be a run time check
for cases where Open MPI is built on a system that has ibcm support but
is later run on a system without ibcm support.

-DON