On Feb 21, 2008, at 12:36 PM, George Bosilca wrote:
> Here are the MCA parameters that you can use:
> MCA btl: parameter "btl_openib_if_include" (current value:
> Comma-delimited list of HCAs/ports to be used (e.g.
> "mthca0,mthca1:2"; empty value means to use all
> ports found).
> Mutually exclusive with btl_openib_if_exclude.
> MCA btl: parameter "btl_openib_if_exclude" (current value:
> Comma-delimited list of HCAs/ports to be excluded
> (empty value
> means to not exclude any ports). Mutually
> exclusive with
These parameters are [upcoming] v1.3 only -- they do not exist in the
> On Feb 21, 2008, at 2:45 PM, Bill Wichser wrote:
>> In trying to get openmpi up and running on a new cluster, I came
>> this error about having both of my IB switches set to the same
>> subnet-gid. Snooping around on my hosts which run the opensm
>> daemon, I
>> indeed found this to be the case in the /var/log/osm-ib[0-1].log
>> giving up finding it with ibstat which showed these values to be
>> different, at least the second part of the GID.
>> Before I try and pursue how to actually change this value for the
>> daemon, I do have a question.
>> Since both of my hosts are connected to each switch, how am I to
>> instruct openmpi to use port0? I'm trying to use port0 as the MPI
>> network and port1 as the storage network. Is there something that I
>> need to add someplace forcing connections only to some default-
The v1.3 series will have the parameters that George mentioned above;
those give you fine-grained control about which HCAs and ports you are
In the v1.2 series, you cannot explicitly control which hcas/ports you
are using. Instead, you can only limit the *number* of active ports
that Open MPI will use:
MCA btl: parameter "btl_openib_max_btls" (current value:
Maximum number of HCA ports to use (-1 = use all
available, otherwise must be >= 1)
Open MPI starts with the first port on the first interface and goes
upward until it finds max_btls of active ports. This is admittedly
imperfect, but it was only somewhat recently that someone asked for
explicit control over which hcas/ports to use. Sorry... :-\