Received-SPF: pass (google.com: domain of jsquyres@cisco.com designates
	64.102.122.148 as permitted sender) client-ip=64.102.122.148; 
Cc: pasha@dev.mellanox.co.il
Message-Id: <A5F43242-D906-42EC-8355-2AAC4B9DB2A9@cisco.com>
From: Jeff Squyres <jsquyres@cisco.com>
To: Open MPI Developers <devel@open-mpi.org>
In-Reply-To: <4AB9E4E2.10203@dev.mellanox.co.il>
Content-Type: text/plain; charset=WINDOWS-1252; format=flowed; delsp=yes
Content-Transfer-Encoding: quoted-printable
Mime-Version: 1.0 (Apple Message framework v936)
Subject: Re: [OMPI devel] [PATCH] Improving heterogeneous IB clusters support.
Date: Thu, 24 Sep 2009 11:56:49 -0400
References: <4AB9E4E2.10203@dev.mellanox.co.il>

Thanks for the patch.  A few comments:

--- a/ompi/mca/btl/openib/btl_openib_endpoint.h	Mon Aug 31 00:00:16 =20
2009 -0700
+++ b/ompi/mca/btl/openib/btl_openib_endpoint.h	Thu Sep 17 18:23:58 =20
2009 +0300
@@ -246,6 +246,12 @@
      /** Whether we've send out CTS to the peer or not (only used in
          CTS protocol) */
      bool endpoint_cts_sent;
+
+    uint32_t vendor_id;
+    uint32_t vendor_part_id;
+
+    uint32_t max_inline_data;
+
  };

The vendor_id and vendor_part_id actually the *remote* values of this =20=

information.  Shouldn't that go in endpoint.rem_info?

Is there a reason to put the max_inline_data on the endpoint rather =20
than accessing it through endpoint->endpoint_btl->device->ib_dev?  =20
(I'm a little confused on how it is used / assigned -- I could be =20
missing something here)

What testing has been done to see that this stuff works?

Has an equivalent patch been made for Pasha's ofacm work?


On Sep 23, 2009, at 5:05 AM, Vasily Filipov wrote:

> Hello,
>
> Some time ago Mellanox proposed design that should improve current =20
> support for heterogeneous clusters (see Design.txt).The design was =20
> accepted by IB vendors, and now we propose patch that adds a =20
> heterogeneous cluster support. The path leaves one issue that we do =20=

> not resolve completely. If 2 different procs have different QPs =20
> configuration (P/S/X) we print nice warning message that describes =20
> that such configuration is not supported and it propose way to =20
> resolve the issue.  Theoretically it will be best to provide =20
> solution that automatically will resolve the problem, but it will =20
> require significant changes on openib blt that we don=92t want to =20
> introduce in this stage.
>
> Please comment.
>
> Regards,
> Vasily
>
> <=20
> Design=20
> .txt><btl_openib.patch>_______________________________________________
> devel mailing list
> devel@open-mpi.org
> http://www.open-mpi.org/mailman/listinfo.cgi/devel


--=20
Jeff Squyres
jsquyres@cisco.com