Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] FDR: bad patch? (was: 1.6rc3 is out)
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2012-05-10 09:01:57


Looks ok to me.

FWIW, I'd put EDR in, if you know the values (even though there won't be any hardware for a while). That'll just give this code a longer shelf life / future-proof-ness...

On May 10, 2012, at 8:57 AM, Yevgeny Kliteynik wrote:

> On 10-May-12 3:42 PM, Yevgeny Kliteynik wrote:
>>
>> Here are the ActiveSpeed values:
>>
>> 1: SDR - 2.5 Gb/s * (8/10 coding)
>> 2: DDR - 5 Gb/s * (8/10 coding)
>> 4: QDR - 10 Gb/s * (8/10 coding)
>> 8: FDR10 - 10 Gb/s * (64/66 coding)
>> 16: FDR - 14 Gbps * (64/66 coding)
>> 32: EDR - 25 Gbs * (64/66 coding)
>>
>> EDR is not out yet, so we can ignore it for the sake of this patch.
>> It is also true that *most* of IB cables are 4x.
>> There are 12x clusters out there (like TACC's Ranger), and sometimes
>> there are 1x links (mostly because the cable is not perfect or too long).
>>
>> So cases of 4 and 8 are not the same for us, because MPI needs actual
>> *data* rate instead of *bit* rate, which is different in these cases.
>>
>> I'll send a patch for review in a minute...
>>
>
> How about this patch? If it looks OK, I'll commit to trunk and open CMR.
>
> ndex: btl_openib_component.c
> ===================================================================
> --- btl_openib_component.c (revision 26420)
> +++ btl_openib_component.c (working copy)
> @@ -810,8 +810,10 @@
> we have to look up the values corresponding to
> port->active_speed and port->active_width. These
> are enums corresponding to the IB spec. Overall
> - forumula is 80% of the reported speed (to get the
> - true link speed) times the number of links. */
> + forumula to get the true link speed is 8/10 or
> + 64/66 of the reported speed (depends on the coding
> + that is being used for the particular speed) times
> + the number of links. */
> switch (ib_port_attr->active_speed) {
> case 1:
> /* 2.5Gbps * 0.8, in megabits */
> @@ -826,9 +828,13 @@
> openib_btl->super.btl_bandwidth = 8000;
> break;
> case 8:
> - /* 40.0Gbps * 0.8, in megabits */
> - openib_btl->super.btl_bandwidth = 32000;
> + /* FDR10: 10.0Gbps * 64/66, in megabits */
> + openib_btl->super.btl_bandwidth = 9700;
> break;
> + case 16:
> + /* FDR: 14.0Gbps * 64/66, in megabits */
> + openib_btl->super.btl_bandwidth = 13600;
> + break;
> default:
> /* Who knows? Declare this port unreachable (do
> *not* return ERR_VALUE_OF_OUT_OF_BOUNDS; that

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/