Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] RoCE (IBoE) & OpenMPI
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2011-03-01 09:14:50

I thought you mentioned in a prior email that you had gotten one or two other OFED sample applications to work properly. How are they setting the SL? Are they not using the RDMA CM?

On Mar 1, 2011, at 7:35 AM, Michael Shuey wrote:

> So, since RoCE has no SM, and setting an SL is required to get
> lossless ethernet on Cisco switches (and possibly others), does this
> mean that RoCE will never work correctly with OpenMPI on Cisco
> hardware?
> --
> Mike Shuey
> On Tue, Mar 1, 2011 at 3:42 AM, Doron Shoham <dorons_at_[hidden]> wrote:
>> Hi,
>> Regarding to using a specific SL with RDMA CM, I've checked in the code and it seems that RDMA_CM uses the SL from the SA.
>> So if you want to configure a specific SL, you need to do it via the SM.
>> Doron
>> -----Original Message-----
>> From: Jeff Squyres [mailto:jsquyres_at_[hidden]]
>> Sent: Thursday, February 24, 2011 3:45 PM
>> To: Michael Shuey
>> Cc: Open MPI Users , Mike Dubman
>> Subject: Re: [OMPI users] RoCE (IBoE) & OpenMPI
>> On Feb 24, 2011, at 8:00 AM, Michael Shuey wrote:
>>> Late yesterday I did have a chance to test the patch Jeff provided
>>> (against 1.4.3 - testing 1.5.x is on the docket for today). While it
>>> works, in that I can specify a gid_index,
>> Great! I'll commit that to the trunk and start the process of moving it to the v1.5.x series (I know you haven't tested it yet, but it's essentially the same patch, just slightly adjusted for each of the 3 branches).
>>> it doesn't do everything
>>> required - my traffic won't match a lossless CoS on the ethernet
>>> switch. Specifying a GID is only half of it; I really need to also
>>> specify a service level.
>> RoCE requires the use of the RDMA CM (I think?), and I didn't think there was a way to request a specific SL via the RDMA CM...? (I could certainly be wrong here)
>> I think Mellanox will need to follow up with these questions...
>> --
>> Jeff Squyres
>> jsquyres_at_[hidden]
>> For corporate legal information go to:
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
> _______________________________________________
> users mailing list
> users_at_[hidden]

Jeff Squyres
For corporate legal information go to: