Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] RDMA over IB between heterogenous processors with different endianness
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2008-08-25 14:05:27


I believe that this is something Brad at IBM worked on, wasn't it? I
*think* you may just need the development trunk (i.e., upcoming v1.3),
but I won't swear to that.

Regardless, you need to have OMPI compiled for heterogeneous support
because control headers still need to be adjusted for endian-ness,
etc. (even if you're only sending MPI_BYTE).

On Aug 25, 2008, at 1:57 PM, Mi Yan wrote:

> Brian,
>
> I'm using OpenMPI 1.2.6 (r17946). Could you plese check which
> version works ? Thanks a lot,
> Mi
> <graycol.gif>"Brian W. Barrett" <brbarret_at_[hidden]>
>
>
> "Brian W. Barrett" <brbarret_at_[hidden]>
> Sent by: users-bounces_at_[hidden]
> 08/25/2008 01:44 PM
> Please respond to
> Open MPI Users <users_at_[hidden]>
> <ecblank.gif>
> To
> <ecblank.gif>
> Open MPI Users <users_at_[hidden]>
> <ecblank.gif>
> cc
> <ecblank.gif>
> Greg Rodgers/Poughkeepsie/IBM_at_IBMUS, Brad Benton/Austin/IBM_at_IBMUS
> <ecblank.gif>
> Subject
> <ecblank.gif>
> Re: [OMPI users] RDMA over IB between heterogenous processors with
> different endianness
> <ecblank.gif>
> <ecblank.gif>
>
> On Mon, 25 Aug 2008, Mi Yan wrote:
>
> > Does OpenMPI always use SEND/RECV protocol between heterogeneous
> > processors with different endianness?
> >
> > I tried btl_openib_flags to be 2 , 4 and 6 respectively to allowe
> RDMA,
> > but the bandwidth between the two heterogeneous nodes is slow,
> same as
> > the bandwidth when btl_openib_flags to be 1. Seems to me SEND/RECV
> is
> > always used no matter btl_openib_flags is. Can I force OpenMPI to
> use
> > RDMA between x86 and PPC? I only transfer MPI_BYTE, so we do not
> need the
> > support for endianness.
>
> Which version of Open MPI are you using? In recent versions (I don't
> remember exactly when the change occured, unfortuantely), the decision
> between send/recv and rdma was moved from being solely based on the
> architecture of the remote process to being based on the
> architecture and
> datatype. It's possible this has been broken again, but there
> defintiely
> was some window (possibly only on the development trunk) when that
> worked
> correctly.
>
> Brian
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users

-- 
Jeff Squyres
Cisco Systems