Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] openib fragment alignment
From: Paul Hargrove (phhargrove_at_[hidden])
Date: 2013-02-20 16:57:36


Sounds like the problem comes down to just 32-bit systems that fault on
unaligned 8-byte loads.
That would be SPARC, IA64 and MIPS. For IB only SPARC is relevant.

So perhaps alignment>2 should be conditional on 32-bit SPARC target.
Additionally, an experiment to see if 4-byte alignment is "good enough"
(instead of 8) might allow one to save a little space.

-Paul

On Wed, Feb 20, 2013 at 1:51 PM, Nathan Hjelm <hjelmn_at_[hidden]> wrote:

> I talked to Pasha about the change. He suggests fragments are 2-byte
> aligned to save space. I suspect that on 64-bit platforms the fragment size
> is already a multiple of 8 bytes so this change will likely only affect
> 32-bit systems (which is where the bus error is occurring).
>
> -Nathan
>
> On Wed, Feb 20, 2013 at 09:39:09PM +0000, Joshua Ladd wrote:
> > I would hold off, if possible, until I can investigate the issue. I
> don't, off-hand, know why the 2-byte alignment, although I would suspect
> it's for performance reasons.
> >
> >
> > Josh
> >
> >
> > -----Original Message-----
> > From: devel-bounces_at_[hidden] [mailto:devel-bounces_at_[hidden]] On
> Behalf Of Jeff Squyres (jsquyres)
> > Sent: Wednesday, February 20, 2013 4:35 PM
> > To: <devel_at_[hidden]>
> > Subject: [OMPI devel] openib fragment alignment
> > Importance: High
> >
> > Arrgh. I think you're going to make me eat my words (
> http://www.open-mpi.org/community/lists/devel/2013/02/12143.php).
> >
> > I just recently lost my access to InfiniBand test gear, so I can't test
> this myself. Hypothetically, it should be fine. But throwing in an
> untested change literally right before a release without IB vendor say-so
> really, really gives me pause...
> >
> > Mellanox?
> >
> >
> > On Feb 20, 2013, at 4:27 PM, Open MPI <bugs_at_[hidden]> wrote:
> >
> > > #3519: Move r28083 to v1.6 branch
> > > -----------------------------------+--------------------------
> > > Reporter: hjelmn | Owner: hjelmn
> > > Type: changeset move request | Status: new
> > > Priority: major | Milestone: Open MPI 1.6
> > > Version: trunk |
> > > -----------------------------------+--------------------------
> > > (In [28083]) btl/openib: don't align fragments on 2 byte boundaries
> > > (changed to 8)
> > >
> > > cmr:v1.6,v1.7
> > >
> > > --
> > > Ticket URL: <https://svn.open-mpi.org/trac/ompi/ticket/3519>
> > > Open MPI <http://www.open-mpi.org/>
> > >
> > > _______________________________________________
> > > bugs mailing list
> > > bugs_at_[hidden]
> > > http://www.open-mpi.org/mailman/listinfo.cgi/bugs
> >
> >
> > --
> > Jeff Squyres
> > jsquyres_at_[hidden]
> > For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
> >
> >
> > _______________________________________________
> > devel mailing list
> > devel_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/devel
> >
> > _______________________________________________
> > devel mailing list
> > devel_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/devel
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>

-- 
Paul H. Hargrove                          PHHargrove_at_[hidden]
Future Technologies Group
Computer and Data Sciences Department     Tel: +1-510-495-2352
Lawrence Berkeley National Laboratory     Fax: +1-510-486-6900