Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] RFC: Component-izing MPI_Op
From: Brian W. Barrett (brbarret_at_[hidden])
Date: 2009-01-05 10:09:38


I think this sounds reasonable, if (and only if) MPI_Accumulate is
properly handled. The interface for calling the op functions was broken
in some fairly obvious way for accumulate when I was writing the one-sided
code. I think I had to call some supposedly internal bits of the
interface to make accumulate work. I can't remember what they are now,
but I do remember it being a problem.

Of course, unless it makes mpi_allreduce on one double-sized floating
point number using sum go faster, I'm not entirely sure a change is
helpful ;).

Brian

On Mon, 5 Jan 2009, Jeff Squyres wrote:

> WHAT: Converting the back-end of MPI_Op's to use components instead of
> hard-coded C functions.
>
> WHY: To support specialized hardware (such as GPUs).
>
> WHERE: Changes most of the MPI_Op code, adds a new ompi/mca/op framework.
>
> WHEN: Work has started in an hg branch
> (http://www.open-mpi.org/hg/hgwebdir.cgi/jsquyres/cuda/).
>
> TIMEOUT: Next Tuesday's teleconference, Jan 13 2008.
>
> ---------------------------------------
>
> Note: I don't plan to finish the work by Jan 13; I just want to get a yea/nay
> from the community on the concept. Final review of the code before coming
> into the trunk can come later when I have more work to show / review.
>
> Background: Today, the back-end MPI_Op functionality of (MPI_Op,
> MPI_Datatype) tuples are implemented as function pointers to a series of
> hard-coded C functions in the ompi/op/ directory.
>
> *** NOTE: Since we already implement MPI_Op functionality via function
> pointer, this proposed extension is not expected to cause any performance
> difference in terms of OMPI's infrastructure.
>
> Proposal: Extend the current implementation by creating a new framework
> ("op") that allows components to provide back-end MPI_Op functions instead
> of/in addition to the hard-coded C functions (we've talked about this idea
> before, but never done it).
>
> The "op" framework will be similar to the MPI coll framework in that
> individual function pointers from multiple different modules can be
> mixed-n-matched. For example, if you want to write a new coll component that
> implements *only* a new MPI_BCAST algorithm, that coll component can be
> mixed-n-matched with other coll components at run time to get a full set of
> collective implementations on a communicator. A similar concept will be
> applied to the "op" framework. Case in point: some specialized hardware is
> only good at *some* operations on *some* datatypes; we'll need to fall back
> to the hard-coded C versions for all other tuples.
>
> It is likely that the the "op" framework base will have all the hard-coded C
> "basic" MPI_Op functions that will always be available for fallback if a
> component is not used at run-time for a specialized implementation.
> Specifically: the intent is that components will be for specialized
> implementations.
>
>