Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] RFC: Pineapple Runtime Interposition Project
From: Josh Hursey (jjhursey_at_[hidden])
Date: 2012-06-26 09:28:54


The final thing to do for the Pineapple project is to give it a real
name. This is an easy sed script to patch the project, but finding a
name is somewhat challenging.

The base parameters for the name are:
 - it has to start with an 'O' word
 - acronym must be 4 letters (O???)
 - acronym must be pronounceable
 - acronym must be G rated

A few suggestions:
 - ORCA: Open Runtime Collaborative Abstraction
 - OCRA: Open Collaborative Runtime Abstraction
 - OREI: Open Runtime Environment Interface
 - ORSL: Open Runtime Services Layer
 - ORRA: Open Runtime Resource Abstraction

We need to decide this on the call today, so have fun thinking of suggestions.

-- Josh

On Mon, Jun 25, 2012 at 8:49 AM, Josh Hursey <jjhursey_at_[hidden]> wrote:
> Just a reminder that this change is headed for the trunk tomorrow
> afternoon/evening. We will discuss it on the teleconf, but if you have
> cycles to test please do so.
>
> Thanks,
> Josh
>
> On Fri, Jun 22, 2012 at 1:45 PM, Josh Hursey <jjhursey_at_[hidden]> wrote:
>> In response to some early feedback, I fixed the following in the
>> branch and updated the wiki:
>>  * Fixed 'make distcheck' (missing a file)
>>  * Cleanup some warnings from autogen and configure for -no-orte builds
>>  * Added an ./autogen.pl -no-pineapple option (give you just the
>> ORTE/OPAl stack) - see wiki
>>
>> Thanks for the feedback and keep it coming.
>>
>> Also - The name will be changed before the commit to the trunk, I'm
>> working on some suggestions, but if you have any that you want to
>> lobby for let me know.
>>
>> Thanks,
>> Josh
>>
>> On Fri, Jun 22, 2012 at 8:41 AM, Josh Hursey <jjhursey_at_[hidden]> wrote:
>>> The pineapple branch is ready for testing. It was last sync'ed to the
>>> trunk at r26620. I'll try to update that later today, and keep current
>>> with the trunk moving forward.
>>>
>>> I created a wiki page to discuss some of the build options, for those
>>> interested. At the bottom of the page are some remaining todo items
>>> that I will probably need some help resolving.
>>>  https://svn.open-mpi.org/trac/ompi/wiki/Runtime_Interposition
>>>
>>> The branch is at:
>>>  https://bitbucket.org/jjhursey/ompi-pineapple
>>>
>>> I would like to bring this into the trunk the evening after the
>>> teleconf on Tuesday, June 26, 2012. So if you have cycles to test this
>>> branch I would appreciate it.
>>>
>>> Thanks,
>>> Josh
>>>
>>> On Fri, Jun 15, 2012 at 2:55 PM, Josh Hursey <jjhursey_at_[hidden]> wrote:
>>>> What: A Runtime Interposition Project - Codename Pineapple
>>>>
>>>> Why: Define clear API and semantics for runtime requirements of the OMPI layer.
>>>>
>>>> When:
>>>>  - F June 22, 2012 - Work completed
>>>>  - T June 26, 2012 - Discuss on teleconf
>>>>  - R June 28, 2012 - Commit to trunk
>>>>
>>>> Where: Trunk (development BitBucket branch below)
>>>>  https://bitbucket.org/jjhursey/ompi-pineapple
>>>>
>>>> Attached:
>>>>  PDF of slides presented on the June 12, 2012 teleconf. Note that the
>>>> timeline was slightly adjusted above (work completed date moved
>>>> ealier).
>>>>
>>>>
>>>> Description: Short Version
>>>> --------------------------
>>>> Define, in an 'rte.h', the interfaces and semantics that the OMPI
>>>> layer requires of a runtime environment. Currently this interface
>>>> matches the subset of ORTE functionality that is used by the OMPI
>>>> layer. Runtime symbols (e.g., orte_ess.proc_get_locality) are isolated
>>>> to a framework inside this project to provide linker-level protection
>>>> against accidental breakage of the pineapple interposition layer.
>>>>
>>>> The interposition project provides researchers working on side
>>>> projects above and below the 'rte.h' interface a single location in
>>>> the code base to watch for interface and semantic changes that they
>>>> need to be concerned about. Researchers working above the pineapple
>>>> layer might explore something other than (or in addition to) OMPI
>>>> (e.g., Extended OMPI, UPC+OMPI). Researchers working below the
>>>> pineapple layer might explore something other than (or in addition to)
>>>> ORTE under OMPI (e.g., specialized runtimes for specific
>>>> environments).
>>>>
>>>>
>>>> Description: Other notes
>>>> ------------------------
>>>> The pineapple interface provides OMPI developers with a runtime API to
>>>> program against without requiring detailed knowledge of the layout of
>>>> ORTE and its frameworks. In some places in OMPI a single source file
>>>> needs to include >5 (up to 12 in one place) different header files to
>>>> get all of the necessary symbols. Developers must not only know where
>>>> these headers are, but must also understand the differences between
>>>> the various frameworks in ORTE to use ORTE. The developer must also be
>>>> aware that there are certain APIs and data structure fields that are
>>>> not available to the MPI process, so should not be used. The pineapple
>>>> project provides an API representing the small subset of ORTE that is
>>>> used by OMPI. With this API a developer only needs to look at a single
>>>> location in the code base to understand what is provided by the
>>>> runtime for use in the OMPI layer.
>>>>
>>>> A similar statement could be made for runtime developers trying to
>>>> figure out what the OMPI layer requires from the a runtime
>>>> environment. Currently they need a deep understanding of the behavior
>>>> of ORTE to understand the semantics of various calls to ORTE from the
>>>> OMPI layer. Then they must develop a custom patch for the OMPI layer
>>>> that extracts the ORTE symbols, and replaces them with their own
>>>> symbols. This process is messy, error prone, and tedious to say the
>>>> least. Having a single set of interfaces and semantics will allow such
>>>> developers to focus their efforts on supporting the Open MPI community
>>>> defined API, and not necessarily the evolution of the ORTE or OMPI
>>>> project internals. This is advantageous when porting Open MPI to an
>>>> environment with a full featured runtime already running on the
>>>> machine, and for researchers exploring radical runtime designs for
>>>> future systems. The pineapple API allows such projects to develop
>>>> beside the mainline Open MPI trunk a little more easily than without
>>>> the pineapple API.
>>>>
>>>>
>>>> FAQ:
>>>> ----
>>>> (1) Why is this a separate project and not a framework of OMPI? or a
>>>> framework of ORTE?
>>>>
>>>> After much deliberation between the developers, from a software
>>>> engineering perspective, making the pineapple rte.h interface a
>>>> separate project was the most flexible solution. So neither the OMPI
>>>> layer nor the ORTE layer 'own' the interface, but it is 'owned' by the
>>>> Open MPI project primarily to support the interaction between these
>>>> two layers.
>>>>
>>>> Consider that if we decided to place the interface in the OMPI layer
>>>> as a framework then we would be able to place something other than (or
>>>> in addition to) ORTE underneath OMPI, but we would be limited in our
>>>> ability to place something other than (or in addition to) OMPI over
>>>> ORTE. Alternatively, if we decided to place the rte.h interface in the
>>>> ORTE layer then we would be able to place something other than (or in
>>>> addition to) OMPI over ORTE, but we would be limited in our ability to
>>>> place something other than (or in addition to) ORTE under OMPI.
>>>> Defining the interposition layer as a separate project between these
>>>> two layers allows maximal flexibility for the project and researchers
>>>> working on side branches.
>>>>
>>>>
>>>> (2) What if another project outside of Open MPI needs interface
>>>> changes to the pineapple 'rte.h'?
>>>>
>>>> The rule of thumb is that 'The OMPI/ORTE/OPAL stack is king!'. This
>>>> means that the pineapple project should always err on the side of
>>>> supporting the OMPI/ORTE/OPAL stack, as that is the flagship product
>>>> of the Open MPI project. Interface suggestions are always welcome and
>>>> the rte.h may be extended/modified in the future as a result of those
>>>> suggestions. However, if a suggested change negatively impacts the
>>>> OMPI/ORTE/OPAL stack then it is unlikely to be accepted upstream by
>>>> the Open MPI community.
>>>>
>>>>
>>>> --
>>>> Joshua Hursey
>>>> Postdoctoral Research Associate
>>>> Oak Ridge National Laboratory
>>>> http://users.nccs.gov/~jjhursey
>>>
>>>
>>>
>>> --
>>> Joshua Hursey
>>> Postdoctoral Research Associate
>>> Oak Ridge National Laboratory
>>> http://users.nccs.gov/~jjhursey
>>
>>
>>
>> --
>> Joshua Hursey
>> Postdoctoral Research Associate
>> Oak Ridge National Laboratory
>> http://users.nccs.gov/~jjhursey
>
>
>
> --
> Joshua Hursey
> Postdoctoral Research Associate
> Oak Ridge National Laboratory
> http://users.nccs.gov/~jjhursey

-- 
Joshua Hursey
Postdoctoral Research Associate
Oak Ridge National Laboratory
http://users.nccs.gov/~jjhursey