Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] RFC: new OMPI RTE define:
From: Jeff Squyres (jsquyres) (jsquyres_at_[hidden])
Date: 2014-02-18 07:16:31


Ok, fair enough. My goal was not to spin up another progress thread in my BTL, but I can certainly do so (to meet the 1.7.5 timeframe).

For the longer term (i.e., 1.9), should we add a little opal infrastructure that contains an event base that is run in its own progress thread? This would allow the MPI layer to consolidate into one progress thread (for things that are event based, at least). I don't believe much work would be needed here.

For example, the openib BTL could use this async-thread event-driven infrastructure, too (vs. spinning up 2 progress threads of its own).

FWIW: the usNIC BTL events I need will be driven by timers and fd's, so it fits into the libevent model just fine (although I have some thoughts of possibly adapting this functionality to run in the orted when possible in the 1.9 timeframe, but I haven't thought that through yet... 1.7.5 first!).

On Feb 18, 2014, at 5:21 AM, George Bosilca <bosilca_at_[hidden]> wrote:

> I concur with Brian, you should not expect the runtime to provide a default event base, especially if you want some level of quality-of-service out of it. Moreover, with the soon-to-happen move of the BTLs down in OPAL this approach will definitively not be suitable.
>
> George.
>
>
> On Feb 18, 2014, at 07:03 , Brian Barrett <brian_at_[hidden]> wrote:
>
>> And what will you do for RTE components that aren't ORTE? This really isn't a feature of a run-time, so it doesn't seem like it should be part of the RTE interface...
>>
>> Brian
>>
>> On Feb 17, 2014, at 3:03 PM, Jeff Squyres (jsquyres) <jsquyres_at_[hidden]> wrote:
>>
>>> WHAT: New OMPI_RTE_EVENT_BASE define
>>>
>>> WHY: The usnic BTL needs to run some events asynchronously; the ORTE event base already exists and is running asynchronously in MPI processes
>>>
>>> WHERE: in ompi/mca/rte/rte.h and rte_orte.h
>>>
>>> TIMEOUT: COB Friday, 21 Feb 2014
>>>
>>> MORE DETAIL:
>>>
>>> The WHY line described it pretty well: we want to run some things asynchronously in the usnic BTL and we don't really want to re-invent the wheel (or add yet another thread in each MPI process). The ORTE event base is already there, there's already a thread servicing it, and Ralph tells me that it is safe to add our own events on to it.
>>>
>>> The patch below adds the new OMPI_RTE_EVENT_BASE #define.
>>>
>>>
>>> diff --git a/ompi/mca/rte/orte/rte_orte.h b/ompi/mca/rte/orte/rte_orte.h
>>> index 3c88c6d..3ceadb8 100644
>>> --- a/ompi/mca/rte/orte/rte_orte.h
>>> +++ b/ompi/mca/rte/orte/rte_orte.h
>>> @@ -142,6 +142,9 @@ typedef struct {
>>> } ompi_orte_tracker_t;
>>> OBJ_CLASS_DECLARATION(ompi_orte_tracker_t);
>>>
>>> +/* define the event base that the RTE exports */
>>> +#define OMPI_RTE_EVENT_BASE orte_event_base
>>> +
>>> END_C_DECLS
>>>
>>> #endif /* MCA_OMPI_RTE_ORTE_H */
>>> diff --git a/ompi/mca/rte/rte.h b/ompi/mca/rte/rte.h
>>> index 69ad488..de10dff 100644
>>> --- a/ompi/mca/rte/rte.h
>>> +++ b/ompi/mca/rte/rte.h
>>> @@ -150,7 +150,9 @@
>>> * a. OMPI_DB_HOSTNAME
>>> * b. OMPI_DB_LOCALITY
>>> *
>>> - * (g) Communication support
>>> + * (g) Asynchronous / event support
>>> + * 1. OMPI_RTE_EVENT_BASE - the libevent base that executes in a
>>> + * separate thread
>>> *
>>> */
>>>
>>> @@ -162,6 +164,7 @@
>>> #include "opal/dss/dss_types.h"
>>> #include "opal/mca/mca.h"
>>> #include "opal/mca/base/base.h"
>>> +#include "opal/mca/event/event.h"
>>>
>>> BEGIN_C_DECLS
>>>
>>>
>>>
>>> _______________________________________________
>>> devel mailing list
>>> devel_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>
>>
>> --
>> Brian Barrett
>>
>> There is an art . . . to flying. The knack lies in learning how to
>> throw yourself at the ground and miss.
>> Douglas Adams, 'The Hitchhikers Guide to the Galaxy'
>>
>> _______________________________________________
>> devel mailing list
>> devel_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/