Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI devel] RFC: make predefined handles extern to pointers
From: Terry Dontje (Terry.Dontje_at_[hidden])
Date: 2009-01-30 14:02:43

After some more experiments I found my issue below wasn't due to the
definitions but due to how I was compiling my sources. So it turns out
that I get the same results from dbx when accessing an MPI_Comm type
whether using the original trunk source or using the struct padding.
Which makes sense because I did not change the type of MPI_Comm but the
actual type of the predefined. Which has no affects on the user
variable when the predefined (like MPI_COMM_WORLD) is used as an
initializer since in essence we're using the ompi_communicator_t type
portion to set the user variable (ie nothing really has changed).

So with this latest information, I am going to start coding the other
predefines with padding. I'll post the hg workspace before putting back
to the trunk.


Terry Dontje wrote:
> Per yesterday's concall I did some experiments with the padding
> changes and looking at MPI_Comm structures in dbx. I believe the
> concern from George Bosilca was that using the padding changes you
> wouldn't be able to print out the structures values.
> What I found with dbx and Sun Studio is that prior to call MPI_Init
> the ompi_communicator_t forward reference was unresolved so any
> attempt t print a communicator structure failed because the structure
> was undefined. However once MPI_Init was called the communicator
> structure printed out fine and exactly the same as the non-padded
> implementation.
> I believe non-padded implementation worked because there was extern
> struct ompi_communicator_t that was resolved to the library which I
> imagine pulled in the real structure definition. One could probably
> force the same for the padded implementation by defining dummy
> structures that can be externed in mpi.h. To me this seems gross
> however I wonder does it actually makes sense to print out an MPI
> communicator before MPI_Init is called? The values of the field
> should be either 0 or garbage. So I am really curious if the above is
> a problem anyways.
> --td
> Terry Dontje wrote:
>> Another update for this RFC. It turns out that using pointers
>> instead of structures as initializers would prevent someone from
>> initializing a global to one of the predefined handles. So instead,
>> we decided to go the route of padding the structures to provide us
>> with the ability to not overrun the bss section.
>> I would like to discuss any objections to this solution on tomorrow's
>> OMPI concall.
>> thanks,
>> --td
>> Terry Dontje wrote:
>>> Just wanted to give an update. On a workspace with just the
>>> predefined communicators converted to opaque pointers I've ran
>>> netpipe and hpcc performance tests and compared the results before
>>> and after the changes. The differences in performance with 10
>>> sample run was undetectable.
>>> I've also tested using comm_world that I can have an a.out compile
>>> and link with a non-debug version of the library and then run the
>>> a.out successfully with a debug version of the library. At a simple
>>> level this proves that the change actually does what we believe it
>>> should.
>>> I will be completing the rest of handles in the next couple days.
>>> Upon completion I will rerun the same tests above and test running
>>> hpcc with a debug and non-debug version of the library without
>>> recompiling.
>>> I believe I am on track to putting this back to the trunk by the end
>>> of next week. So if anyone has any issues with this please speak up.
>>> thanks,
>>> --td
>>> Graham, Richard L. wrote:
>>>> No specific test, just an idea how this might impact an app. I am
>>>> guessing it won't even be noticable.
>>>> Rich
>>>> ----- Original Message -----
>>>> From: devel-bounces_at_[hidden] <devel-bounces_at_[hidden]>
>>>> To: Open MPI Developers <devel_at_[hidden]>
>>>> Sent: Thu Dec 18 07:13:08 2008
>>>> Subject: Re: [OMPI devel] RFC: make predefined handles extern to
>>>> pointers
>>>> Richard Graham wrote:
>>>>> Terry,
>>>>> Is there any way you can quantify the cost ? This seems
>>>>> reasonable, but
>>>>> would be nice to get an idea what the performance cost is (and not
>>>>> within a
>>>>> tight loop where everything stays in cache).
>>>>> Rich
>>>> Ok, I guess that would eliminate any of the simple perf tests like
>>>> IMB, netperf, and such. So do you have something else in mind,
>>>> maybe HPCC?
>>>> --td
>>>>> On 12/16/08 10:41 AM, "Terry D. Dontje" <Terry.Dontje_at_[hidden]> wrote:
>>>>>> WHAT: To make predefined handles extern to pointers instead of an
>>>>>> address of an extern to a structure.
>>>>>> WHY: To make OMPI more backwards compatible in regards to
>>>>>> changes to
>>>>>> structures that define predefined handles.
>>>>>> WHERE: In the trunk. ompi/include/ and places in ompi that
>>>>>> directly use the predefined handles.
>>>>>> WHEN: 01/24/2009
>>>>>> TIMEOUT: 01/10/2009
>>>>>> ____________________
>>>>>> The point of this change is to improve the odds that an MPI
>>>>>> application
>>>>>> does not have to recompile when changes are made to the OMPI
>>>>>> library.
>>>>>> In this case specifically the predefined handles that use the
>>>>>> structures
>>>>>> for communicators, groups, ops, datatypes, error handlers, win,
>>>>>> file,
>>>>>> and info.
>>>>>> An example of the changes for the communicator predefined handles
>>>>>> can be
>>>>>> found in the hg tmp workspace at
>>>>>> ssh://
>>>>>> Note, the one downfall that Jeff and I could think of by doing
>>>>>> this is
>>>>>> you potentially add one level of indirection but I believe that
>>>>>> will be
>>>>>> a small overhead and if you use one of the predefined handles
>>>>>> repetitively (like in a loop) that the address will probably be
>>>>>> stored
>>>>>> in a register once and no additional over should be seen due to
>>>>>> this change.
>>>>>> _______________________________________________
>>>>>> devel mailing list
>>>>>> devel_at_[hidden]
>>>>> _______________________________________________
>>>>> devel mailing list
>>>>> devel_at_[hidden]
>>>> _______________________________________________
>>>> devel mailing list
>>>> devel_at_[hidden]
>>>> _______________________________________________
>>>> devel mailing list
>>>> devel_at_[hidden]
>>> _______________________________________________
>>> devel mailing list
>>> devel_at_[hidden]
>> _______________________________________________
>> devel mailing list
>> devel_at_[hidden]
> _______________________________________________
> devel mailing list
> devel_at_[hidden]