Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] Padding on ompi_predefined_datatype_t running out
From: George Bosilca (bosilca_at_[hidden])
Date: 2014-01-13 18:03:34


Sorry I have misunderstood your question. I looked at what we have in the trunk, and with the exception of the datatype we’re mostly good (a table of predefined types and their corresponding memory consumption is at the end of this email).

So in the worst case if we increase the ompi_predefined_datatype_t to 1k, we will be using an extra 12k (similar to the MPI_Op today). In same time, I don’t know of any plans to alter the current datatype engine implementation, nor any plans in the MPI forum to add more predefined types.

  George.

Here are the objects using this mechanism in the trunk right now:

$ find . -name "*.h" -exec grep -Hn "struct ompi_predefined_" {} \;| grep "{" | awk -Fstruct '{print $2;}'| awk -F{ '{print $1;}'
 ompi_predefined_communicator_t
 ompi_predefined_datatype_t
 ompi_predefined_errhandler_t
 ompi_predefined_file_t
 ompi_predefined_group_t
 ompi_predefined_info_t
 ompi_predefined_message_t
 ompi_predefined_op_t
 ompi_predefined_request_t
 ompi_predefined_win_t

More precisely:

3 communicators (world, self, null)
25 datatype (…)
4 error handlers (null, fatal, return, throw)
1 file (null)
1 group (null)
2 info (null, env)
1 message(no_proc)
14 op (…)
1 request (null)
1 window (null)

And here is the memory overhead per type:

(lldb) p sizeof(ompi_predefined_datatype_t) - sizeof(ompi_datatype_t)
(unsigned long) $11 = 40
(lldb) p sizeof(ompi_predefined_errhandler_t) - sizeof(ompi_errhandler_t)
(unsigned long) $12 = 864
(lldb) p sizeof(ompi_predefined_file_t) - sizeof(ompi_file_t)
(unsigned long) $13 = 744
(lldb) p sizeof(ompi_predefined_group_t) - sizeof(ompi_group_t)
(unsigned long) $14 = 160
(lldb) p sizeof(ompi_predefined_info_t) - sizeof(ompi_info_t)
(unsigned long) $15 = 112
(lldb) p sizeof(ompi_predefined_message_t) - sizeof(ompi_message_t)
(unsigned long) $16 = 144
(lldb) p sizeof(ompi_predefined_op_t) - sizeof(ompi_op_t)
(unsigned long) $17 = 624
(lldb) p sizeof(ompi_predefined_request_t) - sizeof(ompi_request_t)
(unsigned long) $18 = 64
(lldb) p sizeof(ompi_predefined_win_t) - sizeof(ompi_win_t)
(unsigned long) $19 = 200
(lldb) p sizeof(ompi_predefined_communicator_t) - sizeof(ompi_communicator_t)
(unsigned long) $20 = 336

  George.

On Jan 13, 2014, at 23:14 , Jeff Squyres (jsquyres) <jsquyres_at_[hidden]> wrote:

> Ok, fair enough -- I can remove it.
>
> But the question still remains: should we increase the size of the padding for the predefined datatypes? The idea is that it would need to be big enough for the entire 1.9/1.10 (2.0?) series.
>
>
> On Jan 13, 2014, at 4:16 PM, George Bosilca <bosilca_at_[hidden]>
> wrote:
>
>> The code is well protected against such issues. If one of our base structures become larger than corresponding predefined structure, the compiler should trigger an error similar to:
>>
>> In file included from ../../../ompi/ompi/datatype/ompi_datatype_create_contiguous.c:23:0:
>> ../../../ompi/ompi/datatype/ompi_datatype.h:105:10: error: size of array 'padding' is too large
>> char padding[PREDEFINED_DATATYPE_PAD - sizeof(ompi_datatype_t)];
>> ^
>>
>> This is the case for all types that have a predefined enveloppe. Thus, the commit 30268 was unnecessary as the check was already included in the compilation by the inclusion of the ompi_debugger_canary.c.
>>
>> George.
>>
>> On Jan 13, 2014, at 17:10 , Jeff Squyres (jsquyres) <jsquyres_at_[hidden]> wrote:
>>
>>> I noticed the following this morning:
>>>
>>> * sizeof(ompi_predefined_datatype_t): 512
>>> * sizeof(ompi_datatype_t): 488
>>> * padding left: 24
>>>
>>> Do we need to increase the size of padding for predefined communicators for v1.9?
>>>
>>> --
>>> Jeff Squyres
>>> jsquyres_at_[hidden]
>>> For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
>>>
>>> _______________________________________________
>>> devel mailing list
>>> devel_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>
>> _______________________________________________
>> devel mailing list
>> devel_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>
>
> --
> Jeff Squyres
> jsquyres_at_[hidden]
> For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel