Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] autosizing the shared memory backing file
From: Eugene Loh (Eugene.Loh_at_[hidden])
Date: 2009-01-14 01:11:00


Thanks for the reply. I kind of understand, but it's rather weird. The
BTL calls mca_mpool_base_module_create() to create a pool of memory, but
the BTL has no say how big of a pool to create? Could you imagine
having a memory allocation routine ("malloc" or something) that didn't
allow you to control the size of the allocation? Instead, the
allocation routine determines the size. That's weird. I must be
missing something about how this is supposed to work.

E.g., I see that there is a "resources" argument
(mca_mpool_base_resources_t). Maybe that structure should be expanded
to include a "size" field?

Or, maybe I should bypass
mca_mpool_base_module_create()/mca_mpool_sm_init() and just call
mca_common_sm_mmap_init() directly, the way mca/coll/sm does things.
That would allow me to specify the size of the file.

George Bosilca wrote:

> The simple answer is you can't. The mpool is loaded before the BTLs
> and on Linux the loader use the RTLD_NOW flag (i.e. all symbols have
> to be defined or the dlopen call will fail).
>
> Moreover, there is no way in Open MPI to exchange information between
> components except a global variable or something in the mca/common.
> In other words there is no way for you to call from the mpool a
> function from the sm BTL.
>
> On Jan 13, 2009, at 19:22 , Eugene Loh wrote:
>
>> With the sm BTL, there is a file that each process mmaps in for
>> shared memory.
>>
>> I'm trying to get mpool_sm to size the file appropriately. So, I
>> would like mpool_sm to call some mca_btl_sm function that provides a
>> good guess of the size. (mpool_sm creates and mmaps the file, but
>> the size depends on parameters like eager limit and max frag size
>> that are known by the btl_sm.)
>