Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] autosizing the shared memory backing file
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2009-01-14 09:43:41

Ya, that does seem weird to me, but I never fully grokked the whole
mpool / allocator scheme (I haven't had to interact with that part of
the code much).

Would it be useful to get on the phone and discuss this stuff?

On Jan 14, 2009, at 1:11 AM, Eugene Loh wrote:

> Thanks for the reply. I kind of understand, but it's rather weird.
> The BTL calls mca_mpool_base_module_create() to create a pool of
> memory, but the BTL has no say how big of a pool to create? Could
> you imagine having a memory allocation routine ("malloc" or
> something) that didn't allow you to control the size of the
> allocation? Instead, the allocation routine determines the size.
> That's weird. I must be missing something about how this is
> supposed to work.
> E.g., I see that there is a "resources" argument
> (mca_mpool_base_resources_t). Maybe that structure should be
> expanded to include a "size" field?
> Or, maybe I should bypass mca_mpool_base_module_create()/
> mca_mpool_sm_init() and just call
> mca_common_sm_mmap_init() directly, the way mca/coll/sm does
> things. That would allow me to specify the size of the file.
> George Bosilca wrote:
>> The simple answer is you can't. The mpool is loaded before the
>> BTLs and on Linux the loader use the RTLD_NOW flag (i.e. all
>> symbols have to be defined or the dlopen call will fail).
>> Moreover, there is no way in Open MPI to exchange information
>> between components except a global variable or something in the
>> mca/common. In other words there is no way for you to call from
>> the mpool a function from the sm BTL.
>> On Jan 13, 2009, at 19:22 , Eugene Loh wrote:
>>> With the sm BTL, there is a file that each process mmaps in for
>>> shared memory.
>>> I'm trying to get mpool_sm to size the file appropriately. So, I
>>> would like mpool_sm to call some mca_btl_sm function that provides
>>> a good guess of the size. (mpool_sm creates and mmaps the file,
>>> but the size depends on parameters like eager limit and max frag
>>> size that are known by the btl_sm.)
> _______________________________________________
> devel mailing list
> devel_at_[hidden]

Jeff Squyres
Cisco Systems