Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] RFC: System V Shared Memory for Open MPI
From: George Bosilca (bosilca_at_[hidden])
Date: 2010-06-02 10:44:12


On Jun 2, 2010, at 09:28 , Jeff Squyres wrote:

> On Jun 2, 2010, at 5:38 AM, George Bosilca wrote:
>
>> I think adding support for sysv shared memory is a good thing. However, I have some strong objections over the implementation in the hg tree. Here are 2 of the major ones:
>>
>> 1) the sysv shared memory creation is __atomic__ based on the flags used. Therefore, all the RML messages exchange is totally useless.
>
> Not sure what you mean here. common/sm may create new shmem segments at any time (e.g., during coll sm). The RML message exchange is to ensure that only 1 process creates and initializes the segment and then all the others just attach to it.

Absolutely not! The RML messaging is not about initializing the shared memory segment. As stated on my original text it has only one purpose: to ensure the file used by mmap is created atomically. The code for Windows do not exchange any RML messages as the function to allocate the shared memory region provided by the OS is atomic (exactly as the sysv one).

> The initializing of the segment after it is created/attached could be pipelined a little more. E.g, since the init has an atomicly-set flag indicating when it's done, the root could create the seg, signal the others that they can attach, and then do the init -- the non-root procs can wait for flag to change atomicly to know when the seg has been initialized). Is that what you're referring to?

This is actually how the whole stuff is working today. As an example look at the sm BTL in file btl_sm.c line 541.

  george.

>
>> 2) the whole code is replicated in the 3 files (mmap, sysv and windows), even the common parts. However in the sysv case most of the comments have been modified to remove all capitals letter. I'm in favor of extracting all the common parts and moving them in a special file. What should be kept in the particular files should only be the really different parts (small part of the init and finalize).
>
> Sam -- are the common parts really common? I.e., could they be factored out? Or are they "just different enough" that factoring them out would be a PITA?
>
> --
> Jeff Squyres
> jsquyres_at_[hidden]
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
>
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel