Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] System V Shared Memory for Open MPI:Request for Community Input and Testing
From: Samuel K. Gutierrez (samuel_at_[hidden])
Date: 2010-05-03 19:15:48


Hi All,

New configure-time test added - thanks for the suggestion, Jeff.
Update and give it a whirl.

Ethan - could you please try again? This time, I'm hoping sysv
support will be disabled ;-).

Thanks!

--
Samuel K. Gutierrez
Los Alamos National Laboratory
On May 3, 2010, at 9:18 AM, Samuel K. Gutierrez wrote:
> Hi Jeff,
>
> Sounds like a plan :-).
>
> Thanks!
>
> --
> Samuel K. Gutierrez
> Los Alamos National Laboratory
>
> On May 3, 2010, at 9:12 AM, Jeff Squyres wrote:
>
>> It might well be that you need a configure test to determine  
>> whether this behavior occurs or not.  Heck, it may even need to be  
>> a run-time test!  Hrm.
>>
>> Write a small C program that does something like the following  
>> (this is off the top of my head):
>>
>> fork a child
>> child goes to sleep immediately
>> sysv alloc a segment
>> attach to it
>> ipc rm it
>> parent wakes up child
>> child tries to attach to segment
>>
>> If that succeeds, then all is good.  If not, then don't use this  
>> stuff.
>>
>>
>> On May 3, 2010, at 10:55 AM, Samuel K. Gutierrez wrote:
>>
>>> Hi all,
>>>
>>> Does anyone know of a relatively portable solution for querying a
>>> given system for the shmctl behavior that I am relying on, or is  
>>> this
>>> going to be a nightmare?  Because, if I am reading this thread
>>> correctly, the presence of shmget and Linux is not sufficient for
>>> determining an adequate level of sysv support.
>>>
>>> Thanks!
>>>
>>> --
>>> Samuel K. Gutierrez
>>> Los Alamos National Laboratory
>>>
>>> On May 2, 2010, at 7:48 AM, N.M. Maclaren wrote:
>>>
>>>> On May 2 2010, Ashley Pittman wrote:
>>>>> On 2 May 2010, at 04:03, Samuel K. Gutierrez wrote:
>>>>>
>>>>> As to performance there should be no difference in use between  
>>>>> sys-
>>>>> V shared memory and file-backed shared memory, the instructions
>>>>> issued and the MMU flags for the page should both be the same so
>>>>> the performance should be identical.
>>>>
>>>> Not necessarily, and possibly not so even for far-future Linuces.
>>>> On at least one system I used, the poxious kernel wrote the  
>>>> complete
>>>> file to disk before returning - all right, it did that for System V
>>>> shared memory, too, just to a 'hidden' file!  But, if I recall, on
>>>> another it did that only for file-backed shared memory - however,  
>>>> it's
>>>> a decade ago now and I may be misremembering.
>>>>
>>>> Of course, that's a serious issue mainly for large segments.  I was
>>>> using multi-GB ones.  I don't know how big the ones you need are.
>>>>
>>>>> The one area you do need to keep an eye on for performance is on
>>>>> numa machines where it's important which process on a node touches
>>>>> each page first, you can end up using different areas (pages, not
>>>>> regions) for communicating in different directions between the  
>>>>> same
>>>>> pair of processes. I don't believe this is any different to mmap
>>>>> backed shared memory though.
>>>>
>>>> On some systems it may be, but in bizarre, inconsistent,  
>>>> undocumented
>>>> and unpredictable ways :-(  Also, there are usually several system
>>>> (and
>>>> sometimes user) configuration options that change the behaviour, so
>>>> you
>>>> have to allow for that.  My experience of trying to use those is  
>>>> that
>>>> different uses have incompatible requirements, and most of the
>>>> critical
>>>> configuration parameters apply to ALL uses!
>>>>
>>>> In my view, the configuration variability is the number one  
>>>> nightmare
>>>> for trying to write portable code that uses any form of shared  
>>>> memory.
>>>> ARMCI seem to agree.
>>>>
>>>>>> Because of this, sysv support may be limited to Linux systems -
>>>>>> that is,
>>>>>> until we can get a better sense of which systems provide the  
>>>>>> shmctl
>>>>>> IPC_RMID behavior that I am relying on.
>>>>
>>>> And, I suggest, whether they have an evil gotcha on one of the  
>>>> areas
>>>> that
>>>> Ashley Pittman noted.
>>>>
>>>>
>>>> Regards,
>>>> Nick Maclaren.
>>>>
>>>>
>>>> _______________________________________________
>>>> devel mailing list
>>>> devel_at_[hidden]
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>
>>> _______________________________________________
>>> devel mailing list
>>> devel_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>
>>
>>
>> -- 
>> Jeff Squyres
>> jsquyres_at_[hidden]
>> For corporate legal information go to:
>> http://www.cisco.com/web/about/doing_business/legal/cri/
>>
>>
>> _______________________________________________
>> devel mailing list
>> devel_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel