Open MPI Development Mailing List Archives

samuel_at_[hidden]

--
Samuel K. Gutierrez
Los Alamos National Laboratory
On Aug 11, 2010, at 1:15 PM, Samuel K. Gutierrez wrote:
> Hi Terry,
>
>
>
>
>
>
>
>
> On Aug 11, 2010, at 12:34 PM, Terry Dontje wrote:
>
>> I've done some minor testing on Linux looking at resident and  
>> shared memory sizes for np=4, 8 and 16 jobs.  I could not see any  
>> appreciable differences in sizes in the process between sysv, posix  
>> or mmap usage in the SM btl.
>>
>> So I am still somewhat non-plussed about making this the default.   
>> It seems like the biggest gain out of using posix might be one  
>> doesn't need to worry about the location of the backing file.  This  
>> seems kind of frivolous to me since there is a warning that happens  
>> if the backing file is not memory based.
>
> If I'm not mistaken, the warning is only issued if the backing files  
> is stored on the following file systems: Lustre, NFS, Panasas, and  
> GPFS  (see: opal_path_nfs in opal/util/path.c).  Based on the  
> performance numbers that Sylvain provided on June 9th of this year  
> (see attached),  there was a performance difference between mmap on / 
> tmp and mmap on a tmpfs-like file system (/dev/shm in that  
> particular case).  Using the new POSIX component provides us with a  
> portable way to provide similar shared memory performance gains  
> without having to worry about where the OMPI session directory is  
> rooted.
>
> --
> Samuel K. Gutierrez
> Los Alamos National Laboratory
>
> <sm-compared.png>
>
>>
>> I still working on testing the code on Solaris but I don't imagine  
>> I will see anything that will change my mind.
>>
>> --td
>>
>> Samuel K. Gutierrez wrote:
>>>
>>> Hi Rich,
>>>
>>> It's a modification to the existing common sm component.  The  
>>> modifications do include the addition of a new POSIX shared memory  
>>> facility, however.
>>>
>>> Sam
>>>
>>> On Aug 11, 2010, at 10:05 AM, Graham, Richard L. wrote:
>>>
>>>> Is this a modification of the existing component, or a new  
>>>> component ?
>>>>
>>>> Rich
>>>>
>>>>
>>>> On 8/10/10 10:52 AM, "Samuel K. Gutierrez" <samuel_at_[hidden]> wrote:
>>>>
>>>> Hi,
>>>>
>>>> I wanted to give everyone a heads-up about a new POSIX shared  
>>>> memory
>>>> component
>>>> that has been in the works for a while now and is ready to be  
>>>> pushed
>>>> into the
>>>> trunk.
>>>>
>>>> http://bitbucket.org/samuelkgutierrez/ompi_posix_sm_new
>>>>
>>>> Some highlights:
>>>> o New posix component now the new default.
>>>>        o May address some of the shared memory performance issues  
>>>> users
>>>> encounter
>>>>           when OMPI's session directories are inadvertently  
>>>> placed on a non-
>>>> local
>>>>           filesystem.
>>>> o Silent component failover.
>>>>        o In the default case, if the posix component fails  
>>>> initialization,
>>>>           mmap will be selected.
>>>> o The sysv component will only be queried for selection if it is
>>>> placed before
>>>>    the mmap component (for example, -mca mpi_common_sm
>>>> sysv,posix,mmap).  In the
>>>>    default case, sysv will never be queried/selected.
>>>> o Per some on-list discussion, now unlinking mmaped file in both  
>>>> mmap
>>>> and posix
>>>>    components (see: "System V Shared Memory for Open MPI: Request  
>>>> for
>>>> Community
>>>>    Input and Testing" thread).
>>>> o  Assuming local process homogeneity with respect to all utilized
>>>> shared
>>>>     memory facilities. That is, if one local process deems a
>>>> particular shared
>>>>     memory facility acceptable, then ALL local processes should be
>>>> able to
>>>>     utilize that facility. As it stands, this is an important point
>>>> because one
>>>>     process dictates to all other local processes which common sm
>>>> component will
>>>>     be selected based on its own, local run-time test.
>>>> o Addressed some of George's code reuse concerns.
>>>>
>>>> If there are no major objections by August 17th, I'll commit the  
>>>> code
>>>> after the
>>>> Tuesday morning conference call.
>>>>
>>>> Thanks!
>>>>
>>>> -- 
>>>> Samuel K. Gutierrez
>>>> Los Alamos National Laboratory
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> devel mailing list
>>>> devel_at_[hidden]
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>>
>>>>
>>>> _______________________________________________
>>>> devel mailing list
>>>> devel_at_[hidden]
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>
>>> _______________________________________________
>>> devel mailing list
>>> devel_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>
>>
>> -- 
>> <mime-attachment.gif>
>> Terry D. Dontje | Principal Software Engineer
>> Developer Tools Engineering | +1.650.633.7054
>> Oracle - Performance Technologies
>> 95 Network Drive, Burlington, MA 01803
>> Email terry.dontje_at_[hidden]
>>
>> _______________________________________________
>> devel mailing list
>> devel_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel