Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] RFC: Support Cross Memory Attach in sm btl
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2012-01-28 07:38:31


Also +1.

On Jan 13, 2012, at 11:00 AM, George Bosilca wrote:

> Chris,
>
> If the sys call are there we should clearly take advantage of them. The patch looks good, I vote for it!
>
> george.
>
> On Jan 12, 2012, at 04:34 , Christopher Yeoh wrote:
>
>> Hi Brad,
>>
>> WHAT: Adds Cross Memory Attach support to the sm btl
>>
>> WHY: For faster intranode communication
>>
>> WHERE: ompi/mca/btl/sm/
>>
>> WHEN: Open MPI trunk
>>
>> TIMEOUT: 13/2/2012
>>
>> Cross Memory Attach (CMA) is a pair of new syscalls (process_vm_readv
>> and process_vm_writev) which allow for fast intranode
>> communication. It has added to the Linux 3.2 kernel. There is a man page
>> for the new system calls here:
>>
>> http://ozlabs.org/~cyeoh/cma/process_vm_readv.txt
>>
>> Attached is a patch for the OMPI trunk tree which augments the sm btl
>> to use these calls.
>>
>> - CMA is quite similar in many respects to KNEM and what I've done is to
>> pretty much copy what KNEM does in many cases.
>>
>> - Both KNEM and CMA can be compiled in at the same time, though if
>> both are enabled at runtime, only KNEM runs. To enable CMA use
>> --mca btl_sm_use_cma 1
>>
>> - To enable CMA at compile time, add --with-cma=yes to the configure
>> command line. Support for the syscalls is in the git glibc archive,
>> but its not yet out in the distros so as an interim
>> workaround I have added some arch/os specific wrappers which are used
>> if the syscalls are not found at configure time. The syscalls numbers
>> won't change as 3.2 is out.
>>
>> - I'm far from sure that the way I have used CMA in OMPI is the best
>> way to do it, so any feedback is very welcome.
>>
>> Regards,
>>
>> Chris
>> --
>> cyeoh_at_[hidden]
>>
>>
>> <cma-openmpi-v1.patch>_______________________________________________
>> devel mailing list
>> devel_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/