Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] RFC: Support Cross Memory Attach in sm btl
From: George Bosilca (bosilca_at_[hidden])
Date: 2012-01-13 11:00:32


If the sys call are there we should clearly take advantage of them. The patch looks good, I vote for it!


On Jan 12, 2012, at 04:34 , Christopher Yeoh wrote:

> Hi Brad,
> WHAT: Adds Cross Memory Attach support to the sm btl
> WHY: For faster intranode communication
> WHERE: ompi/mca/btl/sm/
> WHEN: Open MPI trunk
> TIMEOUT: 13/2/2012
> Cross Memory Attach (CMA) is a pair of new syscalls (process_vm_readv
> and process_vm_writev) which allow for fast intranode
> communication. It has added to the Linux 3.2 kernel. There is a man page
> for the new system calls here:
> Attached is a patch for the OMPI trunk tree which augments the sm btl
> to use these calls.
> - CMA is quite similar in many respects to KNEM and what I've done is to
> pretty much copy what KNEM does in many cases.
> - Both KNEM and CMA can be compiled in at the same time, though if
> both are enabled at runtime, only KNEM runs. To enable CMA use
> --mca btl_sm_use_cma 1
> - To enable CMA at compile time, add --with-cma=yes to the configure
> command line. Support for the syscalls is in the git glibc archive,
> but its not yet out in the distros so as an interim
> workaround I have added some arch/os specific wrappers which are used
> if the syscalls are not found at configure time. The syscalls numbers
> won't change as 3.2 is out.
> - I'm far from sure that the way I have used CMA in OMPI is the best
> way to do it, so any feedback is very welcome.
> Regards,
> Chris
> --
> cyeoh_at_[hidden]
> <cma-openmpi-v1.patch>_______________________________________________
> devel mailing list
> devel_at_[hidden]