Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] RFC: Support Cross Memory Attach in sm btl
From: Christopher Yeoh (cyeoh_at_[hidden])
Date: 2012-01-12 19:47:00


Hi Chris,

On Fri, 13 Jan 2012 09:54:20 +1100
Christopher Samuel <samuel_at_[hidden]> wrote:
>
> On 12/01/12 20:34, Christopher Yeoh wrote:
>
> > Cross Memory Attach (CMA) is a pair of new syscalls
> > (process_vm_readv and process_vm_writev) which allow for fast
> > intranode communication. It has added to the Linux 3.2 kernel.
>
> Do you have any figures comparing some code with and without CMA ?
>

Here's some benchmarking results I did a while back on a single 64-way
(SMT) POWER6 box.

HPCC results:
=============

MB/s Num Processes
Naturally Ordered 4 8 16 32
Base 1235 935 622 419
CMA 4741 3769 1977 703

                        
MB/s Num Processes
Randomly Ordered 4 8 16 32
Base 1227 947 638 412
CMA 4666 3682 1978 710
                                
MB/s Num Processes
Max Ping Pong 4 8 16 32
Base 2028 1938 1928 1882
CMA 7424 7510 7598 7708

NPB:
====
BT - 12% improvement
FT - 15% improvement
IS - 30% improvement
SP - 34% improvement

IMB:
===
                
Ping Pong - ~30% improvement
Ping Ping - ~120% improvement
SendRecv - ~100% improvement
Exchange - ~150% improvement
Gather(v) - ~20% improvement
Scatter(v) - ~20% improvement
AlltoAll(v) - 30-50% improvement

Regards,

Chris

-- 
cyeoh_at_[hidden]