Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] System V Shared Memory for Open MPI: Request for Community Input and Testing
From: Sylvain Jeaugey (sylvain.jeaugey_at_[hidden])
Date: 2010-06-09 11:58:11


As stated at the conf call, I did some performance testing on a 32 cores
node.

So, here is graph showing 500 timings of an allreduce operation (repeated
15,000 times for good timing) with sysv, mmap on /dev/shm and mmap on
/tmp.

What is shows :
  - sysv has the better performance ;
  - having the mmap file in /dev/shm is very close to sysv. We only have
+0.1 us for a complete allreduce operation, but it seems stable. The noise
is identical to sysv (must be OS noise) ;
  - having the mmap file in /tmp (ext3) decreases performance (+0.4 us
compared to /dev/shm) and seems prone to some "other" noise.

Warning : the graph does not start at 0.

Sylvain

On Tue, 27 Apr 2010, Samuel K. Gutierrez wrote:

> Hi,
>
> With Jeff and Ralph's help, I have completed a System V shared memory
> component for Open MPI. I have conducted some preliminary tests on our
> systems, but would like to get test results from a broader audience.
>
> As it stands, mmap is the defaul, but System V shared memory can be activated
> using: -mca mpi_common_sm sysv
>
> Repository:
> http://bitbucket.org/samuelkgutierrez/ompi_sysv_sm
>
> Input is greatly appreciated!
>
> --
> Samuel K. Gutierrez
> Los Alamos National Laboratory
>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>


sm-compared.png