Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] knem/openmpi performance?
From: Mark Dixon (m.c.dixon_at_[hidden])
Date: 2013-07-12 07:44:13

On Fri, 12 Jul 2013, Jeff Squyres (jsquyres) wrote:
> In short: doing 1 memcopy consumes half the memory bandwidth of 2 mem
> copies. So when you have lots of MPI processes competing for memory
> bandwidth, it turns out that having each MPI process use half the
> bandwidth is a Really Good Idea. :-) This allows more MPI processes to
> do shared memory communications before you hit the memory bandwidth
> bottleneck.

Hi Jeff,

Lots of useful detail in there - thanks. We have plenty of memory-bound
applications in use, so hopefully there's some good news in this.

I was hoping that someone might have some examples of real application
behaviour rather than micro benchmarks. It can be crazy hard to get that
information from users.

Unusually for us, we're putting in a second cluster with the same
architecture, CPUs, memory and OS as the last one. I might be able to use
this as a bigger stick to get some better feedback. If so, I'll pass it

> Darius Buntinas, Brice Goglin, et al. wrote an excellent paper about
> exactly this set of issues; see

I'll definitely take a look - thanks again.

All the best,


Mark Dixon                       Email    : m.c.dixon_at_[hidden]
HPC/Grid Systems Support         Tel (int): 35429
Information Systems Services     Tel (ext): +44(0)113 343 5429
University of Leeds, LS2 9JT, UK