Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Question on handling of memory for communications
From: Jeff Squyres (jsquyres) (jsquyres_at_[hidden])
Date: 2013-07-08 14:07:06


On Jul 8, 2013, at 2:01 PM, Brice Goglin <Brice.Goglin_at_[hidden]> wrote:

> The driver doesn't allocate much memory here. Maybe some small control buffers, but nothing significantly involved in large message transfer performance. Everything critical here is allocated by user-space (either MPI lib or application), so we just have to make sure we bind the process memory properly. I used hwloc-bind to do that.

+1

Remember that the point of IB and other operating-system bypass devices is that the driver is not involved in the fast path of sending / receiving. One of the side-effects of that design point is that userspace does all the allocation of send / receive buffers.

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/