Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] Question on handling of memory for communications
From: Michael Thomadakis (drmichaelt7777_at_[hidden])
Date: 2013-07-08 17:29:50


| Remember that the point of IB and other operating-system bypass devices
is that the driver is not involved in the fast path of sending /
| receiving. One of the side-effects of that design point is that
userspace does all the allocation of send / receive buffers.

That's a good point. It was not clear to me who and with what logic was
allocating memory. But definitely for IB it makes sense that the user
provides pointers to their memory.

thanks
Michael

On Mon, Jul 8, 2013 at 1:07 PM, Jeff Squyres (jsquyres)
<jsquyres_at_[hidden]>wrote:

> On Jul 8, 2013, at 2:01 PM, Brice Goglin <Brice.Goglin_at_[hidden]> wrote:
>
> > The driver doesn't allocate much memory here. Maybe some small control
> buffers, but nothing significantly involved in large message transfer
> performance. Everything critical here is allocated by user-space (either
> MPI lib or application), so we just have to make sure we bind the process
> memory properly. I used hwloc-bind to do that.
>
> +1
>
> Remember that the point of IB and other operating-system bypass devices is
> that the driver is not involved in the fast path of sending / receiving.
> One of the side-effects of that design point is that userspace does all
> the allocation of send / receive buffers.
>
> --
> Jeff Squyres
> jsquyres_at_[hidden]
> For corporate legal information go to:
> http://www.cisco.com/web/about/doing_business/legal/cri/
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>