Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2006-11-03 07:49:49


You shouldn't have to do anything special; just write normal MPI
programs. There are a variety of MPI tutorials available on the web;
a particularly good one is available here:

     http://webct.ncsa.uiuc.edu:8900/public/MPI/

I think someone mentioned that you can use MPI_ALLOC_MEM to get pre-
registered memory (also as pointed out on this thread, "registered"
is typically more than just "pinning" -- it frequently also means
notifying the NIC of the pinned memory). You can also experiment
with using the MCA parameter mpi_leave_pinned (e.g., set it to 1) if
you are unable to use MPI_ALLOC_MEM to get pre-registered memory in
conjunction with your DMA-able PCIx devices.

On Nov 2, 2006, at 10:30 PM, Brian Budge wrote:

> Ha, yeah, I should have been more clear there. I'm simply writing
> an MPI application.
>
> Thanks,
> Brian
>
> On 11/2/06, Jeff Squyres < jsquyres_at_[hidden]> wrote:It depends on
> what you're trying to do. Are you writing new
> components internal to Open MPI, or are you just trying to leverage
> OMPI's PML for some other project? Or are you writing MPI
> applications? Or ...?
>
>
> On Nov 2, 2006, at 2:22 PM, Brian Budge wrote:
>
> > Thanks for the pointer, it was a very interesting read.
> >
> > It seems that by default OpenMPI uses the nifty pipelining trick
> > with pinning pages while transfer is happening. Also the pinning
> > can be (somewhat) perminant and the state is cached so that next
> > usage requires no registration. I guess it is possible to use pre-
> > pinned memory, but do I need to do anything special to do so? I
> > will already have some buffers pinned to allow DMAs to devices
> > across PCI-Express, so it makes sense to use one pinned buffer so
> > that I can avoid memcpys.
> >
> > Are there any HOWTO tutorials or anything? I've searched around,
> > but it's possible I just used the wrong search terms.
> >
> > Thanks,
> > Brian
> >
> >
> >
> > On 11/2/06, Jeff Squyres <jsquyres_at_[hidden] > wrote: This paper
> > explains it pretty well:
> >
> > http://www.open-mpi.org/papers/euro-pvmmpi-2006-hpc-protocols/
> >
> >
> >
> > On Nov 2, 2006, at 1:37 PM, Brian Budge wrote:
> >
> > > Hi all -
> > >
> > > I'm wondering how DMA is handled in OpenMPI when using the
> > > infiniband protocol. In particular, will I get a speed gain if my
> > > read/write buffers are already pinned via mlock?
> > >
> > > Thanks,
> > > Brian
> > > _______________________________________________
> > > users mailing list
> > > users_at_[hidden]
> > > http://www.open-mpi.org/mailman/listinfo.cgi/users
> >
> >
> > --
> > Jeff Squyres
> > Server Virtualization Business Unit
> > Cisco Systems
> >
> > _______________________________________________
> > users mailing list
> > users_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
> >
> > _______________________________________________
> > users mailing list
> > users_at_[hidden]
> > http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> --
> Jeff Squyres
> Server Virtualization Business Unit
> Cisco Systems
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users

-- 
Jeff Squyres
Server Virtualization Business Unit
Cisco Systems