Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: Re: [OMPI users] OMPI Coll Framework and RDMA
From: Jeff Squyres (jsquyres) (jsquyres_at_[hidden])
Date: 2013-06-07 18:53:52


http://www.open-mpi.org/papers/euro-pvmmpi-2007-ib/

On Jun 7, 2013, at 11:11 AM, Jingcha Joba <pukkimonkey_at_[hidden]> wrote:

> Interesting.
>
> I would like to understand more on how QP implementation in OpenMPI, for example, heuristics behind creating multiple QPs between two mpi processes.
>
> Is there any whitepaper / reference / manual that I can refer to for that?
> Or can you point me to the source code region for this?
>
> Thanks,
> --Joba
>
>
> On Fri, Jun 7, 2013 at 8:09 AM, Shamis, Pavel <shamisp_at_[hidden]> wrote:
>
> Does that mean, if there is a AllGatherV and assuming that every process belongs to default comm, there will n-1 Queue Pair between the collecting process and other processes ?
> n = total number of MPI processes.
>
> The answer depends on multiple parameters, like number of processes, message size, etc. Some algorithms will require O(log(n)) connections other O(n).
> Also on OpenIB btl level per rank we create multiple QPs , not just one. To make things even more complicated :-) there are multiple types of QPs, like RC and XRC.
>
> -Pasha
>
>
>
> --
> Joba
>
> On Thu, Jun 6, 2013 at 3:37 PM, Shamis, Pavel <shamisp_at_[hidden]<mailto:shamisp_at_[hidden]>> wrote:
> Default implementation of collectives is based on PML (p2p layer) that is implemented on top of BTL.
> Consequently it laverages RDMA capabilities to some extend.
>
> Pavel (Pasha) Shamis
> ---
> Computer Science Research Group
> Computer Science and Math Division
> Oak Ridge National Laboratory
>
>
>
>
>
>
> On Jun 6, 2013, at 1:59 PM, Jingcha Joba <pukkimonkey_at_[hidden]<mailto:pukkimonkey_at_[hidden]><mailto:pukkimonkey_at_[hidden]<mailto:pukkimonkey_at_[hidden]>>> wrote:
>
> Hi,
>
> I have a quick question.
>
> Is there an openib (in btl framework) equivalent in coll framework?
>
> I have an MPI application with gatherv and scatterv. I am wondering if I can leverage RDMA capabilities of the underlying Infiniband fabric.
>
>
> Thanks,
> --
> Joba
>
> _______________________________________________
> users mailing list
> users_at_[hidden]<mailto:users_at_[hidden]><mailto:users_at_[hidden]<mailto:users_at_[hidden]>>
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]<mailto:users_at_[hidden]>
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]<mailto:users_at_[hidden]>
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to: http://www.cisco.com/web/about/doing_business/legal/cri/