Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] open ib btl and xrc
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2008-01-18 11:43:03

I think the main savings is that mellanox hardware works better when
fewer qp's are open. I.e., it's a resource issue on the HCA, not
necessarily a savings in posting buffers to the qp.

But it's quite a complicated issue. :-)

Gleb has some reservations about XRC; I'll let him expound on them...

On Jan 18, 2008, at 1:06 AM, Don Kerr wrote:

> Those pointers were perfect thanks.
> It easy to see the benefit of fewer qps (per node instead of per peer)
> and less consumption of resources the better but I am curious about
> the
> actual percentage of memory footprint decrease. I am thinking that the
> largest portion of the footprint comes from the fragments. Do you have
> any numbers showing the actual memory footprint savings when using
> xrc?
> Just to be clear, I am not asking for you or anyone else to generate
> these numbers, but if you had them already I would be curious to know
> the over all savings.
> -DON
> Pavel Shamis (Pasha) wrote:
>> Here is paper from openib
>> and here is mvapich presentation
>> Button line: XRC decrease number of QPs that ompi opens and as result
>> decrease ompi's memory footprint.
>> In the openib paper you may see more details about XRC. If you need
>> more
>> details about XRC implemention
>> in openib blt , please let me know.
>> Instead
>> Don Kerr wrote:
>>> Hi,
>>> After searching, about the only thing I can find on xrc is what it
>>> stands for, can someone explain the benefits of open mpi's use of
>>> xrc,
>>> maybe point me to a paper, or both?
>>> TIA
>>> -DON
>>> _______________________________________________
>>> devel mailing list
>>> devel_at_[hidden]
> _______________________________________________
> devel mailing list
> devel_at_[hidden]

Jeff Squyres
Cisco Systems