Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] open ib btl and xrc
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2008-01-18 11:43:03


I think the main savings is that mellanox hardware works better when
fewer qp's are open. I.e., it's a resource issue on the HCA, not
necessarily a savings in posting buffers to the qp.

But it's quite a complicated issue. :-)

Gleb has some reservations about XRC; I'll let him expound on them...

On Jan 18, 2008, at 1:06 AM, Don Kerr wrote:

> Those pointers were perfect thanks.
>
> It easy to see the benefit of fewer qps (per node instead of per peer)
> and less consumption of resources the better but I am curious about
> the
> actual percentage of memory footprint decrease. I am thinking that the
> largest portion of the footprint comes from the fragments. Do you have
> any numbers showing the actual memory footprint savings when using
> xrc?
> Just to be clear, I am not asking for you or anyone else to generate
> these numbers, but if you had them already I would be curious to know
> the over all savings.
>
> -DON
>
> Pavel Shamis (Pasha) wrote:
>> Here is paper from openib http://www.openib.org/archives/nov2007sc/XRC.pdf
>> and here is mvapich presentation
>> http://mvapich.cse.ohio-state.edu/publications/ofa_nov07-mvapich-xrc.pdf
>>
>> Button line: XRC decrease number of QPs that ompi opens and as result
>> decrease ompi's memory footprint.
>> In the openib paper you may see more details about XRC. If you need
>> more
>> details about XRC implemention
>> in openib blt , please let me know.
>>
>>
>> Instead
>> Don Kerr wrote:
>>
>>> Hi,
>>>
>>> After searching, about the only thing I can find on xrc is what it
>>> stands for, can someone explain the benefits of open mpi's use of
>>> xrc,
>>> maybe point me to a paper, or both?
>>>
>>> TIA
>>> -DON
>>>
>>> _______________________________________________
>>> devel mailing list
>>> devel_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/devel
>>>
>>>
>>>
>>
>>
>>
> _______________________________________________
> devel mailing list
> devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/devel

-- 
Jeff Squyres
Cisco Systems