Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] Infiniband memory usage with XRC
From: Pavel Shamis (Pasha) (pashash_at_[hidden])
Date: 2010-05-17 11:21:54

Sylvain Jeaugey wrote:
>>> The XRC protocol seems to create shared receive queues, which is a
>>> good thing. However, comparing memory used by an "X" queue versus
>>> and "S" queue, we can see a large difference. Digging a bit into the
>>> code, we found some
>> So, do you see that X consumes more that S ? This is really odd.
> Yes, but that's what we see. At least after MPI_Init.
What is the difference (in Kb)?
>>> strange things, like the completion queue size not being the same as
>>> "S" queues (the patch below would fix it, but the root of the
>>> problem may be elsewhere).
>>> Is anyone able to comment on this ?
>> The fix looks ok, please submit it to trunk.
> I don't have an account to do this, so I'll let maintainers push it
> into SVN.
Ok, I will push it.
>> BTW do you want to prepare the patch for send queue size factor ? It
>> should be quite simple.
> Maybe we can do this. However, we are a little playing with parameters
> and code without really knowing the deep consequences of what we do.
> Therefore, I would feel more confortable if someone who knows much on
> the openib btl confirms it's not breaking everything.
Well, please feel free to submit a patch for review. Also if you see any
other issues with XRC, MLNX will be happy to help.