Open MPI logo

Open MPI Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Development mailing list

Subject: Re: [OMPI devel] openib max_cqe
From: Shamis, Pavel (shamisp_at_[hidden])
Date: 2012-07-05 15:53:52

> So if I do a run of -np 2 across two separate node than the use of the max_cqe of my ib device (4194303) is ok. Once I go beyond 1 process on the node I start getting the memlocked limits message. So how much memory does a cqe take? Is it 1k by any chance? I ask this because the machine I am running on has 4GB of memory and so I am wondering if I just don't have enough backing memory and if that is so I am wondering how commone of a case this may be?

I mentioned on the call that for Mellanox devices (+OFA verbs) this resource is really cheap. Do you run mellanox hca + OFA verbs ?