With Jeff's latest changes to how we set up the cq_size I am now seeing error messages saying that my machine's memlocked limits are too low.  I am concerned that it might be something else because my max'd locked memory is unlimited on my machine.

So if I do a run of -np 2 across two separate node than the use of the max_cqe of my ib device (4194303) is ok.  Once I go beyond 1 process on the node I start getting the memlocked limits message.  So how much memory does a cqe take?  Is it 1k by any chance?  I ask this because the machine I am running on has 4GB of memory and so I am wondering if I just don't have enough backing memory and if that is so I am wondering how commone of a case this may be?

Terry D. Dontje | Principal Software Engineer
Developer Tools Engineering | +1.781.442.2631
Oracle - Performance Technologies
95 Network Drive, Burlington, MA 01803
Email terry.dontje@oracle.com