Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2005-12-07 09:04:02


I've updated http://www.open-mpi.org/faq/?category=infiniband#ib-
locked-pages -- thanks.

On Dec 6, 2005, at 8:27 PM, Brian Barrett wrote:

> Generally, the user defaults to the soft limit and can use ulimit to
> set values up to the hard limit. Tim's suggestion bumps the soft
> limit up to the hard limit, so all the user could do with ulimit is
> move the per-user limit back down.
>
> brian
>
> On Dec 6, 2005, at 3:35 PM, Jeff Squyres wrote:
>
>> With Tim's response to this -- I'm curious (so that we get correct
>> information on the FAQ) -- is the /etc/security/limit.conf method a
>> system-wide way of setting these values, and "ulimit -l" a per-user
>> way of setting it?
>>
>> That doesn't sound quite right to me -- I'm assuming that a user
>> can't "ulimit -l X" where X is larger than the numbers in /etc/
>> security/limits.conf -- can someone confirm if this is Right?
>>
>> Also, is each user's default "ulimit -l" value the same as what is
>> in /etc/security/limits.conf? (what's the difference between "hard"
>> and "soft" limits?)
>>
>>
>> On Dec 6, 2005, at 2:22 PM, Galen M. Shipman wrote:
>>
>>> Hi Daryl,
>>>
>>> Sounds like this might be a ulimit issue, what do you get when you
>>> run ulimit -l?
>>>
>>> Also, check out: http://www.open-mpi.org/faq/?category=infiniband
>>>
>>> Thanks,
>>>
>>> Galen
>>>
>>> On Dec 6, 2005, at 10:46 AM, Daryl W. Grunau wrote:
>>>
>>>> Hi, I'm running OMPI 1.1a1r8378 on 2.6.14 + recent OpenIB stack and
>>>> getting
>>>> the following runtime error:
>>>>
>>>> [0,1,0][btl_openib.c:803:mca_btl_openib_module_init] error creating
>>>> high priority cq for mthca0 errno says Cannot allocate memory
>>>> [0,1,3][btl_openib.c:803:mca_btl_openib_module_init] error creating
>>>> high priority cq for mthca0 errno says Cannot allocate memory
>>>> [0,1,1][btl_openib.c:803:mca_btl_openib_module_init] error creating
>>>> high priority cq for mthca0 errno says Cannot allocate memory
>>>> [0,1,2][btl_openib.c:803:mca_btl_openib_module_init] error creating
>>>> high priority cq for mthca0 errno says Cannot allocate memory
>>>>
>>>>
>>>> Strange thing is that it works properly when I run as root. A
>>>> permissions
>>>> problem on my part? My devices look like:
>>>>
>>>> # ls -l /dev/infiniband/*
>>>> crw------- 1 root root 231, 64 Dec 5 17:16 /dev/infiniband/
>>>> issm0
>>>> crw------- 1 root root 231, 65 Dec 5 17:16 /dev/infiniband/
>>>> issm1
>>>> crw------- 1 root root 231, 0 Dec 5 17:16 /dev/infiniband/
>>>> umad0
>>>> crw------- 1 root root 231, 1 Dec 5 17:16 /dev/infiniband/
>>>> umad1
>>>> crw-rw-rw- 1 root root 231, 192 Dec 5 17:16 /dev/infiniband/
>>>> uverbs0
>>>>
>>>> Daryl
>>>>
>>>> --
>>>> ***** Correspondence *****
>>>> _______________________________________________
>>>> users mailing list
>>>> users_at_[hidden]
>>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>>
>>> _______________________________________________
>>> users mailing list
>>> users_at_[hidden]
>>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>>
>>
>> --
>> {+} Jeff Squyres
>> {+} The Open MPI Project
>> {+} http://www.open-mpi.org/
>>
>>
>>
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users

--
{+} Jeff Squyres
{+} The Open MPI Project
{+} http://www.open-mpi.org/