Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Open MPI User's mailing list

Subject: Re: [OMPI users] mem lock limit & ulimit
From: Jeff Squyres (jsquyres_at_[hidden])
Date: 2010-08-18 16:23:33


If you're using IB for HPC/MPI, you probably want an unlimited memlock limit.

Keep in mind that you need to start the slurm daemons with the desired memlock limit, too -- if they're started via init.d, for example, they won't be subject to /etc/security/limits.conf.

On Aug 18, 2010, at 12:08 PM, Ralph Castain wrote:

> ulimit basically states that a normal user can increase a specific limit up to the maximum specified by root, either via system limit settings or during configuration of the OS.
>
> So the 256 limit you are seeing is one set locally. I would suggest asking your sys admin for a higher limit if you require one.
>
>
> On Aug 18, 2010, at 12:17 PM, stabeek wrote:
>
>> Hello,
>> I've done a good bit of reading and still cannot get my mpirun programs to execute without giving warnings about the memory lock limit not being "unlimited".
>> I'm on an infiniband-(openib)-networked cluster under slurm resource mgmt using openmpi.
>> Initially I did not want to set max memlock to unlimited because of security concerns, so I concentrated on having slurm negate this limit via its PropagateResourceLimitsExcept=MEMLOCK setting.
>> There was little success with this, and then on reading the MPI FAQ @openmpi for a second time, and seeing that openmpi's source code actually recurs to libc's getrlimit() to ascertain the memlock limit, I decided this setting was not-relevant for me. This is hunch as yet, it would be nice to get it verified, though it won't help me resolve the problem really.
>> Now I'm faced with having to ask about ulimit on a mailing which is not about ulimit, but I'm hoping for some people's experience here:
>> "ulimit -l unlimited" works if:
>> 1) the /etc/security/limits.conf file has unlimited soft and hard limits.
>> 2) if you are root.
>> These conditions solve the problem. But the 2nd is not feasible as should be clear. The problem is that for a normal user running ulimit does not allow a memlock max setting above 256. Its behaviour is one of:
>> "Let normaluser increase max memlock only up to 256"
>> Can anybody shed any light or make any suggestions, it would be appreciated. Many thanks.
>> _______________________________________________
>> users mailing list
>> users_at_[hidden]
>> http://www.open-mpi.org/mailman/listinfo.cgi/users
>
>
> _______________________________________________
> users mailing list
> users_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/users

-- 
Jeff Squyres
jsquyres_at_[hidden]
For corporate legal information go to:
http://www.cisco.com/web/about/doing_business/legal/cri/