Various people contributed:
>>>> Isn't it possible to set this up in torque/moab directly? In SGE I would simply define h_vmem and it's per slot then; and with a tight integration all Open MPI processes will be children of sge_execd and the limit will be enforced.
>>> I could be wrong, but I -think- the issue here is that the soft limits need to be set on a per-job basis.
>> This I also thought, and `qsub -l h_vmem=4G ...` should do it. It can be requested on a per job basis (with further limits on a queue level if necessary).
Well, this sent me in the right direction. I believe h_vmem is
SGE-specific, but our torque environment provides the "pmem"
(physical memory) and "pvmem" (virtual memory) resources on
a per-job basis. These seem to provide exactly the functionality
Sorry to bother you an issue that ended up being independent
of Open MPI!
User Services Group email: dpturner_at_[hidden]
NERSC Division phone: (510) 486-4027
Lawrence Berkeley Lab fax: (510) 486-4316