Open MPI logo

Open MPI User's Mailing List Archives

  |   Home   |   Support   |   FAQ   |  

This web mail archive is frozen.

This page is part of a frozen web archive of this mailing list.

You can still navigate around this archive, but know that no new mails have been added to it since July of 2016.

Click here to be taken to the new web archives of this list; it includes all the mails that are in this frozen archive plus all new mails that have been sent to the list since it was migrated to the new archives.

Subject: [OMPI users] memory limits on remote nodes
From: David Turner (dpturner_at_[hidden])
Date: 2010-10-06 19:55:01


Hi,

We would like to set process memory limits (vmemoryuse, in csh
terms) on remote processes. Our batch system is torque/moab.

The nodes of our cluster each have 24GB of physical memory, of
which 4GB is taken up by the kernel and the root file system.
Note that these are diskless nodes, so no swap either.

We can globally set the per-process limit to 2.5GB. This works
fine if applications run "packed": 8 MPI tasks running on each
8-core node, for an aggregate limit of 20GB. However, if a job
only wants to run 4 tasks, the soft limit can safely be raised
to 5GB. 2 tasks, 10GB. 1 task, the full 20GB.

Upping the soft limit in the batch script itself only affects
the "head node" of the job. Since limits are not part of the
"environment", I can find no way propagate them to remote nodes.

If I understand how this all works, the remote processes are
started by orted, and therefore inherit its limits. Is there
any sort of orted configuration that can help here? Any other
thoughts about how to approach this?

Thanks!

-- 
Best regards,
David Turner
User Services Group        email: dpturner_at_[hidden]
NERSC Division             phone: (510) 486-4027
Lawrence Berkeley Lab        fax: (510) 486-4316