Open MPI logo

Hardware Locality Development Mailing List Archives

  |   Home   |   Support   |   FAQ   |   all Hardware Locality Development mailing list

Subject: Re: [hwloc-devel] Cgroup resource limits
From: Ralph Castain (rhc_at_[hidden])
Date: 2012-11-05 09:43:52


On Nov 4, 2012, at 7:28 PM, Christopher Samuel <samuel_at_[hidden]> wrote:

> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 03/11/12 09:05, Ralph Castain wrote:
>
>> System resource managers don't usually provide this capability, so
>> we will do it at the ORTE level.
>
> I would argue that the resource managers *should* be doing it

No argument from me - I would love for them to provide me with an easy API that mpirun can use to specify the requirements for a given application.

> - however,
> I will also argue that the resource managers should be doing it via
> hwloc (so I'm afraid it's not an out for you folks :-) ).

Agreed, though I leave that to the individual RMs to decide.

>
> It's also worth remembering that the memcg code has an appalling
> reputation with the kernel developers in terms of performance overhead,
> for instance at the recent Kernel Summit numbers were reported showing a
> substantial impact for just having the code present, but not used.
>
> Following that a patch set was sent out trying to avoid that impact if
> it's not in use which doesn't help here but does give a measure of the
> performance hit:
>
> http://lwn.net/Articles/517562/
>
> # So as one can see, the difference between base and nomemcg in terms
> # of both system time and elapsed time is quite drastic, and consistent
> # with the figures shown by Mel Gorman in the Kernel summit. This is a
> # ~7 % drop in performance, just by having memcg enabled. memcg
> # functions appear heavily in the profiles, even if all tasks lives in
> # the root memcg.
>

Yick! However, I would expect the community to reduce that impact over time. If systems don't want that capability, then they can and should disable it. On the other hand, if they do want it, then we want to support it.

> cheers,
> Chris
> - --
> Christopher Samuel Senior Systems Administrator
> VLSCI - Victorian Life Sciences Computation Initiative
> Email: samuel_at_[hidden] Phone: +61 (0)3 903 55545
> http://www.vlsci.org.au/ http://twitter.com/vlsci
>
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
> Comment: Using GnuPG with Mozilla - http://www.enigmail.net/
>
> iEYEARECAAYFAlCXMlUACgkQO2KABBYQAh8eTgCgkruuxIKc3mqpoxwMaeQBI1hR
> /osAn225q4G6FWs1b4Lm6F/9GHDgw9JB
> =jkm0
> -----END PGP SIGNATURE-----
> _______________________________________________
> hwloc-devel mailing list
> hwloc-devel_at_[hidden]
> http://www.open-mpi.org/mailman/listinfo.cgi/hwloc-devel