Subject: Cgroup v2 thread mode oddity: "domain invalid" cgroup with threaded controller enabled

Hello Tejun,

Suppose we have the following scenario:

x [d] (pids)
y [dt] (pids)
p [t]
q [t]
r [t]
z [d]

Here, x/y is a "domain threaded root" with a threaded controller
(the 'pids' controller) enabled. (In this scenario, there are no
member processes in any of the cgroups.)

Suppose we now convert x/z to "threaded" type:

# echo threaded > x/z/cgroup.type

Now we end up in the following state:

x [dt] (pids)
y [inv] (pids)
p [t]
q [t]
r [t]
z [t]

This seems odd. x/y is now of "domain invalid" type with a controller
enabled! This feels like a violation of the rules, since we can't
in other circumstances do anything with a "domain invalid" cgroup
except convert it to "threaded". In particular, we can't create
child cgroups under a "domain invalid" cgroup, or add member processes
to the cgroup, or *enable controllers in the cgroup*. In fact, when
doing the

# echo threaded > x/z/cgroup.type

I had expected a write(2) error because the state of x/y should
(I thought) not be permitted.

Your thoughts?

Thanks,

Michael

--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/


2018-10-04 19:49:58

by Roman Gushchin

[permalink] [raw]
Subject: Re: Cgroup v2 thread mode oddity: "domain invalid" cgroup with threaded controller enabled

On Thu, Oct 04, 2018 at 09:40:57PM +0200, Michael Kerrisk (man-pages) wrote:
> Hello Tejun,
>
> Suppose we have the following scenario:
>
> x [d] (pids)
> y [dt] (pids)
> p [t]
> q [t]
> r [t]
> z [d]
>
> Here, x/y is a "domain threaded root" with a threaded controller
> (the 'pids' controller) enabled. (In this scenario, there are no
> member processes in any of the cgroups.)
>
> Suppose we now convert x/z to "threaded" type:
>
> # echo threaded > x/z/cgroup.type
>
> Now we end up in the following state:
>
> x [dt] (pids)
> y [inv] (pids)
> p [t]
> q [t]
> r [t]
> z [t]
>
> This seems odd. x/y is now of "domain invalid" type with a controller
> enabled! This feels like a violation of the rules, since we can't
> in other circumstances do anything with a "domain invalid" cgroup
> except convert it to "threaded". In particular, we can't create
> child cgroups under a "domain invalid" cgroup, or add member processes
> to the cgroup, or *enable controllers in the cgroup*. In fact, when
> doing the
>
> # echo threaded > x/z/cgroup.type
>
> I had expected a write(2) error because the state of x/y should
> (I thought) not be permitted.
>
> Your thoughts?

Just a note: there are now some cgroup core kernel selftests in
tools/testing/selftests/cgroup/test_core.c . It's fairly easy to add new
tests to cover tricky cases like this one. It always nice to have more tests,
and also easier to show and reproduce a problem.

Thanks!

Subject: Re: Cgroup v2 thread mode oddity: "domain invalid" cgroup with threaded controller enabled

Hello Tejun,

A ping on the below...

Thanks,

Michael

On 10/04/2018 09:40 PM, Michael Kerrisk (man-pages) wrote:
> Hello Tejun,
>
> Suppose we have the following scenario:
>
> x [d] (pids)
> y [dt] (pids)
> p [t]
> q [t]
> r [t]
> z [d]
>
> Here, x/y is a "domain threaded root" with a threaded controller
> (the 'pids' controller) enabled. (In this scenario, there are no
> member processes in any of the cgroups.)
>
> Suppose we now convert x/z to "threaded" type:
>
> # echo threaded > x/z/cgroup.type
>
> Now we end up in the following state:
>
> x [dt] (pids)
> y [inv] (pids)
> p [t]
> q [t]
> r [t]
> z [t]
>
> This seems odd. x/y is now of "domain invalid" type with a controller
> enabled! This feels like a violation of the rules, since we can't
> in other circumstances do anything with a "domain invalid" cgroup
> except convert it to "threaded". In particular, we can't create
> child cgroups under a "domain invalid" cgroup, or add member processes
> to the cgroup, or *enable controllers in the cgroup*. In fact, when
> doing the
>
> # echo threaded > x/z/cgroup.type
>
> I had expected a write(2) error because the state of x/y should
> (I thought) not be permitted.
>
> Your thoughts?
>
> Thanks,
>
> Michael
>


--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

2018-10-17 16:21:10

by Tejun Heo

[permalink] [raw]
Subject: Re: Cgroup v2 thread mode oddity: "domain invalid" cgroup with threaded controller enabled

Hello, Michael.

Sorry about the delay.

On Thu, Oct 04, 2018 at 09:40:57PM +0200, Michael Kerrisk (man-pages) wrote:
> This seems odd. x/y is now of "domain invalid" type with a controller
> enabled! This feels like a violation of the rules, since we can't
> in other circumstances do anything with a "domain invalid" cgroup
> except convert it to "threaded". In particular, we can't create
> child cgroups under a "domain invalid" cgroup, or add member processes
> to the cgroup, or *enable controllers in the cgroup*. In fact, when
> doing the
>
> # echo threaded > x/z/cgroup.type
>
> I had expected a write(2) error because the state of x/y should
> (I thought) not be permitted.

So, both the interim (before turning x/z into threaded) and final
(after) are completely fine - the cgroups are empty and whether
threaded controllers like pids are enabled or not don't really change
things that much.

Maybe it is a bit inconsistent to then deny enabling threaded
controllers on invalid domain cgroups. We can lift that restriction
but I personally can't see why that'd be clearly better.

Thanks.

--
tejun

Subject: Re: Cgroup v2 thread mode oddity: "domain invalid" cgroup with threaded controller enabled

Hi Tejun
On Wed, 17 Oct 2018 at 18:20, Tejun Heo <[email protected]> wrote:
>
> Hello, Michael.
>
> Sorry about the delay.
>
> On Thu, Oct 04, 2018 at 09:40:57PM +0200, Michael Kerrisk (man-pages) wrote:
> > This seems odd. x/y is now of "domain invalid" type with a controller
> > enabled! This feels like a violation of the rules, since we can't
> > in other circumstances do anything with a "domain invalid" cgroup
> > except convert it to "threaded". In particular, we can't create
> > child cgroups under a "domain invalid" cgroup, or add member processes
> > to the cgroup, or *enable controllers in the cgroup*. In fact, when
> > doing the
> >
> > # echo threaded > x/z/cgroup.type
> >
> > I had expected a write(2) error because the state of x/y should
> > (I thought) not be permitted.
>
> So, both the interim (before turning x/z into threaded) and final
> (after) are completely fine - the cgroups are empty and whether
> threaded controllers like pids are enabled or not don't really change
> things that much.
>
> Maybe it is a bit inconsistent to then deny enabling threaded
> controllers on invalid domain cgroups. We can lift that restriction
> but I personally can't see why that'd be clearly better.

So, I also can't see anything that seems harmful in the scenario I
described; it's just an odd inconsistency, and I supposed it was
unintended/overlooked behavior, and I wanted to alert you to it, in
case it seemed problematic. However, as long as you have no concerns,
I see no reason to change the existing behavior.

Thanks,

Michael

--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/