Subject: cgroups(7): documenting /sys/kernel/cgroup files

Hello Roman,

I wrote cgroups(7) text below to document the files added by you in
Linux 4.15. Could you let me know if the following text is okay please:

/sys/kernel/cgroup files
/sys/kernel/cgroup/delegate (since Linux 4.15)
This file exports a list of the cgroups v2 files (one per
line) that are delegatable (i.e., whose ownership should be
changed to the user ID of the delegatee). In the future,
the set of delegatable files may change or grow, and this
file provides a way for the kernel to inform user-space
applications of which files must be delegated. As at Linux
4.15, one sees the following when inspecting this file:

$ cat /sys/kernel/cgroup/delegate
cgroup.procs
cgroup.subtree_control

/sys/kernel/cgroup/features (since Linux 4.15)
Over time, the set of cgroups v2 features that are provided
by the kernel may change or grow, or some features may not
be enabled by default. This file provides a way for user-
space applications to discover what features the running
kernel supports or has enabled. Features are listed one
per line:

$ cat /sys/kernel/cgroup/features nsdelegate

The entries that can appear in this file are:

nsdelegate (since Linux 4.15)
The kernel supports the nsdelegate mount option.

Cheers,

Michael

--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/


2018-01-09 11:28:46

by Roman Gushchin

[permalink] [raw]
Subject: Re: cgroups(7): documenting /sys/kernel/cgroup files

Hello, Michael!

Overall looks good to me, one small nit below.

On Mon, Jan 08, 2018 at 10:11:43PM +0100, Michael Kerrisk (man-pages) wrote:
> Hello Roman,
>
> I wrote cgroups(7) text below to document the files added by you in
> Linux 4.15. Could you let me know if the following text is okay please:
>
> /sys/kernel/cgroup files
> /sys/kernel/cgroup/delegate (since Linux 4.15)
> This file exports a list of the cgroups v2 files (one per
> line) that are delegatable (i.e., whose ownership should be
> changed to the user ID of the delegatee). In the future,
> the set of delegatable files may change or grow, and this
> file provides a way for the kernel to inform user-space
> applications of which files must be delegated. As at Linux
> 4.15, one sees the following when inspecting this file:
>
> $ cat /sys/kernel/cgroup/delegate
> cgroup.procs
> cgroup.subtree_control
>
> /sys/kernel/cgroup/features (since Linux 4.15)
> Over time, the set of cgroups v2 features that are provided
> by the kernel may change or grow, or some features may not
> be enabled by default. This file provides a way for user-
> space applications to discover what features the running
> kernel supports or has enabled. Features are listed one
^^
I would replace "or" with "and" here.

Please, feel free to add
Reviewed-by: Roman Gushchin <[email protected]>

Thank you!

Roman

Subject: Re: cgroups(7): documenting /sys/kernel/cgroup files

Hello ROman,

On 01/09/2018 12:28 PM, Roman Gushchin wrote:
> Hello, Michael!
>
> Overall looks good to me, one small nit below.
>
> On Mon, Jan 08, 2018 at 10:11:43PM +0100, Michael Kerrisk (man-pages) wrote:
>> Hello Roman,
>>
>> I wrote cgroups(7) text below to document the files added by you in
>> Linux 4.15. Could you let me know if the following text is okay please:
>>
>> /sys/kernel/cgroup files
>> /sys/kernel/cgroup/delegate (since Linux 4.15)
>> This file exports a list of the cgroups v2 files (one per
>> line) that are delegatable (i.e., whose ownership should be
>> changed to the user ID of the delegatee). In the future,
>> the set of delegatable files may change or grow, and this
>> file provides a way for the kernel to inform user-space
>> applications of which files must be delegated. As at Linux
>> 4.15, one sees the following when inspecting this file:
>>
>> $ cat /sys/kernel/cgroup/delegate
>> cgroup.procs
>> cgroup.subtree_control
>>
>> /sys/kernel/cgroup/features (since Linux 4.15)
>> Over time, the set of cgroups v2 features that are provided
>> by the kernel may change or grow, or some features may not
>> be enabled by default. This file provides a way for user-
>> space applications to discover what features the running
>> kernel supports or has enabled. Features are listed one
> ^^
> I would replace "or" with "and" here.

Yes. Done.

> Please, feel free to add
> Reviewed-by: Roman Gushchin <[email protected]>
Thanks!

Cheers,

Michael

--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

Subject: Re: cgroups(7): documenting /sys/kernel/cgroup files

Hello Roman,

On 8 January 2018 at 22:11, Michael Kerrisk (man-pages)
<[email protected]> wrote:
> Hello Roman,
>
> I wrote cgroups(7) text below to document the files added by you in
> Linux 4.15. Could you let me know if the following text is okay please:
>
> /sys/kernel/cgroup files
> /sys/kernel/cgroup/delegate (since Linux 4.15)
> This file exports a list of the cgroups v2 files (one per
> line) that are delegatable (i.e., whose ownership should be
> changed to the user ID of the delegatee). In the future,
> the set of delegatable files may change or grow, and this
> file provides a way for the kernel to inform user-space
> applications of which files must be delegated. As at Linux
> 4.15, one sees the following when inspecting this file:
>
> $ cat /sys/kernel/cgroup/delegate
> cgroup.procs
> cgroup.subtree_control

I have a question about /sys/kernel/cgroup/delegate: why does this
file not list cgroup.threads, since that is also a file that
(potentially) should be delegated?

Cheers,

Michael

2018-01-10 12:44:52

by Roman Gushchin

[permalink] [raw]
Subject: Re: cgroups(7): documenting /sys/kernel/cgroup files

Hello Michael,

On Wed, Jan 10, 2018 at 05:23:32AM +0100, Michael Kerrisk (man-pages) wrote:
> Hello Roman,
>
> On 8 January 2018 at 22:11, Michael Kerrisk (man-pages)
> <[email protected]> wrote:
> > Hello Roman,
> >
> > I wrote cgroups(7) text below to document the files added by you in
> > Linux 4.15. Could you let me know if the following text is okay please:
> >
> > /sys/kernel/cgroup files
> > /sys/kernel/cgroup/delegate (since Linux 4.15)
> > This file exports a list of the cgroups v2 files (one per
> > line) that are delegatable (i.e., whose ownership should be
> > changed to the user ID of the delegatee). In the future,
> > the set of delegatable files may change or grow, and this
> > file provides a way for the kernel to inform user-space
> > applications of which files must be delegated. As at Linux
> > 4.15, one sees the following when inspecting this file:
> >
> > $ cat /sys/kernel/cgroup/delegate
> > cgroup.procs
> > cgroup.subtree_control
>
> I have a question about /sys/kernel/cgroup/delegate: why does this
> file not list cgroup.threads, since that is also a file that
> (potentially) should be delegated?

cc Tejun

cgroup.threads is not listed because it's not delegatable (doesn't have
CFTYPE_NS_DELEGATABLE flag set). I don't know if it was intended or just
an accident (I believe the latter).

Thanks!

PS In the latter case the following patch should fix it:

>From fdf19edb6e9594e48e89d4510767b9eb6ca2d9dd Mon Sep 17 00:00:00 2001
From: Roman Gushchin <[email protected]>
Date: Wed, 10 Jan 2018 04:35:12 -0800
Subject: [PATCH] cgroup: make cgroup.threads delegatable

Make cgroup.threads file delegatable.
The behavior of cgroup.threads should follow the behavior of cgroup.procs.

Signed-off-by: Roman Gushchin <[email protected]>
Discovered-by: Michael Kerrisk <[email protected]>
Cc: Tejun Heo <[email protected]>
---
kernel/cgroup/cgroup.c | 1 +
1 file changed, 1 insertion(+)

diff --git a/kernel/cgroup/cgroup.c b/kernel/cgroup/cgroup.c
index 0b1ffe147f24..b74d9ee1425c 100644
--- a/kernel/cgroup/cgroup.c
+++ b/kernel/cgroup/cgroup.c
@@ -4449,6 +4449,7 @@ static struct cftype cgroup_base_files[] = {
},
{
.name = "cgroup.threads",
+ .flags = CFTYPE_NS_DELEGATABLE,
.release = cgroup_procs_release,
.seq_start = cgroup_threads_start,
.seq_next = cgroup_procs_next,
--
2.14.3

2018-01-10 14:02:46

by Tejun Heo

[permalink] [raw]
Subject: Re: cgroups(7): documenting /sys/kernel/cgroup files

Hello,

On Wed, Jan 10, 2018 at 04:44:14AM -0800, Roman Gushchin wrote:
> cgroup.threads is not listed because it's not delegatable (doesn't have
> CFTYPE_NS_DELEGATABLE flag set). I don't know if it was intended or just
> an accident (I believe the latter).

So, thread mode isn't delegatble. Marking a cgroup as threaded turns
that subtree threaded and join the parent's domain, so we can't allow
delegation roots to be turned threaded - just like we can't mark the
root cgroup threaded, and I'm not really not sure what it means to
delegate a portion of threaded subtree.

Thanks.

--
tejun

2018-01-10 14:32:45

by Roman Gushchin

[permalink] [raw]
Subject: Re: cgroups(7): documenting /sys/kernel/cgroup files

On Wed, Jan 10, 2018 at 06:02:39AM -0800, Tejun Heo wrote:
> Hello,
>
> On Wed, Jan 10, 2018 at 04:44:14AM -0800, Roman Gushchin wrote:
> > cgroup.threads is not listed because it's not delegatable (doesn't have
> > CFTYPE_NS_DELEGATABLE flag set). I don't know if it was intended or just
> > an accident (I believe the latter).
>
> So, thread mode isn't delegatble. Marking a cgroup as threaded turns
> that subtree threaded and join the parent's domain, so we can't allow
> delegation roots to be turned threaded - just like we can't mark the
> root cgroup threaded, and I'm not really not sure what it means to
> delegate a portion of threaded subtree.

Thank you for the clarification!

Roman

Subject: Re: cgroups(7): documenting /sys/kernel/cgroup files

Hello Tejun,

On 10 January 2018 at 15:02, Tejun Heo <[email protected]> wrote:
> Hello,
>
> On Wed, Jan 10, 2018 at 04:44:14AM -0800, Roman Gushchin wrote:
>> cgroup.threads is not listed because it's not delegatable (doesn't have
>> CFTYPE_NS_DELEGATABLE flag set). I don't know if it was intended or just
>> an accident (I believe the latter).
>
> So, thread mode isn't delegatble. Marking a cgroup as threaded turns
> that subtree threaded and join the parent's domain, so we can't allow
> delegation roots to be turned threaded - just like we can't mark the
> root cgroup threaded, and I'm not really not sure what it means to
> delegate a portion of threaded subtree.

So, I am confused. According to Documentation/cgroup-v2.txt,
cgoup.threads is delegated:

[[
Model of Delegation
~~~~~~~~~~~~~~~~~~~

A cgroup can be delegated in two ways. First, to a less privileged
user by granting write access of the directory and its "cgroup.procs",
"cgroup.threads" and "cgroup.subtree_control" files to the user.
Second, if the "nsdelegate" mount option is set, automatically to a
cgroup namespace on namespace creation.
]]

So, is that a mistake in that text file?

Cheers,

Michael

--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/

2018-01-10 17:33:58

by Tejun Heo

[permalink] [raw]
Subject: Re: cgroups(7): documenting /sys/kernel/cgroup files

Hello, Michael.

On Wed, Jan 10, 2018 at 06:31:20PM +0100, Michael Kerrisk (man-pages) wrote:
> So, I am confused. According to Documentation/cgroup-v2.txt,
> cgoup.threads is delegated:
>
> [[
> Model of Delegation
> ~~~~~~~~~~~~~~~~~~~
>
> A cgroup can be delegated in two ways. First, to a less privileged
> user by granting write access of the directory and its "cgroup.procs",
> "cgroup.threads" and "cgroup.subtree_control" files to the user.
> Second, if the "nsdelegate" mount option is set, automatically to a
> cgroup namespace on namespace creation.
> ]]
>
> So, is that a mistake in that text file?

Yes, it is. I probably copy&pasted from an earlier version where we
were marking threaded domains instead of threaded roots. Will fix it
right away.

Thanks.

--
tejun

2018-01-10 17:39:48

by Tejun Heo

[permalink] [raw]
Subject: Re: cgroups(7): documenting /sys/kernel/cgroup files

Hello, again.

On Wed, Jan 10, 2018 at 09:33:40AM -0800, Tejun Heo wrote:
> Hello, Michael.
>
> On Wed, Jan 10, 2018 at 06:31:20PM +0100, Michael Kerrisk (man-pages) wrote:
> > So, I am confused. According to Documentation/cgroup-v2.txt,
> > cgoup.threads is delegated:
> >
> > [[
> > Model of Delegation
> > ~~~~~~~~~~~~~~~~~~~
> >
> > A cgroup can be delegated in two ways. First, to a less privileged
> > user by granting write access of the directory and its "cgroup.procs",
> > "cgroup.threads" and "cgroup.subtree_control" files to the user.
> > Second, if the "nsdelegate" mount option is set, automatically to a
> > cgroup namespace on namespace creation.
> > ]]
> >
> > So, is that a mistake in that text file?
>
> Yes, it is. I probably copy&pasted from an earlier version where we
> were marking threaded domains instead of threaded roots. Will fix it
> right away.

Jesus christ, sorry. Roman and the document are right.

* cgroup.type is not delegatble, marking it threaded makes the cgroup
join the parent's domain.

* cgroup.threads is delegatable, because marking the child cgroups as
threaded make them join the subtree root, and the threads can be
dispersed across the parent - the delegated root here - and the
subtree of the child which is marked threaded.

I'll apply Roman's patch. Sorry about the confusion.

Thanks.

--
tejun

2018-01-10 17:45:03

by Tejun Heo

[permalink] [raw]
Subject: Re: cgroups(7): documenting /sys/kernel/cgroup files

On Wed, Jan 10, 2018 at 06:02:39AM -0800, Tejun Heo wrote:
> Hello,
>
> On Wed, Jan 10, 2018 at 04:44:14AM -0800, Roman Gushchin wrote:
> > cgroup.threads is not listed because it's not delegatable (doesn't have
> > CFTYPE_NS_DELEGATABLE flag set). I don't know if it was intended or just
> > an accident (I believe the latter).
>
> So, thread mode isn't delegatble. Marking a cgroup as threaded turns
> that subtree threaded and join the parent's domain, so we can't allow
> delegation roots to be turned threaded - just like we can't mark the
> root cgroup threaded, and I'm not really not sure what it means to
> delegate a portion of threaded subtree.

Please disregard.

Just like we can't mark the root cgroup threaded, we can't mark
delegated subtree root's threaded, so cgroup.type is not delegatable.

However, the delegated root's children can definitely be marked
threadaed and they will join the delegated root's domain and the
threads are free to move between root and the threaded subtree.

Thanks.

--
tejun

2018-01-10 17:45:28

by Tejun Heo

[permalink] [raw]
Subject: Re: cgroups(7): documenting /sys/kernel/cgroup files

On Wed, Jan 10, 2018 at 04:44:14AM -0800, Roman Gushchin wrote:
> From fdf19edb6e9594e48e89d4510767b9eb6ca2d9dd Mon Sep 17 00:00:00 2001
> From: Roman Gushchin <[email protected]>
> Date: Wed, 10 Jan 2018 04:35:12 -0800
> Subject: [PATCH] cgroup: make cgroup.threads delegatable
>
> Make cgroup.threads file delegatable.
> The behavior of cgroup.threads should follow the behavior of cgroup.procs.
>
> Signed-off-by: Roman Gushchin <[email protected]>
> Discovered-by: Michael Kerrisk <[email protected]>
> Cc: Tejun Heo <[email protected]>

Applied to cgroup/for-4.15-fixes.

Thanks.

--
tejun

Subject: Re: cgroups(7): documenting /sys/kernel/cgroup files

On 01/10/2018 06:44 PM, Tejun Heo wrote:
> On Wed, Jan 10, 2018 at 06:02:39AM -0800, Tejun Heo wrote:
>> Hello,
>>
>> On Wed, Jan 10, 2018 at 04:44:14AM -0800, Roman Gushchin wrote:
>>> cgroup.threads is not listed because it's not delegatable (doesn't have
>>> CFTYPE_NS_DELEGATABLE flag set). I don't know if it was intended or just
>>> an accident (I believe the latter).
>>
>> So, thread mode isn't delegatble. Marking a cgroup as threaded turns
>> that subtree threaded and join the parent's domain, so we can't allow
>> delegation roots to be turned threaded - just like we can't mark the
>> root cgroup threaded, and I'm not really not sure what it means to
>> delegate a portion of threaded subtree.
>
> Please disregard.
>
> Just like we can't mark the root cgroup threaded, we can't mark
> delegated subtree root's threaded, so cgroup.type is not delegatable.
>
> However, the delegated root's children can definitely be marked
> threadaed and they will join the delegated root's domain and the
> threads are free to move between root and the threaded subtree.

Exactly, so cgroup.threads in the threaded root must be made
writable by the delegatee.

Thanks, Tejun.

Cheers,

Michael

--
Michael Kerrisk
Linux man-pages maintainer; http://www.kernel.org/doc/man-pages/
Linux/UNIX System Programming Training: http://man7.org/training/