This patch implements a new freezer subsystem in the control groups framework.
It provides a way to stop and resume execution of all tasks in a cgroup by
writing in the cgroup filesystem.
The freezer subsystem in the container filesystem defines a file named
freezer.state. Writing "FROZEN" to the state file will freeze all tasks in the
cgroup. Subsequently writing "RUNNING" will unfreeze the tasks in the cgroup.
Reading will return the current state.
* Examples of usage :
# mkdir /containers/freezer
# mount -t cgroup -ofreezer freezer /containers
# mkdir /containers/0
# echo $some_pid > /containers/0/tasks
to get status of the freezer subsystem :
# cat /containers/0/freezer.state
RUNNING
to freeze all tasks in the container :
# echo FROZEN > /containers/0/freezer.state
# cat /containers/0/freezer.state
FREEZING
# cat /containers/0/freezer.state
FROZEN
to unfreeze all tasks in the container :
# echo RUNNING > /containers/0/freezer.state
# cat /containers/0/freezer.state
RUNNING
This is the basic mechanism which should do the right thing for user space task
in a simple scenario.
It's important to note that freezing can be incomplete. In that case we return
EBUSY. This means that some tasks in the cgroup are busy doing something that
prevents us from completely freezing the cgroup at this time. After EBUSY,
the cgroup will remain partially frozen -- reflected by freezer.state reporting
"FREEZING" when read. The state will remain "FREEZING" until one of these
things happens:
1) Userspace cancels the freezing operation by writing "RUNNING" to
the freezer.state file
2) Userspace retries the freezing operation by writing "FROZEN" to
the freezer.state file (writing "FREEZING" is not legal
and returns EIO)
3) The tasks that blocked the cgroup from entering the "FROZEN"
state disappear from the cgroup's set of tasks.
Signed-off-by: Cedric Le Goater <[email protected]>
Signed-off-by: Matt Helsley <[email protected]>
Acked-by: Serge E. Hallyn <[email protected]>
Tested-by: Matt Helsley <[email protected]>
---
Changes since v5:
v6:
Use the new cgroup write_string method.
Conditionally build and link the freezer code.
Fix a lock ordering problem with the freezer->lock reacquire code
Required order: freezer->lock, css_set_lock, task->alloc_lock
Reacquiring: css_set_lock, task->alloc_lock, freezer->lock
Solution: change freezer_fork() to not require any ordering
between task->alloc_lock and freezer->lock
include/linux/cgroup_subsys.h | 6
include/linux/freezer.h | 22 ++
init/Kconfig | 7
kernel/Makefile | 1
kernel/cgroup_freezer.c | 366 ++++++++++++++++++++++++++++++++++++++++++
kernel/power/Kconfig | 2
6 files changed, 399 insertions(+), 5 deletions(-)
create mode 100644 include/linux/cgroup_freezer.h
create mode 100644 kernel/cgroup_freezer.c
Index: linux-2.6.27-rc1-mm1/include/linux/cgroup_subsys.h
===================================================================
--- linux-2.6.27-rc1-mm1.orig/include/linux/cgroup_subsys.h
+++ linux-2.6.27-rc1-mm1/include/linux/cgroup_subsys.h
@@ -52,3 +52,9 @@ SUBSYS(memrlimit_cgroup)
#endif
/* */
+
+#ifdef CONFIG_CGROUP_FREEZER
+SUBSYS(freezer)
+#endif
+
+/* */
Index: linux-2.6.27-rc1-mm1/include/linux/freezer.h
===================================================================
--- linux-2.6.27-rc1-mm1.orig/include/linux/freezer.h
+++ linux-2.6.27-rc1-mm1/include/linux/freezer.h
@@ -47,22 +47,30 @@ static inline bool should_send_signal(st
/*
* Wake up a frozen process
*
- * task_lock() is taken to prevent the race with refrigerator() which may
+ * task_lock() is needed to prevent the race with refrigerator() which may
* occur if the freezing of tasks fails. Namely, without the lock, if the
* freezing of tasks failed, thaw_tasks() might have run before a task in
* refrigerator() could call frozen_process(), in which case the task would be
* frozen and no one would thaw it.
*/
-static inline int thaw_process(struct task_struct *p)
+static inline int __thaw_process(struct task_struct *p)
{
- task_lock(p);
if (frozen(p)) {
p->flags &= ~PF_FROZEN;
+ return 1;
+ }
+ clear_freeze_flag(p);
+ return 0;
+}
+
+static inline int thaw_process(struct task_struct *p)
+{
+ task_lock(p);
+ if (__thaw_process(p) == 1) {
task_unlock(p);
wake_up_process(p);
return 1;
}
- clear_freeze_flag(p);
task_unlock(p);
return 0;
}
@@ -83,6 +91,12 @@ static inline int try_to_freeze(void)
extern bool freeze_task(struct task_struct *p, bool sig_only);
extern void cancel_freezing(struct task_struct *p);
+#ifdef CONFIG_CGROUP_FREEZER
+extern int cgroup_frozen(struct task_struct *task);
+#else /* !CONFIG_CGROUP_FREEZER */
+static inline int cgroup_frozen(struct task_struct *task) { return 0; }
+#endif /* !CONFIG_CGROUP_FREEZER */
+
/*
* The PF_FREEZER_SKIP flag should be set by a vfork parent right before it
* calls wait_for_completion(&vfork) and reset right after it returns from this
Index: linux-2.6.27-rc1-mm1/init/Kconfig
===================================================================
--- linux-2.6.27-rc1-mm1.orig/init/Kconfig
+++ linux-2.6.27-rc1-mm1/init/Kconfig
@@ -299,6 +299,13 @@ config CGROUP_NS
for instance virtual servers and checkpoint/restart
jobs.
+config CGROUP_FREEZER
+ bool "control group freezer subsystem"
+ depends on CGROUPS
+ help
+ Provides a way to freeze and unfreeze all tasks in a
+ cgroup.
+
config CGROUP_DEVICE
bool "Device controller for cgroups"
depends on CGROUPS && EXPERIMENTAL
Index: linux-2.6.27-rc1-mm1/kernel/Makefile
===================================================================
--- linux-2.6.27-rc1-mm1.orig/kernel/Makefile
+++ linux-2.6.27-rc1-mm1/kernel/Makefile
@@ -54,6 +54,7 @@ obj-$(CONFIG_KEXEC) += kexec.o
obj-$(CONFIG_COMPAT) += compat.o
obj-$(CONFIG_CGROUPS) += cgroup.o
obj-$(CONFIG_CGROUP_DEBUG) += cgroup_debug.o
+obj-$(CONFIG_CGROUP_FREEZER) += cgroup_freezer.o
obj-$(CONFIG_CPUSETS) += cpuset.o
obj-$(CONFIG_CGROUP_NS) += ns_cgroup.o
obj-$(CONFIG_UTS_NS) += utsname.o
Index: linux-2.6.27-rc1-mm1/kernel/cgroup_freezer.c
===================================================================
--- /dev/null
+++ linux-2.6.27-rc1-mm1/kernel/cgroup_freezer.c
@@ -0,0 +1,366 @@
+/*
+ * cgroup_freezer.c - control group freezer subsystem
+ *
+ * Copyright IBM Corporation, 2007
+ *
+ * Author : Cedric Le Goater <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of version 2.1 of the GNU Lesser General Public License
+ * as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it would be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
+ */
+
+#include <linux/module.h>
+#include <linux/cgroup.h>
+#include <linux/fs.h>
+#include <linux/uaccess.h>
+#include <linux/freezer.h>
+#include <linux/seq_file.h>
+
+enum freezer_state {
+ STATE_RUNNING = 0,
+ STATE_FREEZING,
+ STATE_FROZEN,
+};
+
+struct freezer {
+ struct cgroup_subsys_state css;
+ enum freezer_state state;
+ spinlock_t lock; /* protects _writes_ to state */
+};
+
+static inline struct freezer *cgroup_freezer(
+ struct cgroup *cgroup)
+{
+ return container_of(
+ cgroup_subsys_state(cgroup, freezer_subsys_id),
+ struct freezer, css);
+}
+
+static inline struct freezer *task_freezer(struct task_struct *task)
+{
+ return container_of(task_subsys_state(task, freezer_subsys_id),
+ struct freezer, css);
+}
+
+int cgroup_frozen(struct task_struct *task)
+{
+ struct freezer *freezer;
+ enum freezer_state state;
+
+ task_lock(task);
+ freezer = task_freezer(task);
+ state = freezer->state;
+ task_unlock(task);
+
+ return state == STATE_FROZEN;
+}
+
+/*
+ * Buffer size for freezer state is limited by cgroups write_string()
+ * interface. See cgroups code for the current size.
+ */
+static const char *freezer_state_strs[] = {
+ "RUNNING",
+ "FREEZING",
+ "FROZEN",
+};
+
+/*
+ * State diagram
+ * Transitions are caused by userspace writes to the freezer.state file.
+ * The values in parenthesis are state labels. The rest are edge labels.
+ *
+ * (RUNNING) --FROZEN--> (FREEZING) --FROZEN--> (FROZEN)
+ * ^ ^ | |
+ * | \_______RUNNING_______/ |
+ * \_____________________________RUNNING___________/
+ */
+
+struct cgroup_subsys freezer_subsys;
+
+/* Locks taken and their ordering
+ * ------------------------------
+ * css_set_lock
+ * cgroup_mutex (AKA cgroup_lock)
+ * task->alloc_lock (AKA task_lock)
+ * freezer->lock
+ * task->sighand->siglock
+ *
+ * cgroup code forces css_set_lock to be taken before task->alloc_lock
+ *
+ * freezer_create(), freezer_destroy():
+ * cgroup_mutex [ by cgroup core ]
+ *
+ * can_attach():
+ * cgroup_mutex
+ *
+ * cgroup_frozen():
+ * task->alloc_lock (to get task's cgroup)
+ *
+ * freezer_fork() (preserving fork() performance means can't take cgroup_mutex):
+ * task->alloc_lock (to get task's cgroup)
+ * freezer->lock
+ * sighand->siglock (if the cgroup is freezing)
+ *
+ * freezer_read():
+ * cgroup_mutex
+ * freezer->lock
+ * read_lock css_set_lock (cgroup iterator start)
+ *
+ * freezer_write() (freeze):
+ * cgroup_mutex
+ * freezer->lock
+ * read_lock css_set_lock (cgroup iterator start)
+ * sighand->siglock
+ *
+ * freezer_write() (unfreeze):
+ * cgroup_mutex
+ * freezer->lock
+ * read_lock css_set_lock (cgroup iterator start)
+ * task->alloc_lock (to prevent races with freeze_task())
+ * sighand->siglock
+ */
+static struct cgroup_subsys_state *freezer_create(struct cgroup_subsys *ss,
+ struct cgroup *cgroup)
+{
+ struct freezer *freezer;
+
+ freezer = kzalloc(sizeof(struct freezer), GFP_KERNEL);
+ if (!freezer)
+ return ERR_PTR(-ENOMEM);
+
+ spin_lock_init(&freezer->lock);
+ freezer->state = STATE_RUNNING;
+ return &freezer->css;
+}
+
+static void freezer_destroy(struct cgroup_subsys *ss,
+ struct cgroup *cgroup)
+{
+ kfree(cgroup_freezer(cgroup));
+}
+
+
+static int freezer_can_attach(struct cgroup_subsys *ss,
+ struct cgroup *new_cgroup,
+ struct task_struct *task)
+{
+ struct freezer *freezer;
+ int retval = 0;
+
+ /*
+ * The call to cgroup_lock() in the freezer.state write method prevents
+ * a write to that file racing against an attach, and hence the
+ * can_attach() result will remain valid until the attach completes.
+ */
+ freezer = cgroup_freezer(new_cgroup);
+ if (freezer->state == STATE_FROZEN)
+ retval = -EBUSY;
+ return retval;
+}
+
+static void freezer_fork(struct cgroup_subsys *ss, struct task_struct *task)
+{
+ struct freezer *freezer;
+
+ task_lock(task);
+ freezer = task_freezer(task);
+ task_unlock(task);
+
+ BUG_ON(freezer->state == STATE_FROZEN);
+ spin_lock_irq(&freezer->lock);
+ /* Locking avoids race with FREEZING -> RUNNING transitions. */
+ if (freezer->state == STATE_FREEZING)
+ freeze_task(task, true);
+ spin_unlock_irq(&freezer->lock);
+}
+
+/*
+ * caller must hold freezer->lock
+ */
+static void check_if_frozen(struct cgroup *cgroup,
+ struct freezer *freezer)
+{
+ struct cgroup_iter it;
+ struct task_struct *task;
+ unsigned int nfrozen = 0, ntotal = 0;
+
+ cgroup_iter_start(cgroup, &it);
+ while ((task = cgroup_iter_next(cgroup, &it))) {
+ ntotal++;
+ /*
+ * Task is frozen or will freeze immediately when next it gets
+ * woken
+ */
+ if (frozen(task) ||
+ (task_is_stopped_or_traced(task) && freezing(task)))
+ nfrozen++;
+ }
+
+ /*
+ * Transition to FROZEN when no new tasks can be added ensures
+ * that we never exist in the FROZEN state while there are unfrozen
+ * tasks.
+ */
+ if (nfrozen == ntotal)
+ freezer->state = STATE_FROZEN;
+ cgroup_iter_end(cgroup, &it);
+}
+
+static int freezer_read(struct cgroup *cgroup, struct cftype *cft,
+ struct seq_file *m)
+{
+ struct freezer *freezer;
+ enum freezer_state state;
+
+ if (!cgroup_lock_live_group(cgroup))
+ return -ENODEV;
+
+ freezer = cgroup_freezer(cgroup);
+ spin_lock_irq(&freezer->lock);
+ state = freezer->state;
+ if (state == STATE_FREEZING) {
+ /* We change from FREEZING to FROZEN lazily if the cgroup was
+ * only partially frozen when we exitted write. */
+ check_if_frozen(cgroup, freezer);
+ state = freezer->state;
+ }
+ spin_unlock_irq(&freezer->lock);
+ cgroup_unlock();
+
+ seq_puts(m, freezer_state_strs[state]);
+ seq_putc(m, '\n');
+ return 0;
+}
+
+static int try_to_freeze_cgroup(struct cgroup *cgroup, struct freezer *freezer)
+{
+ struct cgroup_iter it;
+ struct task_struct *task;
+ unsigned int num_cant_freeze_now = 0;
+
+ freezer->state = STATE_FREEZING;
+ cgroup_iter_start(cgroup, &it);
+ while ((task = cgroup_iter_next(cgroup, &it))) {
+ if (!freeze_task(task, true))
+ continue;
+ if (task_is_stopped_or_traced(task) && freezing(task))
+ /*
+ * The freeze flag is set so these tasks will
+ * immediately go into the fridge upon waking.
+ */
+ continue;
+ if (!freezing(task) && !freezer_should_skip(task))
+ num_cant_freeze_now++;
+ }
+ cgroup_iter_end(cgroup, &it);
+
+ return num_cant_freeze_now ? -EBUSY : 0;
+}
+
+static int unfreeze_cgroup(struct cgroup *cgroup, struct freezer *freezer)
+{
+ struct cgroup_iter it;
+ struct task_struct *task;
+
+ cgroup_iter_start(cgroup, &it);
+ while ((task = cgroup_iter_next(cgroup, &it))) {
+ int do_wake;
+
+ task_lock(task);
+ do_wake = __thaw_process(task);
+ task_unlock(task);
+ if (do_wake)
+ wake_up_process(task);
+ }
+ cgroup_iter_end(cgroup, &it);
+ freezer->state = STATE_RUNNING;
+
+ return 0;
+}
+
+static int freezer_change_state(struct cgroup *cgroup,
+ enum freezer_state goal_state)
+{
+ struct freezer *freezer;
+ int retval = 0;
+
+ freezer = cgroup_freezer(cgroup);
+ spin_lock_irq(&freezer->lock);
+ check_if_frozen(cgroup, freezer); /* may update freezer->state */
+ if (goal_state == freezer->state)
+ goto out;
+ switch (freezer->state) {
+ case STATE_RUNNING:
+ retval = try_to_freeze_cgroup(cgroup, freezer);
+ break;
+ case STATE_FREEZING:
+ if (goal_state == STATE_FROZEN) {
+ /* Userspace is retrying after
+ * "/bin/echo FROZEN > freezer.state" returned -EBUSY */
+ retval = try_to_freeze_cgroup(cgroup, freezer);
+ break;
+ }
+ /* state == FREEZING and goal_state == RUNNING, so unfreeze */
+ case STATE_FROZEN:
+ retval = unfreeze_cgroup(cgroup, freezer);
+ break;
+ default:
+ break;
+ }
+out:
+ spin_unlock_irq(&freezer->lock);
+
+ return retval;
+}
+
+static int freezer_write(struct cgroup *cgroup,
+ struct cftype *cft,
+ const char *buffer)
+{
+ int retval;
+ enum freezer_state goal_state;
+
+ if (strcmp(buffer, freezer_state_strs[STATE_RUNNING]) == 0)
+ goal_state = STATE_RUNNING;
+ else if (strcmp(buffer, freezer_state_strs[STATE_FROZEN]) == 0)
+ goal_state = STATE_FROZEN;
+ else
+ return -EIO;
+
+ if (!cgroup_lock_live_group(cgroup))
+ return -ENODEV;
+ retval = freezer_change_state(cgroup, goal_state);
+ cgroup_unlock();
+ return retval;
+}
+
+static struct cftype files[] = {
+ {
+ .name = "state",
+ .read_seq_string = freezer_read,
+ .write_string = freezer_write,
+ },
+};
+
+static int freezer_populate(struct cgroup_subsys *ss, struct cgroup *cgroup)
+{
+ return cgroup_add_files(cgroup, ss, files, ARRAY_SIZE(files));
+}
+
+struct cgroup_subsys freezer_subsys = {
+ .name = "freezer",
+ .create = freezer_create,
+ .destroy = freezer_destroy,
+ .populate = freezer_populate,
+ .subsys_id = freezer_subsys_id,
+ .can_attach = freezer_can_attach,
+ .attach = NULL,
+ .fork = freezer_fork,
+ .exit = NULL,
+};
Index: linux-2.6.27-rc1-mm1/kernel/power/Kconfig
===================================================================
--- linux-2.6.27-rc1-mm1.orig/kernel/power/Kconfig
+++ linux-2.6.27-rc1-mm1/kernel/power/Kconfig
@@ -86,7 +86,7 @@ config PM_SLEEP
default y
config FREEZER
- def_bool PM_SLEEP
+ def_bool PM_SLEEP || CGROUP_FREEZER
config SUSPEND
bool "Suspend to RAM and standby"
--
On Mon, 11 Aug 2008 16:53:26 -0700
Matt Helsley <[email protected]> wrote:
> This patch implements a new freezer subsystem in the control groups framework.
> It provides a way to stop and resume execution of all tasks in a cgroup by
> writing in the cgroup filesystem.
>
> The freezer subsystem in the container filesystem defines a file named
> freezer.state. Writing "FROZEN" to the state file will freeze all tasks in the
> cgroup. Subsequently writing "RUNNING" will unfreeze the tasks in the cgroup.
> Reading will return the current state.
>
> * Examples of usage :
>
> # mkdir /containers/freezer
> # mount -t cgroup -ofreezer freezer /containers
> # mkdir /containers/0
> # echo $some_pid > /containers/0/tasks
>
> to get status of the freezer subsystem :
>
> # cat /containers/0/freezer.state
> RUNNING
>
> to freeze all tasks in the container :
>
> # echo FROZEN > /containers/0/freezer.state
> # cat /containers/0/freezer.state
> FREEZING
> # cat /containers/0/freezer.state
> FROZEN
>
> to unfreeze all tasks in the container :
>
> # echo RUNNING > /containers/0/freezer.state
> # cat /containers/0/freezer.state
> RUNNING
>
> This is the basic mechanism which should do the right thing for user space task
> in a simple scenario.
>
> It's important to note that freezing can be incomplete. In that case we return
> EBUSY. This means that some tasks in the cgroup are busy doing something that
> prevents us from completely freezing the cgroup at this time. After EBUSY,
> the cgroup will remain partially frozen -- reflected by freezer.state reporting
> "FREEZING" when read. The state will remain "FREEZING" until one of these
> things happens:
>
> 1) Userspace cancels the freezing operation by writing "RUNNING" to
> the freezer.state file
> 2) Userspace retries the freezing operation by writing "FROZEN" to
> the freezer.state file (writing "FREEZING" is not legal
> and returns EIO)
> 3) The tasks that blocked the cgroup from entering the "FROZEN"
> state disappear from the cgroup's set of tasks.
>
> ...
Is a Documentation/ update planned? Documentation/cgroups.txt might be
the place, or not.
> +
> +#ifdef CONFIG_CGROUP_FREEZER
> +SUBSYS(freezer)
> +#endif
> +
> +/* */
> Index: linux-2.6.27-rc1-mm1/include/linux/freezer.h
> ===================================================================
> --- linux-2.6.27-rc1-mm1.orig/include/linux/freezer.h
> +++ linux-2.6.27-rc1-mm1/include/linux/freezer.h
> @@ -47,22 +47,30 @@ static inline bool should_send_signal(st
> /*
> * Wake up a frozen process
> *
> - * task_lock() is taken to prevent the race with refrigerator() which may
> + * task_lock() is needed to prevent the race with refrigerator() which may
> * occur if the freezing of tasks fails. Namely, without the lock, if the
> * freezing of tasks failed, thaw_tasks() might have run before a task in
> * refrigerator() could call frozen_process(), in which case the task would be
> * frozen and no one would thaw it.
> */
> -static inline int thaw_process(struct task_struct *p)
> +static inline int __thaw_process(struct task_struct *p)
> {
> - task_lock(p);
> if (frozen(p)) {
> p->flags &= ~PF_FROZEN;
> + return 1;
> + }
> + clear_freeze_flag(p);
> + return 0;
> +}
> +
> +static inline int thaw_process(struct task_struct *p)
> +{
> + task_lock(p);
> + if (__thaw_process(p) == 1) {
> task_unlock(p);
> wake_up_process(p);
> return 1;
> }
> - clear_freeze_flag(p);
> task_unlock(p);
> return 0;
> }
I wonder why these are inlined.
> @@ -83,6 +91,12 @@ static inline int try_to_freeze(void)
> extern bool freeze_task(struct task_struct *p, bool sig_only);
> extern void cancel_freezing(struct task_struct *p);
>
> +#ifdef CONFIG_CGROUP_FREEZER
> +extern int cgroup_frozen(struct task_struct *task);
> +#else /* !CONFIG_CGROUP_FREEZER */
> +static inline int cgroup_frozen(struct task_struct *task) { return 0; }
> +#endif /* !CONFIG_CGROUP_FREEZER */
> +
> /*
> * The PF_FREEZER_SKIP flag should be set by a vfork parent right before it
> * calls wait_for_completion(&vfork) and reset right after it returns from this
> Index: linux-2.6.27-rc1-mm1/init/Kconfig
> ===================================================================
> --- linux-2.6.27-rc1-mm1.orig/init/Kconfig
> +++ linux-2.6.27-rc1-mm1/init/Kconfig
> @@ -299,6 +299,13 @@ config CGROUP_NS
> for instance virtual servers and checkpoint/restart
> jobs.
>
> +config CGROUP_FREEZER
> + bool "control group freezer subsystem"
> + depends on CGROUPS
Should it depend on FREEZER also?
oh,
> --- linux-2.6.27-rc1-mm1.orig/kernel/power/Kconfig
> +++ linux-2.6.27-rc1-mm1/kernel/power/Kconfig
> @@ -86,7 +86,7 @@ config PM_SLEEP
> default y
>
> config FREEZER
> - def_bool PM_SLEEP
> + def_bool PM_SLEEP || CGROUP_FREEZER
>
we did it that way. Spose that makes sense.
> + help
> + Provides a way to freeze and unfreeze all tasks in a
> + cgroup.
> +
> config CGROUP_DEVICE
> bool "Device controller for cgroups"
> depends on CGROUPS && EXPERIMENTAL
> Index: linux-2.6.27-rc1-mm1/kernel/Makefile
> ===================================================================
> --- linux-2.6.27-rc1-mm1.orig/kernel/Makefile
> +++ linux-2.6.27-rc1-mm1/kernel/Makefile
> @@ -54,6 +54,7 @@ obj-$(CONFIG_KEXEC) += kexec.o
> obj-$(CONFIG_COMPAT) += compat.o
> obj-$(CONFIG_CGROUPS) += cgroup.o
> obj-$(CONFIG_CGROUP_DEBUG) += cgroup_debug.o
> +obj-$(CONFIG_CGROUP_FREEZER) += cgroup_freezer.o
> obj-$(CONFIG_CPUSETS) += cpuset.o
> obj-$(CONFIG_CGROUP_NS) += ns_cgroup.o
> obj-$(CONFIG_UTS_NS) += utsname.o
> Index: linux-2.6.27-rc1-mm1/kernel/cgroup_freezer.c
> ===================================================================
> --- /dev/null
> +++ linux-2.6.27-rc1-mm1/kernel/cgroup_freezer.c
> @@ -0,0 +1,366 @@
> +/*
> + * cgroup_freezer.c - control group freezer subsystem
> + *
> + * Copyright IBM Corporation, 2007
> + *
> + * Author : Cedric Le Goater <[email protected]>
> + *
> + * This program is free software; you can redistribute it and/or modify it
> + * under the terms of version 2.1 of the GNU Lesser General Public License
> + * as published by the Free Software Foundation.
> + *
> + * This program is distributed in the hope that it would be useful, but
> + * WITHOUT ANY WARRANTY; without even the implied warranty of
> + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> + */
> +
> +#include <linux/module.h>
> +#include <linux/cgroup.h>
> +#include <linux/fs.h>
> +#include <linux/uaccess.h>
> +#include <linux/freezer.h>
> +#include <linux/seq_file.h>
> +
> +enum freezer_state {
> + STATE_RUNNING = 0,
That's a pretty vanilla-sounding identifier. Let's hope this file
never ends up including drivers/net/sfc/net_driver.h by some means.
That's rather unlikely, but someone could easily choose to implement a
new STATE_RUNNING somewhere else.
> + STATE_FREEZING,
> + STATE_FROZEN,
> +};
> +
> +struct freezer {
> + struct cgroup_subsys_state css;
> + enum freezer_state state;
> + spinlock_t lock; /* protects _writes_ to state */
> +};
> +
> +static inline struct freezer *cgroup_freezer(
> + struct cgroup *cgroup)
> +{
> + return container_of(
> + cgroup_subsys_state(cgroup, freezer_subsys_id),
> + struct freezer, css);
> +}
> +
> +static inline struct freezer *task_freezer(struct task_struct *task)
> +{
> + return container_of(task_subsys_state(task, freezer_subsys_id),
> + struct freezer, css);
> +}
> +
> +int cgroup_frozen(struct task_struct *task)
> +{
> + struct freezer *freezer;
> + enum freezer_state state;
> +
> + task_lock(task);
> + freezer = task_freezer(task);
> + state = freezer->state;
> + task_unlock(task);
> +
> + return state == STATE_FROZEN;
> +}
> +
> +/*
> + * Buffer size for freezer state is limited by cgroups write_string()
> + * interface. See cgroups code for the current size.
> + */
Is this comment in the correct place?
> +static const char *freezer_state_strs[] = {
> + "RUNNING",
> + "FREEZING",
> + "FROZEN",
> +};
> +
>
> ...
>
> +
> +/*
> + * caller must hold freezer->lock
> + */
> +static void check_if_frozen(struct cgroup *cgroup,
> + struct freezer *freezer)
check_if_frozen() is an unfortunate name, I suspect. Normally one
would expect a check_foo() to return a bool and have no side-effects.
Perhaps some comments explaining what it does would help.
> +{
> + struct cgroup_iter it;
> + struct task_struct *task;
> + unsigned int nfrozen = 0, ntotal = 0;
> +
> + cgroup_iter_start(cgroup, &it);
> + while ((task = cgroup_iter_next(cgroup, &it))) {
> + ntotal++;
> + /*
> + * Task is frozen or will freeze immediately when next it gets
> + * woken
> + */
> + if (frozen(task) ||
> + (task_is_stopped_or_traced(task) && freezing(task)))
> + nfrozen++;
> + }
> +
> + /*
> + * Transition to FROZEN when no new tasks can be added ensures
> + * that we never exist in the FROZEN state while there are unfrozen
> + * tasks.
> + */
> + if (nfrozen == ntotal)
> + freezer->state = STATE_FROZEN;
> + cgroup_iter_end(cgroup, &it);
> +}
> +
>
> ...
>
> +static int freezer_write(struct cgroup *cgroup,
> + struct cftype *cft,
> + const char *buffer)
> +{
> + int retval;
> + enum freezer_state goal_state;
> +
> + if (strcmp(buffer, freezer_state_strs[STATE_RUNNING]) == 0)
Did some higher-level code take care of removing the trailing \n?
> + goal_state = STATE_RUNNING;
> + else if (strcmp(buffer, freezer_state_strs[STATE_FROZEN]) == 0)
> + goal_state = STATE_FROZEN;
> + else
> + return -EIO;
> +
> + if (!cgroup_lock_live_group(cgroup))
> + return -ENODEV;
> + retval = freezer_change_state(cgroup, goal_state);
> + cgroup_unlock();
> + return retval;
> +}
> +
>
> ...
>
On Tue, 2008-08-12 at 15:56 -0700, Andrew Morton wrote:
> On Mon, 11 Aug 2008 16:53:26 -0700
> Matt Helsley <[email protected]> wrote:
>
> > This patch implements a new freezer subsystem in the control groups framework.
> > It provides a way to stop and resume execution of all tasks in a cgroup by
> > writing in the cgroup filesystem.
> >
> > The freezer subsystem in the container filesystem defines a file named
> > freezer.state. Writing "FROZEN" to the state file will freeze all tasks in the
> > cgroup. Subsequently writing "RUNNING" will unfreeze the tasks in the cgroup.
> > Reading will return the current state.
> >
> > * Examples of usage :
> >
> > # mkdir /containers/freezer
> > # mount -t cgroup -ofreezer freezer /containers
> > # mkdir /containers/0
> > # echo $some_pid > /containers/0/tasks
> >
> > to get status of the freezer subsystem :
> >
> > # cat /containers/0/freezer.state
> > RUNNING
> >
> > to freeze all tasks in the container :
> >
> > # echo FROZEN > /containers/0/freezer.state
> > # cat /containers/0/freezer.state
> > FREEZING
> > # cat /containers/0/freezer.state
> > FROZEN
> >
> > to unfreeze all tasks in the container :
> >
> > # echo RUNNING > /containers/0/freezer.state
> > # cat /containers/0/freezer.state
> > RUNNING
> >
> > This is the basic mechanism which should do the right thing for user space task
> > in a simple scenario.
> >
> > It's important to note that freezing can be incomplete. In that case we return
> > EBUSY. This means that some tasks in the cgroup are busy doing something that
> > prevents us from completely freezing the cgroup at this time. After EBUSY,
> > the cgroup will remain partially frozen -- reflected by freezer.state reporting
> > "FREEZING" when read. The state will remain "FREEZING" until one of these
> > things happens:
> >
> > 1) Userspace cancels the freezing operation by writing "RUNNING" to
> > the freezer.state file
> > 2) Userspace retries the freezing operation by writing "FROZEN" to
> > the freezer.state file (writing "FREEZING" is not legal
> > and returns EIO)
> > 3) The tasks that blocked the cgroup from entering the "FROZEN"
> > state disappear from the cgroup's set of tasks.
> >
> > ...
>
> Is a Documentation/ update planned? Documentation/cgroups.txt might be
> the place, or not.
I'll post a patch for that.
> > +
> > +#ifdef CONFIG_CGROUP_FREEZER
> > +SUBSYS(freezer)
> > +#endif
> > +
> > +/* */
> > Index: linux-2.6.27-rc1-mm1/include/linux/freezer.h
> > ===================================================================
> > --- linux-2.6.27-rc1-mm1.orig/include/linux/freezer.h
> > +++ linux-2.6.27-rc1-mm1/include/linux/freezer.h
> > @@ -47,22 +47,30 @@ static inline bool should_send_signal(st
> > /*
> > * Wake up a frozen process
> > *
> > - * task_lock() is taken to prevent the race with refrigerator() which may
> > + * task_lock() is needed to prevent the race with refrigerator() which may
> > * occur if the freezing of tasks fails. Namely, without the lock, if the
> > * freezing of tasks failed, thaw_tasks() might have run before a task in
> > * refrigerator() could call frozen_process(), in which case the task would be
> > * frozen and no one would thaw it.
> > */
> > -static inline int thaw_process(struct task_struct *p)
> > +static inline int __thaw_process(struct task_struct *p)
> > {
> > - task_lock(p);
> > if (frozen(p)) {
> > p->flags &= ~PF_FROZEN;
> > + return 1;
> > + }
> > + clear_freeze_flag(p);
> > + return 0;
> > +}
> > +
> > +static inline int thaw_process(struct task_struct *p)
> > +{
> > + task_lock(p);
> > + if (__thaw_process(p) == 1) {
> > task_unlock(p);
> > wake_up_process(p);
> > return 1;
> > }
> > - clear_freeze_flag(p);
> > task_unlock(p);
> > return 0;
> > }
>
> I wonder why these are inlined.
I wanted the changes to be obvious. I think uninlining this would be a
separate improvement. I'll post a patch uninlining these.
> > @@ -83,6 +91,12 @@ static inline int try_to_freeze(void)
> > extern bool freeze_task(struct task_struct *p, bool sig_only);
> > extern void cancel_freezing(struct task_struct *p);
> >
> > +#ifdef CONFIG_CGROUP_FREEZER
> > +extern int cgroup_frozen(struct task_struct *task);
> > +#else /* !CONFIG_CGROUP_FREEZER */
> > +static inline int cgroup_frozen(struct task_struct *task) { return 0; }
> > +#endif /* !CONFIG_CGROUP_FREEZER */
> > +
> > /*
> > * The PF_FREEZER_SKIP flag should be set by a vfork parent right before it
> > * calls wait_for_completion(&vfork) and reset right after it returns from this
> > Index: linux-2.6.27-rc1-mm1/init/Kconfig
> > ===================================================================
> > --- linux-2.6.27-rc1-mm1.orig/init/Kconfig
> > +++ linux-2.6.27-rc1-mm1/init/Kconfig
> > @@ -299,6 +299,13 @@ config CGROUP_NS
> > for instance virtual servers and checkpoint/restart
> > jobs.
> >
> > +config CGROUP_FREEZER
> > + bool "control group freezer subsystem"
> > + depends on CGROUPS
>
> Should it depend on FREEZER also?
>
> oh,
>
> > --- linux-2.6.27-rc1-mm1.orig/kernel/power/Kconfig
> > +++ linux-2.6.27-rc1-mm1/kernel/power/Kconfig
> > @@ -86,7 +86,7 @@ config PM_SLEEP
> > default y
> >
> > config FREEZER
> > - def_bool PM_SLEEP
> > + def_bool PM_SLEEP || CGROUP_FREEZER
> >
>
> we did it that way. Spose that makes sense.
I did consider a few alternatives for this. Makefile and cpp didn't
seem as nice as this. "select" didn't fit. Using "depends on" does
directly translate the build dependency. However I didn't think it would
be clear to everyone configuring a kernel that they had to enable
"FREEZER" before they could get PM_SLEEP or CGROUP_FREEZER.
Also, Rafael has asked to see this in a kernel/Kconfig file instead
(see his reply to patch 2).
> > + help
> > + Provides a way to freeze and unfreeze all tasks in a
> > + cgroup.
> > +
> > config CGROUP_DEVICE
> > bool "Device controller for cgroups"
> > depends on CGROUPS && EXPERIMENTAL
> > Index: linux-2.6.27-rc1-mm1/kernel/Makefile
> > ===================================================================
> > --- linux-2.6.27-rc1-mm1.orig/kernel/Makefile
> > +++ linux-2.6.27-rc1-mm1/kernel/Makefile
> > @@ -54,6 +54,7 @@ obj-$(CONFIG_KEXEC) += kexec.o
> > obj-$(CONFIG_COMPAT) += compat.o
> > obj-$(CONFIG_CGROUPS) += cgroup.o
> > obj-$(CONFIG_CGROUP_DEBUG) += cgroup_debug.o
> > +obj-$(CONFIG_CGROUP_FREEZER) += cgroup_freezer.o
> > obj-$(CONFIG_CPUSETS) += cpuset.o
> > obj-$(CONFIG_CGROUP_NS) += ns_cgroup.o
> > obj-$(CONFIG_UTS_NS) += utsname.o
> > Index: linux-2.6.27-rc1-mm1/kernel/cgroup_freezer.c
> > ===================================================================
> > --- /dev/null
> > +++ linux-2.6.27-rc1-mm1/kernel/cgroup_freezer.c
> > @@ -0,0 +1,366 @@
> > +/*
> > + * cgroup_freezer.c - control group freezer subsystem
> > + *
> > + * Copyright IBM Corporation, 2007
> > + *
> > + * Author : Cedric Le Goater <[email protected]>
> > + *
> > + * This program is free software; you can redistribute it and/or modify it
> > + * under the terms of version 2.1 of the GNU Lesser General Public License
> > + * as published by the Free Software Foundation.
> > + *
> > + * This program is distributed in the hope that it would be useful, but
> > + * WITHOUT ANY WARRANTY; without even the implied warranty of
> > + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
> > + */
> > +
> > +#include <linux/module.h>
> > +#include <linux/cgroup.h>
> > +#include <linux/fs.h>
> > +#include <linux/uaccess.h>
> > +#include <linux/freezer.h>
> > +#include <linux/seq_file.h>
> > +
> > +enum freezer_state {
> > + STATE_RUNNING = 0,
>
> That's a pretty vanilla-sounding identifier. Let's hope this file
> never ends up including drivers/net/sfc/net_driver.h by some means.
> That's rather unlikely, but someone could easily choose to implement a
> new STATE_RUNNING somewhere else.
Good point. Do CGROUP_THAWED, CGROUP_FREEZING, CGROUP_FROZEN make
sensible substitutions?
> > + STATE_FREEZING,
> > + STATE_FROZEN,
> > +};
> > +
> > +struct freezer {
> > + struct cgroup_subsys_state css;
> > + enum freezer_state state;
> > + spinlock_t lock; /* protects _writes_ to state */
> > +};
> > +
> > +static inline struct freezer *cgroup_freezer(
> > + struct cgroup *cgroup)
> > +{
> > + return container_of(
> > + cgroup_subsys_state(cgroup, freezer_subsys_id),
> > + struct freezer, css);
> > +}
> > +
> > +static inline struct freezer *task_freezer(struct task_struct *task)
> > +{
> > + return container_of(task_subsys_state(task, freezer_subsys_id),
> > + struct freezer, css);
> > +}
> > +
> > +int cgroup_frozen(struct task_struct *task)
> > +{
> > + struct freezer *freezer;
> > + enum freezer_state state;
> > +
> > + task_lock(task);
> > + freezer = task_freezer(task);
> > + state = freezer->state;
> > + task_unlock(task);
> > +
> > + return state == STATE_FROZEN;
> > +}
> > +
> > +/*
> > + * Buffer size for freezer state is limited by cgroups write_string()
> > + * interface. See cgroups code for the current size.
> > + */
>
> Is this comment in the correct place?
I think so. Perhaps I should have more clearly connected it with
freezer_state_strs. How about:
/*
* cgroups_write_string() limits the size of these strings to
* CGROUP_LOCAL_BUFFER_SIZE
*/
> > +static const char *freezer_state_strs[] = {
> > + "RUNNING",
> > + "FREEZING",
> > + "FROZEN",
> > +};
> > +
> >
> > ...
> >
> > +
> > +/*
> > + * caller must hold freezer->lock
> > + */
> > +static void check_if_frozen(struct cgroup *cgroup,
> > + struct freezer *freezer)
>
> check_if_frozen() is an unfortunate name, I suspect. Normally one
> would expect a check_foo() to return a bool and have no side-effects.
>
> Perhaps some comments explaining what it does would help.
OK. I'll try to think up a better name and if that's not sufficiently
explanatory I'll add a comment explaining what it should do.
> > +{
> > + struct cgroup_iter it;
> > + struct task_struct *task;
> > + unsigned int nfrozen = 0, ntotal = 0;
> > +
> > + cgroup_iter_start(cgroup, &it);
> > + while ((task = cgroup_iter_next(cgroup, &it))) {
> > + ntotal++;
> > + /*
> > + * Task is frozen or will freeze immediately when next it gets
> > + * woken
> > + */
> > + if (frozen(task) ||
> > + (task_is_stopped_or_traced(task) && freezing(task)))
> > + nfrozen++;
> > + }
> > +
> > + /*
> > + * Transition to FROZEN when no new tasks can be added ensures
> > + * that we never exist in the FROZEN state while there are unfrozen
> > + * tasks.
> > + */
> > + if (nfrozen == ntotal)
> > + freezer->state = STATE_FROZEN;
> > + cgroup_iter_end(cgroup, &it);
> > +}
> > +
> >
> > ...
> >
> > +static int freezer_write(struct cgroup *cgroup,
> > + struct cftype *cft,
> > + const char *buffer)
> > +{
> > + int retval;
> > + enum freezer_state goal_state;
> > +
> > + if (strcmp(buffer, freezer_state_strs[STATE_RUNNING]) == 0)
>
> Did some higher-level code take care of removing the trailing \n?
Yes. cgroup_write_string() in kernel/cgroup.c does strstrip(buffer)
Thanks for the review!
Cheers,
-Matt Helsley
On Wednesday, 13 of August 2008, Matt Helsley wrote:
>
> On Tue, 2008-08-12 at 15:56 -0700, Andrew Morton wrote:
> > On Mon, 11 Aug 2008 16:53:26 -0700
> > Matt Helsley <[email protected]> wrote:
> >
> > > This patch implements a new freezer subsystem in the control groups framework.
> > > It provides a way to stop and resume execution of all tasks in a cgroup by
> > > writing in the cgroup filesystem.
> > >
> > > The freezer subsystem in the container filesystem defines a file named
> > > freezer.state. Writing "FROZEN" to the state file will freeze all tasks in the
> > > cgroup. Subsequently writing "RUNNING" will unfreeze the tasks in the cgroup.
> > > Reading will return the current state.
> > >
> > > * Examples of usage :
> > >
> > > # mkdir /containers/freezer
> > > # mount -t cgroup -ofreezer freezer /containers
> > > # mkdir /containers/0
> > > # echo $some_pid > /containers/0/tasks
> > >
> > > to get status of the freezer subsystem :
> > >
> > > # cat /containers/0/freezer.state
> > > RUNNING
> > >
> > > to freeze all tasks in the container :
> > >
> > > # echo FROZEN > /containers/0/freezer.state
> > > # cat /containers/0/freezer.state
> > > FREEZING
> > > # cat /containers/0/freezer.state
> > > FROZEN
> > >
> > > to unfreeze all tasks in the container :
> > >
> > > # echo RUNNING > /containers/0/freezer.state
> > > # cat /containers/0/freezer.state
> > > RUNNING
> > >
> > > This is the basic mechanism which should do the right thing for user space task
> > > in a simple scenario.
> > >
> > > It's important to note that freezing can be incomplete. In that case we return
> > > EBUSY. This means that some tasks in the cgroup are busy doing something that
> > > prevents us from completely freezing the cgroup at this time. After EBUSY,
> > > the cgroup will remain partially frozen -- reflected by freezer.state reporting
> > > "FREEZING" when read. The state will remain "FREEZING" until one of these
> > > things happens:
> > >
> > > 1) Userspace cancels the freezing operation by writing "RUNNING" to
> > > the freezer.state file
> > > 2) Userspace retries the freezing operation by writing "FROZEN" to
> > > the freezer.state file (writing "FREEZING" is not legal
> > > and returns EIO)
> > > 3) The tasks that blocked the cgroup from entering the "FROZEN"
> > > state disappear from the cgroup's set of tasks.
> > >
> > > ...
> >
> > Is a Documentation/ update planned? Documentation/cgroups.txt might be
> > the place, or not.
>
> I'll post a patch for that.
>
> > > +
> > > +#ifdef CONFIG_CGROUP_FREEZER
> > > +SUBSYS(freezer)
> > > +#endif
> > > +
> > > +/* */
> > > Index: linux-2.6.27-rc1-mm1/include/linux/freezer.h
> > > ===================================================================
> > > --- linux-2.6.27-rc1-mm1.orig/include/linux/freezer.h
> > > +++ linux-2.6.27-rc1-mm1/include/linux/freezer.h
> > > @@ -47,22 +47,30 @@ static inline bool should_send_signal(st
> > > /*
> > > * Wake up a frozen process
> > > *
> > > - * task_lock() is taken to prevent the race with refrigerator() which may
> > > + * task_lock() is needed to prevent the race with refrigerator() which may
> > > * occur if the freezing of tasks fails. Namely, without the lock, if the
> > > * freezing of tasks failed, thaw_tasks() might have run before a task in
> > > * refrigerator() could call frozen_process(), in which case the task would be
> > > * frozen and no one would thaw it.
> > > */
> > > -static inline int thaw_process(struct task_struct *p)
> > > +static inline int __thaw_process(struct task_struct *p)
> > > {
> > > - task_lock(p);
> > > if (frozen(p)) {
> > > p->flags &= ~PF_FROZEN;
> > > + return 1;
> > > + }
> > > + clear_freeze_flag(p);
> > > + return 0;
> > > +}
> > > +
> > > +static inline int thaw_process(struct task_struct *p)
> > > +{
> > > + task_lock(p);
> > > + if (__thaw_process(p) == 1) {
> > > task_unlock(p);
> > > wake_up_process(p);
> > > return 1;
> > > }
> > > - clear_freeze_flag(p);
> > > task_unlock(p);
> > > return 0;
> > > }
> >
> > I wonder why these are inlined.
>
> I wanted the changes to be obvious. I think uninlining this would be a
> separate improvement. I'll post a patch uninlining these.
>
> > > @@ -83,6 +91,12 @@ static inline int try_to_freeze(void)
> > > extern bool freeze_task(struct task_struct *p, bool sig_only);
> > > extern void cancel_freezing(struct task_struct *p);
> > >
> > > +#ifdef CONFIG_CGROUP_FREEZER
> > > +extern int cgroup_frozen(struct task_struct *task);
> > > +#else /* !CONFIG_CGROUP_FREEZER */
> > > +static inline int cgroup_frozen(struct task_struct *task) { return 0; }
> > > +#endif /* !CONFIG_CGROUP_FREEZER */
> > > +
> > > /*
> > > * The PF_FREEZER_SKIP flag should be set by a vfork parent right before it
> > > * calls wait_for_completion(&vfork) and reset right after it returns from this
> > > Index: linux-2.6.27-rc1-mm1/init/Kconfig
> > > ===================================================================
> > > --- linux-2.6.27-rc1-mm1.orig/init/Kconfig
> > > +++ linux-2.6.27-rc1-mm1/init/Kconfig
> > > @@ -299,6 +299,13 @@ config CGROUP_NS
> > > for instance virtual servers and checkpoint/restart
> > > jobs.
> > >
> > > +config CGROUP_FREEZER
> > > + bool "control group freezer subsystem"
> > > + depends on CGROUPS
> >
> > Should it depend on FREEZER also?
> >
> > oh,
> >
> > > --- linux-2.6.27-rc1-mm1.orig/kernel/power/Kconfig
> > > +++ linux-2.6.27-rc1-mm1/kernel/power/Kconfig
> > > @@ -86,7 +86,7 @@ config PM_SLEEP
> > > default y
> > >
> > > config FREEZER
> > > - def_bool PM_SLEEP
> > > + def_bool PM_SLEEP || CGROUP_FREEZER
> > >
> >
> > we did it that way. Spose that makes sense.
>
> I did consider a few alternatives for this. Makefile and cpp didn't
> seem as nice as this. "select" didn't fit. Using "depends on" does
> directly translate the build dependency. However I didn't think it would
> be clear to everyone configuring a kernel that they had to enable
> "FREEZER" before they could get PM_SLEEP or CGROUP_FREEZER.
>
> Also, Rafael has asked to see this in a kernel/Kconfig file instead
> (see his reply to patch 2).
Yes, I have and there's a good reason for that IMO - you want FREEZER to be
used by architectures that don't include 'kernel/power/Kconfig', apparently.
Thanks,
Rafael
On Mon, Aug 11, 2008 at 3:53 PM, Matt Helsley <[email protected]> wrote:
> +}
> +
> +static void freezer_fork(struct cgroup_subsys *ss, struct task_struct *task)
> +{
> + struct freezer *freezer;
> +
> + task_lock(task);
> + freezer = task_freezer(task);
> + task_unlock(task);
> +
> + BUG_ON(freezer->state == STATE_FROZEN);
> + spin_lock_irq(&freezer->lock);
> + /* Locking avoids race with FREEZING -> RUNNING transitions. */
> + if (freezer->state == STATE_FREEZING)
> + freeze_task(task, true);
> + spin_unlock_irq(&freezer->lock);
> +}
Sorry for such a delayed response to this patch, but I just noticed
(in mainline now)
the change to move the task_lock() to only encompass the
task_freezer() call.
That results in absolutely zero protection from the task_lock() - as
soon as you drop it, then in theory the current task could move cgroup
and the old freezer structure be freed.
Having said that, I think that in this case any locking my be
unnecessary since task isn't on the tasklist yet, so can't be selected
to move cgroups. (Although this does make me wonder whether
cpuset.c:move_member_tasks_to_cpuset() can fail silently if it races
with a fork).
On top of that, for a system that configures in the cgroup freezer
system but doesn't ever use it, every task is in the same freezer
cgroup (the root cgroup) and task_freezer(task)->lock becomes a global
spinlock. I think this would certainly show up on some benchmarks
although I don't know how bad it would be in a practical sense. But it
might be worth considering making using of the cgroup bind() callback
to track whether or not the freezer subsystem is in use, and
short-circuiting this in freezer_fork() without any locking if you're
not active.
Paul
Paul Menage wrote:
> On Mon, Aug 11, 2008 at 3:53 PM, Matt Helsley <[email protected]> wrote:
>> +}
>> +
>> +static void freezer_fork(struct cgroup_subsys *ss, struct task_struct *task)
>> +{
>> + struct freezer *freezer;
>> +
>> + task_lock(task);
>> + freezer = task_freezer(task);
>> + task_unlock(task);
>> +
>> + BUG_ON(freezer->state == STATE_FROZEN);
>> + spin_lock_irq(&freezer->lock);
>> + /* Locking avoids race with FREEZING -> RUNNING transitions. */
>> + if (freezer->state == STATE_FREEZING)
>> + freeze_task(task, true);
>> + spin_unlock_irq(&freezer->lock);
>> +}
>
> Sorry for such a delayed response to this patch, but I just noticed
> (in mainline now)
> the change to move the task_lock() to only encompass the
> task_freezer() call.
>
> That results in absolutely zero protection from the task_lock() - as
> soon as you drop it, then in theory the current task could move cgroup
> and the old freezer structure be freed.
>
> Having said that, I think that in this case any locking my be
> unnecessary since task isn't on the tasklist yet, so can't be selected
> to move cgroups. (Although this does make me wonder whether
> cpuset.c:move_member_tasks_to_cpuset() can fail silently if it races
> with a fork).
>
> On top of that, for a system that configures in the cgroup freezer
> system but doesn't ever use it, every task is in the same freezer
> cgroup (the root cgroup) and task_freezer(task)->lock becomes a global
> spinlock. I think this would certainly show up on some benchmarks
> although I don't know how bad it would be in a practical sense. But it
> might be worth considering making using of the cgroup bind() callback
> to track whether or not the freezer subsystem is in use, and
> short-circuiting this in freezer_fork() without any locking if you're
> not active.
>
I think another reasonable and easier way is to disable writing freezer.state
of top cgroup, so we can skip checks in freezer_fork() for tasks in top cgroup.
On Mon, Nov 3, 2008 at 10:12 PM, Li Zefan <[email protected]> wrote:
>
> I think another reasonable and easier way is to disable writing freezer.state
> of top cgroup, so we can skip checks in freezer_fork() for tasks in top cgroup.
>
Yes, if freezing the root cgroup isn't needed then that is definitely
simpler - I wasn't sure if Matt wanted to be able to freeze the root
group with this subsystem.
Paul
In theory the task can be moved to another cgroup and the freezer will
be freed right after task_lock is dropped, so the lock results in zero
protection.
But in the case of freezer_fork() no lock is needed, since the task
is not in tasklist yet so it won't be moved to another cgroup, so
task->cgroups won't be changed or invalidated.
Signed-off-by: Li Zefan <[email protected]>
---
kernel/cgroup_freezer.c | 8 ++++++--
1 files changed, 6 insertions(+), 2 deletions(-)
diff --git a/kernel/cgroup_freezer.c b/kernel/cgroup_freezer.c
index 7fa476f..6605907 100644
--- a/kernel/cgroup_freezer.c
+++ b/kernel/cgroup_freezer.c
@@ -184,9 +184,13 @@ static void freezer_fork(struct cgroup_subsys *ss, struct task_struct *task)
{
struct freezer *freezer;
- task_lock(task);
+ /*
+ * No lock is needed, since the task isn't on tasklist yet,
+ * so it can't be moved to another cgroup, which means the
+ * freezer won't be removed and will be valid during this
+ * function call.
+ */
freezer = task_freezer(task);
- task_unlock(task);
spin_lock_irq(&freezer->lock);
BUG_ON(freezer->state == CGROUP_FROZEN);
--
1.5.4.rc3
Paul Menage wrote:
> On Mon, Nov 3, 2008 at 10:12 PM, Li Zefan <[email protected]> wrote:
>> I think another reasonable and easier way is to disable writing freezer.state
>> of top cgroup, so we can skip checks in freezer_fork() for tasks in top cgroup.
>>
>
> Yes, if freezing the root cgroup isn't needed then that is definitely
> simpler - I wasn't sure if Matt wanted to be able to freeze the root
> group with this subsystem.
>
If someone just mounts the freezer subsystem and freeze the root cgroup,
all the tasks will be freezed, and then seems all he can do is reset the
machine (or do a suspend/resume).
That's why I think it's reasonable to disable it.
With this change, the root cgroup is unfreezable, and writing to
its freezer.state returns -EIO.
I think it's reasonable to disallow freezing all the tasks in the
root cgroup. And this avoids fork overhead when freezer subsystem
is compiled but not used.
Signed-off-by: Li Zefan <[email protected]>
---
Documentation/cgroups/freezer-subsystem.txt | 18 ++++++++++--------
kernel/cgroup_freezer.c | 7 +++++++
2 files changed, 17 insertions(+), 8 deletions(-)
diff --git a/Documentation/cgroups/freezer-subsystem.txt b/Documentation/cgroups/freezer-subsystem.txt
index c50ab58..e842bad 100644
--- a/Documentation/cgroups/freezer-subsystem.txt
+++ b/Documentation/cgroups/freezer-subsystem.txt
@@ -1,4 +1,4 @@
- The cgroup freezer is useful to batch job management system which start
+The cgroup freezer is useful to batch job management system which start
and stop sets of tasks in order to schedule the resources of a machine
according to the desires of a system administrator. This sort of program
is often used on HPC clusters to schedule access to the cluster as a
@@ -6,7 +6,7 @@ whole. The cgroup freezer uses cgroups to describe the set of tasks to
be started/stopped by the batch job management system. It also provides
a means to start and stop the tasks composing the job.
- The cgroup freezer will also be useful for checkpointing running groups
+The cgroup freezer will also be useful for checkpointing running groups
of tasks. The freezer allows the checkpoint code to obtain a consistent
image of the tasks by attempting to force the tasks in a cgroup into a
quiescent state. Once the tasks are quiescent another task can
@@ -16,7 +16,7 @@ recoverable error occur. This also allows the checkpointed tasks to be
migrated between nodes in a cluster by copying the gathered information
to another node and restarting the tasks there.
- Sequences of SIGSTOP and SIGCONT are not always sufficient for stopping
+Sequences of SIGSTOP and SIGCONT are not always sufficient for stopping
and resuming tasks in userspace. Both of these signals are observable
from within the tasks we wish to freeze. While SIGSTOP cannot be caught,
blocked, or ignored it can be seen by waiting or ptracing parent tasks.
@@ -37,26 +37,28 @@ demonstrate this problem using nested bash shells:
<at this point 16990 exits and causes 16644 to exit too>
- This happens because bash can observe both signals and choose how it
+This happens because bash can observe both signals and choose how it
responds to them.
- Another example of a program which catches and responds to these
+Another example of a program which catches and responds to these
signals is gdb. In fact any program designed to use ptrace is likely to
have a problem with this method of stopping and resuming tasks.
- In contrast, the cgroup freezer uses the kernel freezer code to
+In contrast, the cgroup freezer uses the kernel freezer code to
prevent the freeze/unfreeze cycle from becoming visible to the tasks
being frozen. This allows the bash example above and gdb to run as
expected.
- The freezer subsystem in the container filesystem defines a file named
+The freezer subsystem in the container filesystem defines a file named
freezer.state. Writing "FROZEN" to the state file will freeze all tasks in the
cgroup. Subsequently writing "THAWED" will unfreeze the tasks in the cgroup.
Reading will return the current state.
+Note it's not allowed to freeze the root cgroup.
+
* Examples of usage :
- # mkdir /containers/freezer
+ # mkdir /containers/
# mount -t cgroup -ofreezer freezer /containers
# mkdir /containers/0
# echo $some_pid > /containers/0/tasks
diff --git a/kernel/cgroup_freezer.c b/kernel/cgroup_freezer.c
index 6605907..6b5c45d 100644
--- a/kernel/cgroup_freezer.c
+++ b/kernel/cgroup_freezer.c
@@ -192,6 +192,9 @@ static void freezer_fork(struct cgroup_subsys *ss, struct task_struct *task)
*/
freezer = task_freezer(task);
+ if (!freezer->css.cgroup->parent)
+ return;
+
spin_lock_irq(&freezer->lock);
BUG_ON(freezer->state == CGROUP_FROZEN);
@@ -330,6 +333,10 @@ static int freezer_write(struct cgroup *cgroup,
int retval;
enum freezer_state goal_state;
+ /* It's not allowed to freeze the root cgroup */
+ if (!cgroup->parent)
+ return -EIO;
+
if (strcmp(buffer, freezer_state_strs[CGROUP_THAWED]) == 0)
goal_state = CGROUP_THAWED;
else if (strcmp(buffer, freezer_state_strs[CGROUP_FROZEN]) == 0)
--
1.5.4.rc3
On Mon, Nov 3, 2008 at 11:35 PM, Li Zefan <[email protected]> wrote:
> With this change, the root cgroup is unfreezable, and writing to
> its freezer.state returns -EIO.
EINVAL might be more consistent with other systems like cpusets.
Or maybe just skip populating the file entirely for the root cgroup?
It's not useful if it can't be written and always returns the same
value.
Paul
Paul Menage wrote:
> On Mon, Nov 3, 2008 at 11:35 PM, Li Zefan <[email protected]> wrote:
>> With this change, the root cgroup is unfreezable, and writing to
>> its freezer.state returns -EIO.
>
> EINVAL might be more consistent with other systems like cpusets.
>
Because writing FREEZE (or other invalid value) to freezer.state returns EIO..
Or that EIO should be also changed to EINVAL.
> Or maybe just skip populating the file entirely for the root cgroup?
> It's not useful if it can't be written and always returns the same
> value.
>
I thought about this. I'll see which way Matt and Cedric prefer.
Hello !
>> Or maybe just skip populating the file entirely for the root cgroup?
>> It's not useful if it can't be written and always returns the same
>> value.
>
> I thought about this. I'll see which way Matt and Cedric prefer.
I think that Paul's suggestion of disabling the root freezer makes sense.
Thanks,
C.
On Tue, Nov 4, 2008 at 12:01 AM, Li Zefan <[email protected]> wrote:
> Paul Menage wrote:
>> On Mon, Nov 3, 2008 at 11:35 PM, Li Zefan <[email protected]> wrote:
>>> With this change, the root cgroup is unfreezable, and writing to
>>> its freezer.state returns -EIO.
>>
>> EINVAL might be more consistent with other systems like cpusets.
>>
>
> Because writing FREEZE (or other invalid value) to freezer.state returns EIO..
>
> Or that EIO should be also changed to EINVAL.
Yes, I think that should be EINVAL too - seems more consistent.
Paul