2011-05-07 06:32:54

by Paul Turner

Subject: [patch 12/15] sched: migrate throttled tasks on HOTPLUG

Throttled tasks are invisible to cpu-offline since they are not eligible for
selection by pick_next_task(). The regular 'escape' path for a thread that is
blocked at offline is via ttwu->select_task_rq, however this will not handle a
throttled group since there are no individual thread wakeups on an unthrottle.

Resolve this by unthrottling offline cpus so that threads can be migrated.

Signed-off-by: Paul Turner <[email protected]>
---
kernel/sched.c | 29 +++++++++++++++++++++++++++++
1 file changed, 29 insertions(+)

Index: tip/kernel/sched.c
===================================================================
--- tip.orig/kernel/sched.c
+++ tip/kernel/sched.c
@@ -6145,6 +6145,32 @@ static void calc_global_load_remove(stru
rq->calc_load_active = 0;
}

+#ifdef CONFIG_CFS_BANDWIDTH
+static void unthrottle_offline_cfs_rqs(struct rq *rq)
+{
+ struct cfs_rq *cfs_rq;
+
+ for_each_leaf_cfs_rq(rq, cfs_rq) {
+ struct cfs_bandwidth *cfs_b = tg_cfs_bandwidth(cfs_rq->tg);
+
+ if (!cfs_rq->runtime_enabled)
+ continue;
+
+ /*
+ * clock_task is not advancing so we just need to make sure
+ * there's some valid quota amount
+ */
+ cfs_rq->runtime_remaining = cfs_b->quota;
+ if (cfs_rq_throttled(cfs_rq))
+ unthrottle_cfs_rq(cfs_rq);
+ }
+}
+#else
+static void unthrottle_offline_cfs_rqs(struct rq *rq)
+{
+}
+#endif
+
/*
* Migrate all tasks from the rq, sleeping tasks will be migrated by
* try_to_wake_up()->select_task_rq().
@@ -6170,6 +6196,9 @@ static void migrate_tasks(unsigned int d
*/
rq->stop = NULL;

+ /* Ensure any throttled groups are reachable by pick_next_task */
+ unthrottle_offline_cfs_rqs(rq);
+
for ( ; ; ) {
/*
* There's this thread running, bail when that's the only

2011-05-10 07:27:43

by Hidetoshi Seto

Subject: Re: [patch 12/15] sched: migrate throttled tasks on HOTPLUG

(2011/05/03 18:28), Paul Turner wrote:
> +#else
> +static void unthrottle_offline_cfs_rqs(struct rq *rq)
> +{
> +}
> +#endif
> +

Nit: To follow others, alternative style is in a line:

+static void unthrottle_offline_cfs_rqs(struct rq *rq) {}

Reviewed-by: Hidetoshi Seto <[email protected]>


Thanks,
H.Seto

2011-05-11 16:12:55

by Paul Turner

Subject: Re: [patch 12/15] sched: migrate throttled tasks on HOTPLUG

On Tue, May 10, 2011 at 12:27 AM, Hidetoshi Seto
<[email protected]> wrote:
> (2011/05/03 18:28), Paul Turner wrote:
>> +#else
>> +static void unthrottle_offline_cfs_rqs(struct rq *rq)
>> +{
>> +}
>> +#endif
>> +
>
> Nit: To follow others, alternative style is in a line:
>
> +static void unthrottle_offline_cfs_rqs(struct rq *rq) {}
>

Agree, updated. Thanks

> Reviewed-by: Hidetoshi Seto <[email protected]>
>
>
> Thanks,
> H.Seto
>
>