Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755781AbbGVFYu (ORCPT ); Wed, 22 Jul 2015 01:24:50 -0400 Received: from mail-wi0-f177.google.com ([209.85.212.177]:37535 "EHLO mail-wi0-f177.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751836AbbGVFYt (ORCPT ); Wed, 22 Jul 2015 01:24:49 -0400 Message-ID: <1437542686.3106.55.camel@gmail.com> Subject: [patch] workqueue: schedule WORK_CPU_UNBOUND work on wq_unbound_cpumask CPUs From: Mike Galbraith To: Tejun Heo , Lai Jiangshan Cc: Frederic Weisbecker , Daniel Bristot de Oliveira , LKML , Rik van Riel , "Luis Claudio R. Goncalves" Date: Wed, 22 Jul 2015 07:24:46 +0200 In-Reply-To: <1437468925.12755.57.camel@gmail.com> References: <9e53de7c91c885ee255e16ee25f401d9eedf08d9.1437067317.git.bristot@redhat.com> <20150716192448.GY15934@mtj.duckdns.org> <1437107190.3438.23.camel@gmail.com> <20150717152720.GD15934@mtj.duckdns.org> <1437153348.5860.32.camel@gmail.com> <20150718133602.GA3041@lerouge> <1437292973.3505.83.camel@gmail.com> <1437468925.12755.57.camel@gmail.com> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.12.11 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2871 Lines: 81 On Tue, 2015-07-21 at 10:55 +0200, Mike Galbraith wrote: > On Sun, 2015-07-19 at 10:02 +0200, Mike Galbraith wrote: > > > Why do we do nothing about these allegedly unbound work items? > > My box seems to think the answer is: no reason other than nobody having > asked the source to please not do that. Guess I'll go ask a NUMA box. My [128] socket boxen show zero signs of caring, and it's dirt simple, so it's no longer an experiment. Fly or die little patchlet... WORK_CPU_UNBOUND work items queued to a bound workqueue always run locally. This is a good thing normally, but not when the user has asked us to keep unbound work away from certain CPUs. Round robin these to wq_unbound_cpumask CPUs instead, as perturbation avoidance trumps performance. Signed-off-by: Mike Galbraith --- kernel/workqueue.c | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-) --- a/kernel/workqueue.c +++ b/kernel/workqueue.c @@ -301,6 +301,9 @@ static bool workqueue_freezing; /* PL: static cpumask_var_t wq_unbound_cpumask; /* PL: low level cpumask for all unbound wqs */ +/* CPU where WORK_CPU_UNBOUND work was last round robin scheduled from this CPU */ +static DEFINE_PER_CPU(unsigned int, wq_unbound_rr_cpu_last); + /* the per-cpu worker pools */ static DEFINE_PER_CPU_SHARED_ALIGNED(struct worker_pool [NR_STD_WORKER_POOLS], cpu_worker_pools); @@ -1294,6 +1297,24 @@ static bool is_chained_work(struct workq return worker && worker->current_pwq->wq == wq; } +/* + * When queueing WORK_CPU_UNBOUND work to a !WQ_UNBOUND queue, round + * robin among wq_unbound_cpumask to avoid perturbing sensitive tasks. + */ +static unsigned int select_round_robin_cpu(unsigned int cpu) +{ + if (cpumask_test_cpu(cpu, wq_unbound_cpumask)) + return cpu; + if (cpumask_empty(wq_unbound_cpumask)) + return cpu; + cpu = __this_cpu_read(wq_unbound_rr_cpu_last); + cpu = cpumask_next_and(cpu, wq_unbound_cpumask, cpu_online_mask); + if (cpu >= nr_cpu_ids) + cpu = 0; + __this_cpu_write(wq_unbound_rr_cpu_last, cpu); + return cpu; +} + static void __queue_work(int cpu, struct workqueue_struct *wq, struct work_struct *work) { @@ -1322,9 +1343,11 @@ static void __queue_work(int cpu, struct cpu = raw_smp_processor_id(); /* pwq which will be used unless @work is executing elsewhere */ - if (!(wq->flags & WQ_UNBOUND)) + if (!(wq->flags & WQ_UNBOUND)) { + if (req_cpu == WORK_CPU_UNBOUND) + cpu = select_round_robin_cpu(cpu); pwq = per_cpu_ptr(wq->cpu_pwqs, cpu); - else + } else pwq = unbound_pwq_by_node(wq, cpu_to_node(cpu)); /* -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/