Received: by 2002:a25:4158:0:0:0:0:0 with SMTP id o85csp893612yba; Thu, 18 Apr 2019 11:20:11 -0700 (PDT) X-Google-Smtp-Source: APXvYqy/ygXbHM0EIc+Smw54hP0Vtgv6Kq7WLsmpKYCdf9uLHpA0LjMhuNSPko5HqsHmwWscFNbx X-Received: by 2002:a65:5184:: with SMTP id h4mr11246464pgq.109.1555611611856; Thu, 18 Apr 2019 11:20:11 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1555611611; cv=none; d=google.com; s=arc-20160816; b=mrpEM/ZBbSnTQeAQgQSUXzcnTpfIwCIgeo+ew/u8ChX2m3G0pgeuDK6rW4UcByj0K5 LRm3JJ+EYcOCLA2dDRVaN0kdGwG4eR1KkLs7oopDrqCZTadaBsq0/7UTTSabqt0N15v2 6ahguJFwPj6CSrdHCWfUD/A+0WbYNRAB1MvmhixxdLkD6qVYzi1/eqf63gdgluH9unOh X/9ttdWpUune1ILYeD/JLCZDgle+6BSGsnh/Hcz9uSt1Q0A+ZsllgOFzo3tFvLDucU9c wdm58i6jhJgD4DnuHxJ4B+wGmRLqlMHc7+bY05wDxRZuey5p6pzzPS6zQljFi4LyctmE GG9A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=TWAszTSHdJZYelhN9xThYWZCBnv65H7askXK3BLiJLI=; b=eEOqRbynwhbLIRyQ0Pctn5CymnwV+0j4hMRVLY7DV9IJ5JKI5f4S6mF+NyG4KSGxFG 0wc8ReqZdbcmUl+E0qcZadpZE/ExZFSGAs8iiKipFujl1HFUr295lwstISUJ33Iq88PT QE5JTbWzH4TgcsZGYr5u7+qcVlimEBprMQgR5/59OmGPlcJSJhthC0BMX3f1c88LU4X+ zsxITqknbRefPJ8g3n2YzIzs98WVQ6Zl0engOABx/n3FXV2rViwRBZatuGQDMBgUM29h /Ni4Kp/yo++ynTOQmC6ivhk8mYOQluogzdyAldTTzyCOTuehKak5tf2yBkH4PlaPyaBH XwYg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=SlU87ZUj; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id e1si2916848pfc.149.2019.04.18.11.19.56; Thu, 18 Apr 2019 11:20:11 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=SlU87ZUj; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2404148AbfDRSKk (ORCPT + 99 others); Thu, 18 Apr 2019 14:10:40 -0400 Received: from mail.kernel.org ([198.145.29.99]:42448 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2391608AbfDRSKh (ORCPT ); Thu, 18 Apr 2019 14:10:37 -0400 Received: from localhost (83-86-89-107.cable.dynamic.v4.ziggo.nl [83.86.89.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 1713A20675; Thu, 18 Apr 2019 18:10:36 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1555611036; bh=wOz438wfGRJ0y8eOASyS7ZjEoL1gxH/yMtbNGHV09zU=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=SlU87ZUjjfxr1cwCV6kvwx1rqGEid1zWjt9E6giaDNl5lWAPpD0+iBq4TZ7WVBf7i 51M4ybsgVbksbgR1fPlrGDkiVl5bJmQzckiERfC5TUa53Jxv4ysysCBodV5lFOUQS+ sjFkOn2l/HDg5wVe8ZTFWmHrudggy0JFgNuTzrBk= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Vincent Wang , "Peter Zijlstra (Intel)" , "Rafael J. Wysocki" , Andy Lutomirski , Borislav Petkov , Chunyan Zhang , Dave Hansen , "H. Peter Anvin" , Linus Torvalds , Quentin Perret , "Rafael J. Wysocki" , Rik van Riel , Thomas Gleixner , Ingo Molnar , Sasha Levin Subject: [PATCH 5.0 27/93] sched/cpufreq: Fix 32-bit math overflow Date: Thu, 18 Apr 2019 19:57:05 +0200 Message-Id: <20190418160439.639670384@linuxfoundation.org> X-Mailer: git-send-email 2.21.0 In-Reply-To: <20190418160436.781762249@linuxfoundation.org> References: <20190418160436.781762249@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org [ Upstream commit a23314e9d88d89d49e69db08f60b7caa470f04e1 ] Vincent Wang reported that get_next_freq() has a mult overflow bug on 32-bit platforms in the IOWAIT boost case, since in that case {util,max} are in freq units instead of capacity units. Solve this by moving the IOWAIT boost to capacity units. And since this means @max is constant; simplify the code. Reported-by: Vincent Wang Tested-by: Vincent Wang Signed-off-by: Peter Zijlstra (Intel) Acked-by: Rafael J. Wysocki Cc: Andy Lutomirski Cc: Borislav Petkov Cc: Chunyan Zhang Cc: Dave Hansen Cc: H. Peter Anvin Cc: Linus Torvalds Cc: Peter Zijlstra Cc: Quentin Perret Cc: Rafael J. Wysocki Cc: Rik van Riel Cc: Thomas Gleixner Link: https://lkml.kernel.org/r/20190305083202.GU32494@hirez.programming.kicks-ass.net Signed-off-by: Ingo Molnar Signed-off-by: Sasha Levin --- kernel/sched/cpufreq_schedutil.c | 59 ++++++++++++++------------------ 1 file changed, 25 insertions(+), 34 deletions(-) diff --git a/kernel/sched/cpufreq_schedutil.c b/kernel/sched/cpufreq_schedutil.c index 033ec7c45f13..1ccf77f6d346 100644 --- a/kernel/sched/cpufreq_schedutil.c +++ b/kernel/sched/cpufreq_schedutil.c @@ -48,10 +48,10 @@ struct sugov_cpu { bool iowait_boost_pending; unsigned int iowait_boost; - unsigned int iowait_boost_max; u64 last_update; unsigned long bw_dl; + unsigned long min; unsigned long max; /* The field below is for single-CPU policies only: */ @@ -303,8 +303,7 @@ static bool sugov_iowait_reset(struct sugov_cpu *sg_cpu, u64 time, if (delta_ns <= TICK_NSEC) return false; - sg_cpu->iowait_boost = set_iowait_boost - ? sg_cpu->sg_policy->policy->min : 0; + sg_cpu->iowait_boost = set_iowait_boost ? sg_cpu->min : 0; sg_cpu->iowait_boost_pending = set_iowait_boost; return true; @@ -344,14 +343,13 @@ static void sugov_iowait_boost(struct sugov_cpu *sg_cpu, u64 time, /* Double the boost at each request */ if (sg_cpu->iowait_boost) { - sg_cpu->iowait_boost <<= 1; - if (sg_cpu->iowait_boost > sg_cpu->iowait_boost_max) - sg_cpu->iowait_boost = sg_cpu->iowait_boost_max; + sg_cpu->iowait_boost = + min_t(unsigned int, sg_cpu->iowait_boost << 1, SCHED_CAPACITY_SCALE); return; } /* First wakeup after IO: start with minimum boost */ - sg_cpu->iowait_boost = sg_cpu->sg_policy->policy->min; + sg_cpu->iowait_boost = sg_cpu->min; } /** @@ -373,47 +371,38 @@ static void sugov_iowait_boost(struct sugov_cpu *sg_cpu, u64 time, * This mechanism is designed to boost high frequently IO waiting tasks, while * being more conservative on tasks which does sporadic IO operations. */ -static void sugov_iowait_apply(struct sugov_cpu *sg_cpu, u64 time, - unsigned long *util, unsigned long *max) +static unsigned long sugov_iowait_apply(struct sugov_cpu *sg_cpu, u64 time, + unsigned long util, unsigned long max) { - unsigned int boost_util, boost_max; + unsigned long boost; /* No boost currently required */ if (!sg_cpu->iowait_boost) - return; + return util; /* Reset boost if the CPU appears to have been idle enough */ if (sugov_iowait_reset(sg_cpu, time, false)) - return; + return util; - /* - * An IO waiting task has just woken up: - * allow to further double the boost value - */ - if (sg_cpu->iowait_boost_pending) { - sg_cpu->iowait_boost_pending = false; - } else { + if (!sg_cpu->iowait_boost_pending) { /* - * Otherwise: reduce the boost value and disable it when we - * reach the minimum. + * No boost pending; reduce the boost value. */ sg_cpu->iowait_boost >>= 1; - if (sg_cpu->iowait_boost < sg_cpu->sg_policy->policy->min) { + if (sg_cpu->iowait_boost < sg_cpu->min) { sg_cpu->iowait_boost = 0; - return; + return util; } } + sg_cpu->iowait_boost_pending = false; + /* - * Apply the current boost value: a CPU is boosted only if its current - * utilization is smaller then the current IO boost level. + * @util is already in capacity scale; convert iowait_boost + * into the same scale so we can compare. */ - boost_util = sg_cpu->iowait_boost; - boost_max = sg_cpu->iowait_boost_max; - if (*util * boost_max < *max * boost_util) { - *util = boost_util; - *max = boost_max; - } + boost = (sg_cpu->iowait_boost * max) >> SCHED_CAPACITY_SHIFT; + return max(boost, util); } #ifdef CONFIG_NO_HZ_COMMON @@ -460,7 +449,7 @@ static void sugov_update_single(struct update_util_data *hook, u64 time, util = sugov_get_util(sg_cpu); max = sg_cpu->max; - sugov_iowait_apply(sg_cpu, time, &util, &max); + util = sugov_iowait_apply(sg_cpu, time, util, max); next_f = get_next_freq(sg_policy, util, max); /* * Do not reduce the frequency if the CPU has not been idle @@ -500,7 +489,7 @@ static unsigned int sugov_next_freq_shared(struct sugov_cpu *sg_cpu, u64 time) j_util = sugov_get_util(j_sg_cpu); j_max = j_sg_cpu->max; - sugov_iowait_apply(j_sg_cpu, time, &j_util, &j_max); + j_util = sugov_iowait_apply(j_sg_cpu, time, j_util, j_max); if (j_util * max > j_max * util) { util = j_util; @@ -837,7 +826,9 @@ static int sugov_start(struct cpufreq_policy *policy) memset(sg_cpu, 0, sizeof(*sg_cpu)); sg_cpu->cpu = cpu; sg_cpu->sg_policy = sg_policy; - sg_cpu->iowait_boost_max = policy->cpuinfo.max_freq; + sg_cpu->min = + (SCHED_CAPACITY_SCALE * policy->cpuinfo.min_freq) / + policy->cpuinfo.max_freq; } for_each_cpu(cpu, policy->cpus) { -- 2.19.1