Received: by 2002:a05:6520:4d:b0:139:a872:a4c9 with SMTP id i13csp2567306lkm; Mon, 20 Sep 2021 18:54:53 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzXPMwK0NlS7Nz42eJz9soAcTgJn26HFGnYurnvDf8RJ/Rt9x9Ba3ZGYj0p1s66Q1mcVLDi X-Received: by 2002:a5d:94c4:: with SMTP id y4mr9089902ior.131.1632188890420; Mon, 20 Sep 2021 18:48:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1632188890; cv=none; d=google.com; s=arc-20160816; b=fPjddx3MVMxXBgKqSsUblBwqzUrZrO3nbF+rPpnX3/WKRQp592vMuWtedLs+O7MVBM R3gA1sZBkIeucakd5HEoo6MILRCQnY8S+litZBRMCAWx4MtGWjEMCL+WStuREV56XIt/ wt6xk1TfQ9ImHlISbvwu1KeH58+tD5bLxecc2O393CXlpjqXVsmDWN3tzysOTnmu97Wu A/gW6wR1YhzOVXHkQtiVWq8Gg0YE4fCuBOysWrjIwN0+At0uu8dEogL1UmsCMhXx5Vr6 XjdJXy+MKyth3TjOX5G17xzBNSnoEvWD3aqs5/rYfCO6jGzTPGWSrdIp1EizIjxIAn5F bwmg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=dFcFbiR/vIEzjull6dKVyTKSnlusYL8hH8UvIR7vHR8=; b=KAYYFlpQoxgwoVUiAWOpXaSmImIuxVIHKu0ukskDqWFfDNGXVhtRDi8aj1cAgGlRem cm9zv42sVPI+gAmoFDb0lmgr+3TEpTNkk+KNTowOzFhS5UuwW48yj+rJ5BwVBfMwSDgV NtnexVJjKrcmDxZobCUiq40XxnVkcrkg/zKqGMYAQ29AccPp41GEPImlMPhb6xpFcvJa VLS4u5uIxak/IJX/+jKfu3bYo91gNLFV3uhXDjIHCVqAYYN5MYq+Ow4uxQQAsznB3cYs wOQiXZ8atPWOE0X9k4knvSiI1ZsE6GihUIgyOTwojMBb1dzds9GXVWYaC+GpoD4RFgwM p41g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=zCHDfdis; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id s9si14174695jat.70.2021.09.20.18.47.59; Mon, 20 Sep 2021 18:48:10 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=zCHDfdis; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1348588AbhITRkB (ORCPT + 99 others); Mon, 20 Sep 2021 13:40:01 -0400 Received: from mail.kernel.org ([198.145.29.99]:40198 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1352041AbhITRha (ORCPT ); Mon, 20 Sep 2021 13:37:30 -0400 Received: by mail.kernel.org (Postfix) with ESMTPSA id 390A761B2F; Mon, 20 Sep 2021 17:06:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1632157613; bh=oCgVy4C0ao2CcSv6Ia0kN4a6gXzVhiETyH7vLIdD6a4=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=zCHDfdis4/n5TsBGvsOcW+YdToH4KSVEcfVG/SOvma7rV/0nh1ASpdXeT+HZgDTk5 Ep7B2tEyIgnO8ypoROR2IA7X6XonofAuO0hSqzCIc3HG2awNlm35aGSrZ8FiR0KDi7 Sa5I2qSU22umgdWOlaB6Gy0sxjqOJNAuYI/WQvgU= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Lorenzo Colitti , Thomas Gleixner , Sasha Levin Subject: [PATCH 4.19 039/293] hrtimer: Avoid double reprogramming in __hrtimer_start_range_ns() Date: Mon, 20 Sep 2021 18:40:01 +0200 Message-Id: <20210920163934.615753497@linuxfoundation.org> X-Mailer: git-send-email 2.33.0 In-Reply-To: <20210920163933.258815435@linuxfoundation.org> References: <20210920163933.258815435@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Thomas Gleixner [ Upstream commit 627ef5ae2df8eeccb20d5af0e4cfa4df9e61ed28 ] If __hrtimer_start_range_ns() is invoked with an already armed hrtimer then the timer has to be canceled first and then added back. If the timer is the first expiring timer then on removal the clockevent device is reprogrammed to the next expiring timer to avoid that the pending expiry fires needlessly. If the new expiry time ends up to be the first expiry again then the clock event device has to reprogrammed again. Avoid this by checking whether the timer is the first to expire and in that case, keep the timer on the current CPU and delay the reprogramming up to the point where the timer has been enqueued again. Reported-by: Lorenzo Colitti Signed-off-by: Thomas Gleixner Link: https://lore.kernel.org/r/20210713135157.873137732@linutronix.de Signed-off-by: Sasha Levin --- kernel/time/hrtimer.c | 60 ++++++++++++++++++++++++++++++++++++++----- 1 file changed, 53 insertions(+), 7 deletions(-) diff --git a/kernel/time/hrtimer.c b/kernel/time/hrtimer.c index 0e04b24cec81..32ee24f5142a 100644 --- a/kernel/time/hrtimer.c +++ b/kernel/time/hrtimer.c @@ -1020,12 +1020,13 @@ static void __remove_hrtimer(struct hrtimer *timer, * remove hrtimer, called with base lock held */ static inline int -remove_hrtimer(struct hrtimer *timer, struct hrtimer_clock_base *base, bool restart) +remove_hrtimer(struct hrtimer *timer, struct hrtimer_clock_base *base, + bool restart, bool keep_local) { u8 state = timer->state; if (state & HRTIMER_STATE_ENQUEUED) { - int reprogram; + bool reprogram; /* * Remove the timer and force reprogramming when high @@ -1038,8 +1039,16 @@ remove_hrtimer(struct hrtimer *timer, struct hrtimer_clock_base *base, bool rest debug_deactivate(timer); reprogram = base->cpu_base == this_cpu_ptr(&hrtimer_bases); + /* + * If the timer is not restarted then reprogramming is + * required if the timer is local. If it is local and about + * to be restarted, avoid programming it twice (on removal + * and a moment later when it's requeued). + */ if (!restart) state = HRTIMER_STATE_INACTIVE; + else + reprogram &= !keep_local; __remove_hrtimer(timer, base, state, reprogram); return 1; @@ -1093,9 +1102,31 @@ static int __hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim, struct hrtimer_clock_base *base) { struct hrtimer_clock_base *new_base; + bool force_local, first; - /* Remove an active timer from the queue: */ - remove_hrtimer(timer, base, true); + /* + * If the timer is on the local cpu base and is the first expiring + * timer then this might end up reprogramming the hardware twice + * (on removal and on enqueue). To avoid that by prevent the + * reprogram on removal, keep the timer local to the current CPU + * and enforce reprogramming after it is queued no matter whether + * it is the new first expiring timer again or not. + */ + force_local = base->cpu_base == this_cpu_ptr(&hrtimer_bases); + force_local &= base->cpu_base->next_timer == timer; + + /* + * Remove an active timer from the queue. In case it is not queued + * on the current CPU, make sure that remove_hrtimer() updates the + * remote data correctly. + * + * If it's on the current CPU and the first expiring timer, then + * skip reprogramming, keep the timer local and enforce + * reprogramming later if it was the first expiring timer. This + * avoids programming the underlying clock event twice (once at + * removal and once after enqueue). + */ + remove_hrtimer(timer, base, true, force_local); if (mode & HRTIMER_MODE_REL) tim = ktime_add_safe(tim, base->get_time()); @@ -1105,9 +1136,24 @@ static int __hrtimer_start_range_ns(struct hrtimer *timer, ktime_t tim, hrtimer_set_expires_range_ns(timer, tim, delta_ns); /* Switch the timer base, if necessary: */ - new_base = switch_hrtimer_base(timer, base, mode & HRTIMER_MODE_PINNED); + if (!force_local) { + new_base = switch_hrtimer_base(timer, base, + mode & HRTIMER_MODE_PINNED); + } else { + new_base = base; + } + + first = enqueue_hrtimer(timer, new_base, mode); + if (!force_local) + return first; - return enqueue_hrtimer(timer, new_base, mode); + /* + * Timer was forced to stay on the current CPU to avoid + * reprogramming on removal and enqueue. Force reprogram the + * hardware by evaluating the new first expiring timer. + */ + hrtimer_force_reprogram(new_base->cpu_base, 1); + return 0; } /** @@ -1168,7 +1214,7 @@ int hrtimer_try_to_cancel(struct hrtimer *timer) base = lock_hrtimer_base(timer, &flags); if (!hrtimer_callback_running(timer)) - ret = remove_hrtimer(timer, base, false); + ret = remove_hrtimer(timer, base, false, false); unlock_hrtimer_base(timer, &flags); -- 2.30.2