Received: by 2002:a05:6a10:af89:0:0:0:0 with SMTP id iu9csp3609864pxb; Mon, 24 Jan 2022 13:27:53 -0800 (PST) X-Google-Smtp-Source: ABdhPJxSC/Rb6wg02wKMm+JdE8XJhJG2iScJcdOxI4v5bMvajbHlZoDOQQptojSRIt6UMwFE91hf X-Received: by 2002:a63:cf13:: with SMTP id j19mr12841520pgg.279.1643059673257; Mon, 24 Jan 2022 13:27:53 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1643059673; cv=none; d=google.com; s=arc-20160816; b=YAlTyHr74Es5SpVbtqz31SdquARY+4d87Df6v/rDsZh4op3UPg3IEZA5M2LlR1qQk2 bP3atQ+QVoJ3JSOOGcHLU5UdmpSI7EBygvO3PDtz/+UA9UL/Ov5lWJbNkVuAjSgcf8CP HkkN4jqJoKyp5ax+5xaftiiDi0+gAGNGZ//BfvtfhC9F3swtAfiJvum+7QFRxifPDP0/ YRx9e5Jd+g9HMDEjnAxk6tHR4S3sH7/itEGZaThpWryy+Uh9AIHTebhJDKkP5v3/0A5i XydUCJ3zivrA7bCpPBIcXAmZT6vlQktoHr6/zwj75neg7Y7fIfqBfnaM7hIX94s9sazX w+Tg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=7juGHYjYC8OPdg5QNlhkQZ2yv/SRsVa0+nXdUh+1NSI=; b=x2luYvg/gabwR31ODXrNrAWUgGUyn4KQJu3EHAbBN9Q36sVU2baZeyaHMWv1LrHYZW VNo5N8hlynCF+N84QHEL6+FXb+kEGpPK2K5NErB+X1ilf1g96inrwngKw+XVDDlWGK9p kfCUzHLZvpmih5M3f3/9U2NSGe1hnBSRCkqjGk/FKeVB7iVSOm/xV1cl6nx5hrY0A+1h jqkwy6KcKDnMHxW37kDYWzu7P/NSnWlyJYF+aaEYbjmag6DCnDuf7EvxHotFv2mVz/NP q3StHw7I/pIeDb24z/2qzVuPhCiFCdchvF1Igj5QjoegX/WQ5n6ZmI6cmRFC/mrjDAO8 sL8g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=dRDSrF8N; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id j35si15130588pgm.138.2022.01.24.13.27.41; Mon, 24 Jan 2022 13:27:53 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@linuxfoundation.org header.s=korg header.b=dRDSrF8N; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=linuxfoundation.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1445488AbiAXVD6 (ORCPT + 99 others); Mon, 24 Jan 2022 16:03:58 -0500 Received: from ams.source.kernel.org ([145.40.68.75]:42816 "EHLO ams.source.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1389470AbiAXUlM (ORCPT ); Mon, 24 Jan 2022 15:41:12 -0500 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id B17DEB80FA1; Mon, 24 Jan 2022 20:41:09 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id D5F2DC340E5; Mon, 24 Jan 2022 20:41:07 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=linuxfoundation.org; s=korg; t=1643056868; bh=tg8iEjvzYlmjjMbrWjc40apMLT5XKRdF0J/xCoow65U=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=dRDSrF8NdgXtaj/aGpXXz3chvRlnTOfxwGeay68wft/cdW2jtn2nNjHCkKd6BZNDN kQaMdQvmo24e4bUtg3o+8sor7K2DvjetIogNQwZWs4KGIb5BSivBpSXySr4FyRiZ2l OzTRuXWtG6QyBS25ezF7GlI1sGe10ILxlkiozdNc= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Nicholas Piggin , Laurent Dufour , Michael Ellerman , Sasha Levin Subject: [PATCH 5.15 627/846] powerpc/watchdog: Fix missed watchdog reset due to memory ordering race Date: Mon, 24 Jan 2022 19:42:24 +0100 Message-Id: <20220124184122.663478162@linuxfoundation.org> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20220124184100.867127425@linuxfoundation.org> References: <20220124184100.867127425@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Nicholas Piggin [ Upstream commit 5dad4ba68a2483fc80d70b9dc90bbe16e1f27263 ] It is possible for all CPUs to miss the pending cpumask becoming clear, and then nobody resetting it, which will cause the lockup detector to stop working. It will eventually expire, but watchdog_smp_panic will avoid doing anything if the pending mask is clear and it will never be reset. Order the cpumask clear vs the subsequent test to close this race. Add an extra check for an empty pending mask when the watchdog fires and finds its bit still clear, to try to catch any other possible races or bugs here and keep the watchdog working. The extra test in arch_touch_nmi_watchdog is required to prevent the new warning from firing off. Signed-off-by: Nicholas Piggin Reviewed-by: Laurent Dufour Debugged-by: Laurent Dufour Signed-off-by: Michael Ellerman Link: https://lore.kernel.org/r/20211110025056.2084347-2-npiggin@gmail.com Signed-off-by: Sasha Levin --- arch/powerpc/kernel/watchdog.c | 41 +++++++++++++++++++++++++++++++++- 1 file changed, 40 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/kernel/watchdog.c b/arch/powerpc/kernel/watchdog.c index 3fa6d240bade2..ad94a2c6b7337 100644 --- a/arch/powerpc/kernel/watchdog.c +++ b/arch/powerpc/kernel/watchdog.c @@ -135,6 +135,10 @@ static void set_cpumask_stuck(const struct cpumask *cpumask, u64 tb) { cpumask_or(&wd_smp_cpus_stuck, &wd_smp_cpus_stuck, cpumask); cpumask_andnot(&wd_smp_cpus_pending, &wd_smp_cpus_pending, cpumask); + /* + * See wd_smp_clear_cpu_pending() + */ + smp_mb(); if (cpumask_empty(&wd_smp_cpus_pending)) { wd_smp_last_reset_tb = tb; cpumask_andnot(&wd_smp_cpus_pending, @@ -221,13 +225,44 @@ static void wd_smp_clear_cpu_pending(int cpu, u64 tb) cpumask_clear_cpu(cpu, &wd_smp_cpus_stuck); wd_smp_unlock(&flags); + } else { + /* + * The last CPU to clear pending should have reset the + * watchdog so we generally should not find it empty + * here if our CPU was clear. However it could happen + * due to a rare race with another CPU taking the + * last CPU out of the mask concurrently. + * + * We can't add a warning for it. But just in case + * there is a problem with the watchdog that is causing + * the mask to not be reset, try to kick it along here. + */ + if (unlikely(cpumask_empty(&wd_smp_cpus_pending))) + goto none_pending; } return; } + cpumask_clear_cpu(cpu, &wd_smp_cpus_pending); + + /* + * Order the store to clear pending with the load(s) to check all + * words in the pending mask to check they are all empty. This orders + * with the same barrier on another CPU. This prevents two CPUs + * clearing the last 2 pending bits, but neither seeing the other's + * store when checking if the mask is empty, and missing an empty + * mask, which ends with a false positive. + */ + smp_mb(); if (cpumask_empty(&wd_smp_cpus_pending)) { unsigned long flags; +none_pending: + /* + * Double check under lock because more than one CPU could see + * a clear mask with the lockless check after clearing their + * pending bits. + */ wd_smp_lock(&flags); if (cpumask_empty(&wd_smp_cpus_pending)) { wd_smp_last_reset_tb = tb; @@ -318,8 +353,12 @@ void arch_touch_nmi_watchdog(void) { unsigned long ticks = tb_ticks_per_usec * wd_timer_period_ms * 1000; int cpu = smp_processor_id(); - u64 tb = get_tb(); + u64 tb; + if (!cpumask_test_cpu(cpu, &watchdog_cpumask)) + return; + + tb = get_tb(); if (tb - per_cpu(wd_timer_tb, cpu) >= ticks) { per_cpu(wd_timer_tb, cpu) = tb; wd_smp_clear_cpu_pending(cpu, tb); -- 2.34.1