Received: by 10.213.65.68 with SMTP id h4csp1771002imn; Mon, 19 Mar 2018 12:49:09 -0700 (PDT) X-Google-Smtp-Source: AG47ELsYOl2PDvXEsOm/LmQ4Tc2IPtJlhYpE70ymTXyZSPzlHUsVOTO1awLO/JbXeutOyGMpMfYO X-Received: by 10.101.75.202 with SMTP id p10mr10114110pgr.339.1521488949217; Mon, 19 Mar 2018 12:49:09 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1521488949; cv=none; d=google.com; s=arc-20160816; b=Y0WIiqWEC0sqJiAqBCTP2Me8xe/f61qj7LLXjEpLixJFgjK6S7WOBynY8ddHvLqTqu QYyavHmx6jYhfeoSsArvULL7u0F4qIM+tpPBL2+4Sy+Mn9XFA3ARLx8J4fDowvTtRyzE mCdviC1cWOUHBdkoG1F+Sl7O76QSdYYIqKb5s4vLHwskCcxOuXaLkPtunnEZnfC+LS7e MMZd/eroLZ2iKWsdZdm9D0Fj3WKn2ti2n3QBkUbTJP8uSgm5WUyTjDY/sRSRsSdXs9iU cDeaRIlnRvj9BP3agG6o/uDK4rCzNdQhDISyGj0IdcX1tlz+HftE7i+lgskP2oLUbLHB 26kg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:mime-version:user-agent:references :in-reply-to:message-id:date:subject:cc:to:from :arc-authentication-results; bh=lmQeJJhPu4CO3hprvlexu3M4cpmDeEjXX2cE3/HW/jY=; b=TBmAOeNJwzLwU0B+JdhDvtB5qWpsrFgoRGSodJDRZhPZxQoBVi4ElsvTb0kutPBJ12 ENCcmHajaYKYuYlSy0s8c1JQQOKpjvqTCsgQtaIDmSNIX873HK1TkiXm+x4qFCqBKgwm ak2v0rLzgyXRZUlo0lqDzW4i3LMTG2Z/9S8tlGs4tPrIKfOdXLCnCQF6mc4IMZOy/TDV 92n2YKpkLx2XmgeeDNyeGfXG9M5Rl2h8sAsF0u6gpnNWc/GakogBOfO0AMUYJwkKQOrB kLlyg2EIAtRe2TLNcVc5nlmiGbiTl+V+lMKbBLQf82aKmv0yEWY8p3StmsN2AmrOVOYY Q87A== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id e63si459146pfb.268.2018.03.19.12.48.55; Mon, 19 Mar 2018 12:49:09 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S969640AbeCSTrO (ORCPT + 99 others); Mon, 19 Mar 2018 15:47:14 -0400 Received: from mail.linuxfoundation.org ([140.211.169.12]:47690 "EHLO mail.linuxfoundation.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1030995AbeCSSU3 (ORCPT ); Mon, 19 Mar 2018 14:20:29 -0400 Received: from localhost (LFbn-1-12247-202.w90-92.abo.wanadoo.fr [90.92.61.202]) by mail.linuxfoundation.org (Postfix) with ESMTPSA id 994D61090; Mon, 19 Mar 2018 18:20:28 +0000 (UTC) From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Tom Hromatka , Rob Gardner , John Stultz , Sasha Levin Subject: [PATCH 4.9 057/241] sysrq: Reset the watchdog timers while displaying high-resolution timers Date: Mon, 19 Mar 2018 19:05:22 +0100 Message-Id: <20180319180753.569858459@linuxfoundation.org> X-Mailer: git-send-email 2.16.2 In-Reply-To: <20180319180751.172155436@linuxfoundation.org> References: <20180319180751.172155436@linuxfoundation.org> User-Agent: quilt/0.65 X-stable: review MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org 4.9-stable review patch. If anyone has any objections, please let me know. ------------------ From: Tom Hromatka [ Upstream commit 0107042768658fea9f5f5a9c00b1c90f5dab6a06 ] On systems with a large number of CPUs, running sysrq- can cause watchdog timeouts. There are two slow sections of code in the sysrq- path in timer_list.c. 1. print_active_timers() - This function is called by print_cpu() and contains a slow goto loop. On a machine with hundreds of CPUs, this loop took approximately 100ms for the first CPU in a NUMA node. (Subsequent CPUs in the same node ran much quicker.) The total time to print all of the CPUs is ultimately long enough to trigger the soft lockup watchdog. 2. print_tickdevice() - This function outputs a large amount of textual information. This function also took approximately 100ms per CPU. Since sysrq- is not a performance critical path, there should be no harm in touching the nmi watchdog in both slow sections above. Touching it in just one location was insufficient on systems with hundreds of CPUs as occasional timeouts were still observed during testing. This issue was observed on an Oracle T7 machine with 128 CPUs, but I anticipate it may affect other systems with similarly large numbers of CPUs. Signed-off-by: Tom Hromatka Reviewed-by: Rob Gardner Signed-off-by: John Stultz Signed-off-by: Sasha Levin Signed-off-by: Greg Kroah-Hartman --- kernel/time/timer_list.c | 6 ++++++ 1 file changed, 6 insertions(+) --- a/kernel/time/timer_list.c +++ b/kernel/time/timer_list.c @@ -16,6 +16,7 @@ #include #include #include +#include #include @@ -96,6 +97,9 @@ print_active_timers(struct seq_file *m, next_one: i = 0; + + touch_nmi_watchdog(); + raw_spin_lock_irqsave(&base->cpu_base->lock, flags); curr = timerqueue_getnext(&base->active); @@ -207,6 +211,8 @@ print_tickdevice(struct seq_file *m, str { struct clock_event_device *dev = td->evtdev; + touch_nmi_watchdog(); + SEQ_printf(m, "Tick Device: mode: %d\n", td->mode); if (cpu < 0) SEQ_printf(m, "Broadcast device\n");