Received: by 2002:a05:7412:5112:b0:fa:6e18:a558 with SMTP id fm18csp426417rdb; Tue, 23 Jan 2024 04:12:43 -0800 (PST) X-Google-Smtp-Source: AGHT+IFlzs8nT0g3AyxJrX0Prw6+uzrWoAh6QZJhNo+Ouo9GIAdREJRCCVGQ3PQX2d85fRLM5vxc X-Received: by 2002:a05:620a:14b4:b0:783:3496:967a with SMTP id x20-20020a05620a14b400b007833496967amr6895562qkj.43.1706011963580; Tue, 23 Jan 2024 04:12:43 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706011963; cv=pass; d=google.com; s=arc-20160816; b=QP8/b7F0jgpYBG1xuIAvrRzFhrz4Z8Sh6vImGCBG44DYx6NZB06MvbTyOclylWbjYd UkqUnxXfnmNV9mgMqDYyoJRPxptChHoKN1qCHVdZhL1uU56czd/cMbdbZILj5amF5kRV W1zrl4hNPJwqW113pv/JehmF14/BO4gzd1B/kJI/gC8HEUupSjb2MKBGjwCMIqQgQGd7 WGBCja6BlZDyhAi1hVRt/KqVu1a+eZ9SBjcW8FEsqlOne1DJ3Vx3oheyGAoK37YC/JYT FJl/Q7xv8bKEdK1LkHLcssKKHJOEBs/uBypa4lWTStla32etUnFdp0sC3z+JbpFPz4yy 0Eow== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from; bh=xlEnTNFH2RywJaKbfEqW1WN2uM8mKX/KCd4/IefmLrk=; fh=E8Kf3tAYQUJQiYwjk7DEi5yT6N0tsnUPOgoJKt1bx2o=; b=rpKctR7eFa1zdLSU/gffSQRcWsncnAGffBJ4ZMPn8pvx8qQV4JdtVZu75xt2I3Yvaq GaueJ+hrvzWGw+Sjv2E7f4hEjosu2rcR3TsTjkFY9rkB+UO2JUIeGAYywoPo0y0nXIla GmfriVRFtkQySlbLmpiklltHq8RlkdF5/Tm9kKkfgUQUFE3BEiTvF4rTY0Oxt5989Wrh 8ngrInt91DUtsxtxLv3uym1ppX3miWLuTBDCTiqgELXL5Wtd8z39lBcBPgxm8/PTe5DB OVF3yVOXJgn4iJfoVMs4NKu466XZtq1ypdpOyT3q17ZzL7FAbXyEHoVVYGo7qfCAOMse E3Rw== ARC-Authentication-Results: i=2; mx.google.com; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-35233-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-35233-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Return-Path: Received: from ny.mirrors.kernel.org (ny.mirrors.kernel.org. [147.75.199.223]) by mx.google.com with ESMTPS id z6-20020ae9f446000000b00783a96b5ee9si1509724qkl.92.2024.01.23.04.12.43 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 23 Jan 2024 04:12:43 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-35233-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223; Authentication-Results: mx.google.com; arc=pass (i=1 spf=pass spfdomain=linux.alibaba.com dmarc=pass fromdomain=linux.alibaba.com); spf=pass (google.com: domain of linux-kernel+bounces-35233-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) smtp.mailfrom="linux-kernel+bounces-35233-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=alibaba.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ny.mirrors.kernel.org (Postfix) with ESMTPS id 3ADDC1C22F64 for ; Tue, 23 Jan 2024 12:12:43 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 751645DF01; Tue, 23 Jan 2024 12:12:36 +0000 (UTC) Received: from out30-112.freemail.mail.aliyun.com (out30-112.freemail.mail.aliyun.com [115.124.30.112]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 304205D8EB for ; Tue, 23 Jan 2024 12:12:31 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=115.124.30.112 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706011956; cv=none; b=rWN1PzWf39gZ8vpQJ4GK/tsmv58yTUwEOvCy0ZjLfyOj6D77rNAX4Q3da8Z9YBr8Gb/S1DT4eYQ1XCZanWVqYu4/swteqE0/NrW8HAuwzJ59QSiwsU4u/oLRAV5Kj81VQHjMLVxO/A+aaurag8m86vwO94HJo3nir7qytfLz0i0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706011956; c=relaxed/simple; bh=aoKMubrs7V0U9xeWv4qjkRfY0jZgwBiuAWXb6vO6uq0=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version:Content-Type; b=eWv3Flbu8tyx0fx142N52Bm01apymZRu4hX+V9YwlSh9TnQHay04onHRY6RT5gPZaCjfLVVsOg7HHlS3a75WeczGUw47eW8tbK+aoLSM44pGhhg8DiLsb14Bgo/FHuisagju833T8ZnolzT8i0/j+ep8M4ugVzplhfLqOYlptB0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com; spf=pass smtp.mailfrom=linux.alibaba.com; arc=none smtp.client-ip=115.124.30.112 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=linux.alibaba.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=linux.alibaba.com X-Alimail-AntiSpam:AC=PASS;BC=-1|-1;BR=01201311R801e4;CH=green;DM=||false|;DS=||;FP=0|-1|-1|-1|0|-1|-1|-1;HT=ay29a033018046059;MF=yaoma@linux.alibaba.com;NM=1;PH=DS;RN=8;SR=0;TI=SMTPD_---0W.CnxE1_1706011947; Received: from localhost.localdomain(mailfrom:yaoma@linux.alibaba.com fp:SMTPD_---0W.CnxE1_1706011947) by smtp.aliyun-inc.com; Tue, 23 Jan 2024 20:12:29 +0800 From: Bitao Hu To: dianders@chromium.org, akpm@linux-foundation.org, pmladek@suse.com, tglx@linutronix.de, maz@kernel.org, liusong@linux.alibaba.com Cc: linux-kernel@vger.kernel.org, Bitao Hu Subject: [PATCH 0/3] *** Detect interrupt storm in softlockup *** Date: Tue, 23 Jan 2024 20:12:20 +0800 Message-Id: <20240123121223.22318-1-yaoma@linux.alibaba.com> X-Mailer: git-send-email 2.37.1 (Apple Git-137.1) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Hi guys, I have previously encountered an issue where an NVMe interrupt storm caused a softlockup, but the call tree did not provide useful information. This is because the call tree is merely a snapshot and does not fully reflect the CPU's state over the duration of the softlockup_thresh period. Consequently, I think that reporting CPU utilization (system, softirq, hardirq, idle) during a softlockup would be beneficial for identifying issues related to interrupt storms, as well as assisting in the analysis of other causes of softlockup. Furthermore, reporting the most time-consuming hardirqs during a softlockup could directly pinpoint which interrupt is responsible for the issue. Bitao Hu (3): watchdog/softlockup: low-overhead detection of interrupt storm watchdog/softlockup: report the most time-consuming hardirq watchdog/softlockup: add parameter to control the reporting of time-consuming hardirq include/linux/irq.h | 9 ++ include/linux/irqdesc.h | 2 + kernel/irq/irqdesc.c | 9 +- kernel/watchdog.c | 289 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 308 insertions(+), 1 deletion(-) -- 2.37.1 (Apple Git-137.1)