Received: by 2002:a05:7412:bbc7:b0:fc:a2b0:25d7 with SMTP id kh7csp828701rdb; Fri, 2 Feb 2024 05:29:38 -0800 (PST) X-Google-Smtp-Source: AGHT+IGDJ+hZJPZMlDSRN4oHhYrmLgsu9MZj5+FOG9FCmXartiQ2mjNKF7cjv4TnQptozjYMwUXF X-Received: by 2002:a17:902:e5c5:b0:1d9:37a2:5ef4 with SMTP id u5-20020a170902e5c500b001d937a25ef4mr3068288plf.23.1706880577931; Fri, 02 Feb 2024 05:29:37 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1706880577; cv=pass; d=google.com; s=arc-20160816; b=jhfWnoUnYMp8a5RI6R2sUnDH07CpKRBF+TjR//Mk8MArKZNkmxFnqcChXXZRvVJ9um RXQQdE4gedRrRkj7+BkLB8r/fk9fg1WA6DlUMpkgWMhzFD6BYw2rNTq1ouoZrA819a73 kO1zerFI8Ics03WKWsShyjIzyW1u8SPS3mAYhgv2JNsJ6nQV+tdSeIJPiO+y4OYUNbSs xjKcWsyDw1kqvJ4wOzNmTB1svTT97IJs/hp8VXCQcnLXBf18n7fUeQpdDWi2h9B1wtY7 HK/K9m/Z7V96Cs1wPyxMe1aJpmaFVp69kBCUK5jPyDT+2KoiiGrAftph+eanKP1npoUA vAMQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:message-id:date:subject:cc:to :from:dkim-signature; bh=Qi46XHgQ5S4G8uwPSvfwaJYskD1YLq6Uj9BiflDY3Fw=; fh=yLYkKR1RUy8keIVSmoFMMUQvalXEoJ40No+12w/l8RE=; b=zAOmU6JDYI6xbYqt70tChTyFEcNJO6tZCNfHv8uB02QTOjfG2zajEghUvT+FWDhJfa fSMwEUOfa0ZWwrof7U6FlZRalTIrmR0wh2un5/V+Zl1PRCfk7sFOBmdQAo9X8dRX+Fkh bgWuPlVgfgBd3X/NBDSdwX0UMQ9WXjJpVc6XL93fHEKFK3STar2q8xKawi5JCt2oPvBI FsDSNSZjrwIUVVEH1X8CFaPz0tDZly62T7fjiVPf977DxwsVnNp3wMXIJnU68N03YJWy iJZfF8AWyDvglrqoEmhQ/qQzr8QTATTWSNSM9Ef4Tu50z7BQaXJm/CunsRsH6VmJwan1 x0+w==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YM6ccpn0; arc=pass (i=1 spf=pass spfdomain=intel.com dkim=pass dkdomain=intel.com dmarc=pass fromdomain=intel.com); spf=pass (google.com: domain of linux-kernel+bounces-49898-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-49898-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com X-Forwarded-Encrypted: i=1; AJvYcCV1TfrS/3iPECccUEoe8FqJbrcitfaAJwPp2dsdF5ZTomr/JyvdhTYTh38Wommd/QGjhc0/85vBkoyH89Lxd+lQ2oUje4enwEaH0lCY6Q== Return-Path: Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [139.178.88.99]) by mx.google.com with ESMTPS id p6-20020a170902eac600b001d92eb54a57si1667288pld.158.2024.02.02.05.29.37 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 02 Feb 2024 05:29:37 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-49898-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=YM6ccpn0; arc=pass (i=1 spf=pass spfdomain=intel.com dkim=pass dkdomain=intel.com dmarc=pass fromdomain=intel.com); spf=pass (google.com: domain of linux-kernel+bounces-49898-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) smtp.mailfrom="linux-kernel+bounces-49898-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 17F1A286504 for ; Fri, 2 Feb 2024 13:29:19 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 5A1024653A; Fri, 2 Feb 2024 13:29:14 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b="YM6ccpn0" Received: from mgamail.intel.com (mgamail.intel.com [192.198.163.8]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id C7FB046430 for ; Fri, 2 Feb 2024 13:29:11 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=192.198.163.8 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706880553; cv=none; b=KemsSDFaq62xsNZiRvSgYZO10chBnwPo0Z9rtVkViFHx043hCY4SeO1jQ4rZfHHHOy2exgOqRNevHg2ewUT9OZxTWX1mPoVle+sOQjFCrEVC596yDuTX+xX1+cxhPNWHxP/c3w7alnjrIuXkL+GWW6CULGewpGAU3Y0xa/sH60g= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1706880553; c=relaxed/simple; bh=EOuVn7h5rBzXLjSX6vgvwOxg8VenfLXWcdGIB/LkqTA=; h=From:To:Cc:Subject:Date:Message-Id:MIME-Version; b=iR1GfXi6tOWpYCnoLiAxu705CPv6PjPfedkKMOAsnsnFkntXGABd1TuICF6OK/ogEF4eQCAdR0Fs2tp6jdcQHKSzdz3rKxH5JFUr/eSQ9VpQ/LtD6YkMAJDpB28wux2t8XMMarq6nMV6ycMnYn/XLoVmck7imYLE1qz0BVZdqd4= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com; spf=pass smtp.mailfrom=intel.com; dkim=pass (2048-bit key) header.d=intel.com header.i=@intel.com header.b=YM6ccpn0; arc=none smtp.client-ip=192.198.163.8 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=intel.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=intel.com DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1706880552; x=1738416552; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=EOuVn7h5rBzXLjSX6vgvwOxg8VenfLXWcdGIB/LkqTA=; b=YM6ccpn05oQ4zXVo6+8kYvOCz8E9QtsIEPdsqwr+6LDTFVFpQXpqJF/F 3zHVtGNnZMjVPWh4zdFkVzH9WNL7Zlnq2+dBe9HuXs4kx4ijgHbAiDWOB tTYz3brnARTgbPIy32PK7N46LxCF7kC7JnWXQENWbCLK8v4sp/tefuXJP zEg/qWn7Sab7caS57b8v5MDc6LWcppZVD9iptrwcCuB2PBrOuNSVq0p8G l/cuYQ6c06r8IyXGARx4xxG/HOIS9C2jCyz+ctDRgvQwoGpml7x3fwdW3 RerbP3szGiLHKyY2DzB7EMkC/2KXQiBYqDOgCkHS8v40Ks+nSgu7B1YlH A==; X-IronPort-AV: E=McAfee;i="6600,9927,10971"; a="17685033" X-IronPort-AV: E=Sophos;i="6.05,238,1701158400"; d="scan'208";a="17685033" Received: from fmviesa001.fm.intel.com ([10.60.135.141]) by fmvoesa102.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 02 Feb 2024 05:29:11 -0800 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="6.05,238,1701158400"; d="scan'208";a="31174623" Received: from feng-clx.sh.intel.com ([10.239.159.50]) by fmviesa001.fm.intel.com with ESMTP; 02 Feb 2024 05:29:09 -0800 From: Feng Tang To: Andrew Morton , Jonathan Corbet , linux-kernel@vger.kernel.org Cc: Randy Dunlap , Josh Poimboeuf , Peter Zijlstra , gpiccoli@igalia.com, Feng Tang Subject: [PATCH] panic: add option to dump blocked tasks in panic_print Date: Fri, 2 Feb 2024 21:20:42 +0800 Message-Id: <20240202132042.3609657-1-feng.tang@intel.com> X-Mailer: git-send-email 2.34.1 Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit For debugging kernel panic and other bugs, there is already option of panic_print to dump all tasks' call stacks. On today's large servers running many containers, there could be thousands of tasks or more, and it will print out huge amount of call stacks, and take a lot of time (for serial console which is main target user case of panic_print). And in many cases, only those several tasks being blocked is key for the panic, so add an option to only dump blocked tasks' call stack. Signed-off-by: Feng Tang --- Documentation/admin-guide/kernel-parameters.txt | 1 + Documentation/admin-guide/sysctl/kernel.rst | 1 + kernel/panic.c | 4 ++++ 3 files changed, 6 insertions(+) diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt index 31b3a25680d0..0f2369e87175 100644 --- a/Documentation/admin-guide/kernel-parameters.txt +++ b/Documentation/admin-guide/kernel-parameters.txt @@ -4182,6 +4182,7 @@ bit 4: print ftrace buffer bit 5: print all printk messages in buffer bit 6: print all CPUs backtrace (if available in the arch) + bit 7: print tasks in uninterruptible (blocked) state *Be aware* that this option may print a _lot_ of lines, so there are risks of losing older messages in the log. Use this option carefully, maybe worth to setup a diff --git a/Documentation/admin-guide/sysctl/kernel.rst b/Documentation/admin-guide/sysctl/kernel.rst index 6584a1f9bfe3..e066a16b35d5 100644 --- a/Documentation/admin-guide/sysctl/kernel.rst +++ b/Documentation/admin-guide/sysctl/kernel.rst @@ -850,6 +850,7 @@ bit 3 print locks info if ``CONFIG_LOCKDEP`` is on bit 4 print ftrace buffer bit 5 print all printk messages in buffer bit 6 print all CPUs backtrace (if available in the arch) +bit 7 print tasks in uninterruptible (blocked) state ===== ============================================ So for example to print tasks and memory info on panic, user can:: diff --git a/kernel/panic.c b/kernel/panic.c index 2807639aab51..aa17ae0897c0 100644 --- a/kernel/panic.c +++ b/kernel/panic.c @@ -73,6 +73,7 @@ EXPORT_SYMBOL_GPL(panic_timeout); #define PANIC_PRINT_FTRACE_INFO 0x00000010 #define PANIC_PRINT_ALL_PRINTK_MSG 0x00000020 #define PANIC_PRINT_ALL_CPU_BT 0x00000040 +#define PANIC_PRINT_BLOCKED_TASKS 0x00000080 unsigned long panic_print; ATOMIC_NOTIFIER_HEAD(panic_notifier_list); @@ -227,6 +228,9 @@ static void panic_print_sys_info(bool console_flush) if (panic_print & PANIC_PRINT_FTRACE_INFO) ftrace_dump(DUMP_ALL); + + if (panic_print & PANIC_PRINT_BLOCKED_TASKS) + show_state_filter(TASK_UNINTERRUPTIBLE); } void check_panic_on_warn(const char *origin) -- 2.34.1