Received: by 2002:a25:1985:0:0:0:0:0 with SMTP id 127csp1173911ybz; Fri, 17 Apr 2020 17:48:18 -0700 (PDT) X-Google-Smtp-Source: APiQypLD3mf+e06hjI3Z78PwOnhhySvEc8CkG4jMWwf45CPfCAOz8qcjn2RJkApnH8cH/SxtESCE X-Received: by 2002:a17:906:3291:: with SMTP id 17mr5538193ejw.343.1587170897966; Fri, 17 Apr 2020 17:48:17 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1587170897; cv=none; d=google.com; s=arc-20160816; b=OZGhD30dyl1v5di7l1WqLK0/0slK64ZK9bkhE/oMDK3gET+PS2IUnUxNXPNUe+aT+s IDXha1J193QPqeFG1W2pap0hEIJ8enDZYlqAQPNn9sswHtoALn1gXQLNcIpBDz84sjJR R7bvM9yswqzX/WeQt9RexSbCNkDdVO59mk0pVs81Ab04GlL/yKipdAWIBe3/RDQgfsBE RxSshCt8AU5TsA61SEItNjCKIP2k0dKGXKmB+HNtcKqJtXWfIto4gqJqWUez61egFFv/ 914w6g1oTdo0JDV/OETiCeR/iux3arcLIeNEBTbR2s8Sm5DvOtsUbMik24QfHBYWz8Wq hFpg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:subject:cc:to:from:date :dkim-signature; bh=q2G+9K9aLpEcKFuTeqmXkQkGW9exu+YZz7eg/BAqEfc=; b=k5IERZmWt0KPqwjTrriQuPmGusZsUjUoASWpFIgcWnchuqgTnrWAGNAS7BymmdToeZ id3CyAYbXPml3pDIXZ1GvE66W8p6VWiA+2J6CqrbTfss8kunH/oDaIQTC65hfQQI4cuD ji9KS5TG5UgLbogbO+WwTeQh3ejf1GSvGAkqcDlxwDlHgm+1NUVHJqY51026BV7zmgTT LtRb2X+8YKgoh7cyuKkcmwRYxpj6IPGc/k9gAUKc8U2FlUZtch3Vn8UzqSqmGygZQclt TMAwOmuPt12GT/9/BdQnjr+PgPzI33Au4qaSKoJ5O9CWU6I/Ro7ZlcgwgdjhLdry1nY2 U+7g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=yxkA8fcX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id i14si10704618ejh.494.2020.04.17.17.47.55; Fri, 17 Apr 2020 17:48:17 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=yxkA8fcX; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1725958AbgDRAq4 (ORCPT + 99 others); Fri, 17 Apr 2020 20:46:56 -0400 Received: from mail.kernel.org ([198.145.29.99]:53136 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725867AbgDRAq4 (ORCPT ); Fri, 17 Apr 2020 20:46:56 -0400 Received: from localhost.localdomain (c-73-231-172-41.hsd1.ca.comcast.net [73.231.172.41]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 2E767214D8; Sat, 18 Apr 2020 00:46:55 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1587170815; bh=14qSYdrLemwVEC2uAz97e2xRaw+5yWSJoO7bWCN/K+M=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=yxkA8fcXsZ3kHBWGVChkWfhLXCziPW7a3jxL/C2RqnxvOCRPknlE3ur4ow8QKyd1C KRYzLVsbvXBu5n4P5eslol7AvQWnzw0Lxwn2EEBNGYbRtXL0EHdnqPh9H05MLVrh4w FNWiKKtyNJWl/oZBQwCLwS+px7QRgKEm1hk2sSi8= Date: Fri, 17 Apr 2020 17:46:54 -0700 From: Andrew Morton To: "Guilherme G. Piccoli" Cc: linux-kernel@vger.kernel.org, linux-api@vger.kernel.org, linux-doc@vger.kernel.org, mcgrof@kernel.org, keescook@chromium.org, yzaikin@google.com, tglx@linutronix.de, vbabka@suse.cz, rdunlap@infradead.org, willy@infradead.org, kernel@gpiccoli.net Subject: Re: [PATCH V3] panic: Add sysctl to dump all CPUs backtraces on oops event Message-Id: <20200417174654.9af0c51afb5d9e35e5519113@linux-foundation.org> In-Reply-To: <20200327224116.21030-1-gpiccoli@canonical.com> References: <20200327224116.21030-1-gpiccoli@canonical.com> X-Mailer: Sylpheed 3.5.1 (GTK+ 2.24.31; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 27 Mar 2020 19:41:16 -0300 "Guilherme G. Piccoli" wrote: > Usually when kernel reach an oops condition, it's a point of no return; > in case not enough debug information is available in the kernel splat, > one of the last resorts would be to collect a kernel crash dump and > analyze it. The problem with this approach is that in order to collect > the dump, a panic is required (to kexec-load the crash kernel). When > in an environment of multiple virtual machines, users may prefer to > try living with the oops, at least until being able to properly > shutdown their VMs / finish their important tasks. > > This patch implements a way to collect a bit more debug details when an > oops event is reached, by printing all the CPUs backtraces through the > usage of NMIs (on architectures that support that). The sysctl added > (and documented) here was called "oops_all_cpu_backtrace", and when > set will (as the name suggests) dump all CPUs backtraces. > > Far from ideal, this may be the last option though for users that for > some reason cannot panic on oops. Most of times oopses are clear enough > to indicate the kernel portion that must be investigated, but in virtual > environments it's possible to observe hypervisor/KVM issues that could > lead to oopses shown in other guests CPUs (like virtual APIC crashes). > This patch hence aims to help debug such complex issues without > resorting to kdump. > > ... > > --- a/include/linux/kernel.h > +++ b/include/linux/kernel.h > @@ -513,6 +513,12 @@ static inline u32 int_sqrt64(u64 x) > } > #endif > > +#ifdef CONFIG_SMP > +extern unsigned int sysctl_oops_all_cpu_backtrace; > +#else > +#define sysctl_oops_all_cpu_backtrace 0 > +#endif /* CONFIG_SMP */ > + hm, we have a ton of junk in kernel.h just to communicate between sysctl.c and a handful of other files. Perhaps one day someone can move all that into a new sysctl-externs.h.