Received: by 2002:a89:d88:0:b0:1fa:5c73:8e2d with SMTP id eb8csp2425775lqb; Mon, 27 May 2024 21:52:17 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCU7YJDlRTIfP3z6omPDIVAyrNGfukb6hB5ik3PEmDf7BTbhhWvhNP/HwCdnkIMjs4mIZpU+SCVTsX5wcAZumH7Rn4GF4dO8bF5HRUSuSg== X-Google-Smtp-Source: AGHT+IFwX93lDWkkO3SY3iOHJOQZEAkaQIwxwHxepGPPZIKnuyz5WpiZXpeW/yIbffIWCmWgkPXz X-Received: by 2002:a05:6300:8089:b0:1b0:2865:fc28 with SMTP id adf61e73a8af0-1b212d1bec1mr12926584637.18.1716871937601; Mon, 27 May 2024 21:52:17 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1716871937; cv=pass; d=google.com; s=arc-20160816; b=Kk0rYdYCdT+WL4pWUvvBr/HPsvZhN6LIXtyR9Lzvt79PnlMzDTbWKjq1499lqG4Gb7 EmhBMnxT3nMV+/5XQW2ltFZ8wBQyzn60TWfzOS3cxZOccBc0VzywnSbVa+hEMjZvrDBH +zrr26p/tmA9a53ShTOElqhS30x9bTwMgRGin8fwC+r9+hfA+5GPp0GUSY7Poxsh+dUh pRr70QNWJBOJ+0Gad/VEyruPe98goj/I8mQsk1VxgjyOrEReyb7xLFIMg+zMUEcxGByF J2rEaHyGvKyLYtWiz4SaCoMyf10hnNOFeW86kVFMZ1bZyrnR0mK1RgUf9b3a4pnrbpaF yGBw== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:list-unsubscribe:list-subscribe :list-id:precedence:dkim-signature; bh=OtlJErHFMtX/7aNJSWoJWAzYMBeqsDWDnwMet809dVw=; fh=LYSYy5XYQN69VuhHXpxdRZLnsXeyPZW9Nd+WpkiJh84=; b=Yy73RJrS7T0cyFglG4H231aXQrnLkQlN2cJ9jN6qFOVK6BYpdS4+FUJoZR+nFp1waJ EFwggfVq/0y9xXJHVgvGIKOWA+mb1Y/UORRM42JEa350bS0n5GwfTFjNYubMe4QkdXvo zz1k91qMw/JmTsQVxguu0pD0tGkIlA0cf2rb/JY2Nkq5trbFp58HgbQnzE5jrGTYcyMQ YeDCmVf9VELw7ElPCqD2q1XCRW3IVAkF+OGNyK6mgShLqaqtF68IQT0HkNbvR6i0sU8x CqIuUO+MxfR3yylERSazzU/ZtQW2PeHaVoV7bPTlB4SFXtgQNO/it9Y6Hc9Fh6SSrcM9 ZDyw==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=TXxpEynC; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com dmarc=pass fromdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-191742-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-191742-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id 98e67ed59e1d1-2bdd9f0ab10si9379501a91.81.2024.05.27.21.52.17 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 27 May 2024 21:52:17 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-191742-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=TXxpEynC; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com dmarc=pass fromdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-191742-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-191742-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 2E238283F6C for ; Tue, 28 May 2024 04:52:17 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id D95DB1BC4B; Tue, 28 May 2024 04:52:10 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="TXxpEynC" Received: from mail-lj1-f181.google.com (mail-lj1-f181.google.com [209.85.208.181]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 754814400; Tue, 28 May 2024 04:52:08 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.181 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716871930; cv=none; b=PSVyonmkkB8S7ohE3IzmzXKBN9hkotzyu5Qk4u1QJyTKKuHVFPPxxcGgPC7Wne2qkFKaPyxfLokMh2ewbmUdRIXxZTgJi43hWycK0TKhs39xaerTlSbp251JKdI3Idcg1xBWPhzeaId3VIBPtZaan40GVthAKjaWLTdlWHTUCH4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1716871930; c=relaxed/simple; bh=g7yMPL+lshsT2TyhdNE17+VKbTpRnVLb1qhQEeVrl8M=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=kAXuQOZ5jLcto+Q8HLSXuvPmgmbJfXJYXFxto+RJRQ3JpPg9ytSK4ogX26USrAPTSnPRVMCsk7fZK4ZShdj9X07aLOaDguJBcOpRPDfQj3KoS8qv2wtn/dxm+N29aG5WsCjnc4wMA9DCMgm0u0j85yLpMww7Rhm1I9gC3DI1dEs= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=TXxpEynC; arc=none smtp.client-ip=209.85.208.181 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-lj1-f181.google.com with SMTP id 38308e7fff4ca-2e95a7545bdso3117781fa.2; Mon, 27 May 2024 21:52:08 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1716871926; x=1717476726; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=OtlJErHFMtX/7aNJSWoJWAzYMBeqsDWDnwMet809dVw=; b=TXxpEynCcJxCxO0EpO9wA4aCnf6kpAl/qCV2yO++SR1lHP66Sc9218u5CdgSEQePcW gVuihn251EjESm4K3RwbpfDMziD4arH25C7DAcSCN6eLcZ404MKWtOBq3RgJx11D82be MDhTrwl2MpZoXSnjZppiMlRLiGW5ruxqiL3rwFf9wjd+QxAmSAF6Q46a2IE0A7M90jig Hqmg4VFSmbxJ3PSCCRVI861lA4zEBHlQviuUVF4iVnImdEeIrpNISONhSolbZgmdq5em fr6sQG0ajrx0y5DLpHUqyHQwwD7fUx266uuWT3cegetBiH3Gen3V4HRGNxUi0Nt5PwEB 29zw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1716871926; x=1717476726; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=OtlJErHFMtX/7aNJSWoJWAzYMBeqsDWDnwMet809dVw=; b=IUF74YWkMRdhHGw7FpzvFBwN3gmLFL1ZVpVY2ozOvX0IF7fcMK/9wsqOA3CzVyevZ7 BQ2F+PFG5eTrQJn6adef6Ks5drehS1ZeVwTJa/jLpsuDBDtUAWCgUcHRBon1cSsU3S3q Nhp9ka0mj9DmOnuOt5pzyyI9Q0eD45Z4UluZwlVQW0uyadtELZCz7s3ayk2C3kGZwr9V BS+AgImq1Y6vNE5Q1uM7IbdA7HDODGbQcWHeBdOnmxBqxC1Y2MS2UyWWKb6HCJuGTmE9 HZcqFwushaSeXm9L5DpflIsT7t1TkwOj6GkX+xhKxtrLMSzqCfEbkqcSodlloTdSuy2k b0MQ== X-Forwarded-Encrypted: i=1; AJvYcCWfosq1meKcMf9l05k7tUPBQJvFRl055mlho5z93M9Nv01uxYoCUBuWkjnYVuuTgBIaJWPmyJn8+8FNRMx5inpm4o7jkVZwC1ONl6bRLTdGcQbssPaTBV4NvfhdWjF2g7YN0BAgamThBcWbQS2jEEBy0yr6MFqb0awa98qTiz2owqRgnqJ0k2Iq X-Gm-Message-State: AOJu0YzdJYzRWvkpo/z6ESxCee6eZYQxhrZFdbmqGxBCEhU/T35hoB3r 4cLN9x3+7N0zDZ8M6Hq/JelvspFb12fUl+1q8bw1r+UyHzY6uXeh+RIyFp4JOG1l/0rEYGQrAW6 QrezQH+Daajeq6/CkYM4BMxX/5Ig= X-Received: by 2002:a2e:a78d:0:b0:2e9:87bb:1ce8 with SMTP id 38308e7fff4ca-2e987bb1d9amr3617351fa.35.1716871926371; Mon, 27 May 2024 21:52:06 -0700 (PDT) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <5b79732b-087c-411f-a477-9b837566673e@leemhuis.info> <20240527183139.42b6123c@rorschach.local.home> In-Reply-To: <20240527183139.42b6123c@rorschach.local.home> From: =?UTF-8?B?SWxra2EgTmF1bGFww6TDpA==?= Date: Tue, 28 May 2024 07:51:30 +0300 Message-ID: Subject: Re: Bug in Kernel 6.8.x, 6.9.x Causing Trace/Panic During Shutdown/Reboot To: Steven Rostedt Cc: "Linux regression tracking (Thorsten Leemhuis)" , Linux regressions mailing list , stable@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable yeah, the cache_from_obj tracing bug (without panic) has been displayed quite some time now - maybe even since 6.7.x or so. I could try checking a few versions back for this and try bisecting it if I can find when this started. --Ilkka On Tue, May 28, 2024 at 1:31=E2=80=AFAM Steven Rostedt wrote: > > On Fri, 24 May 2024 12:50:08 +0200 > "Linux regression tracking (Thorsten Leemhuis)" wrote: > > > > - Affected Versions: Before kernel version 6.8.10, the bug caused a > > > quick display of a kernel trace dump before the shutdown/reboot > > > completed. Starting from version 6.8.10 and continuing into version > > > 6.9.0 and 6.9.1, this issue has escalated to a kernel panic, > > > preventing the shutdown or reboot from completing and leaving the > > > machine stuck. > > You state "Before kernel version 6.8.10, the bug caused ...". Does that > mean that a bug was happening before v6.8.10? But did not cause a panic? > > I just noticed your second screen shot from your report, and it has: > > "cache_from_obj: Wrong slab cache, tracefs_inode_cache but object is fro= m inode_cache" > > So somehow an tracefs_inode was allocated from the inode_cache and is > now being freed by the tracefs_inode logic? Did this happen before > 6.8.10? If so, this code could just be triggering an issue from an > unrelated bug. > > Thanks, > > -- Steve