Received: by 2002:ab2:6f44:0:b0:1fd:c486:4f03 with SMTP id l4csp220547lqq; Thu, 13 Jun 2024 00:34:29 -0700 (PDT) X-Forwarded-Encrypted: i=3; AJvYcCWhbN30FIz4GJDH74g+94Mj5cEc9sQYmckve0ZwceU769mBRdLX9xNHGxM3IVIuYoEKsD0wsZjKmlvpkdUKfI8gn7cS8lMF+NPEx48Liw== X-Google-Smtp-Source: AGHT+IE633mBxO58D1BAKd4PgUni8iIstvaAmQWg28ur4Ylo79umpLzSiYV/9qD69+EV+49yutg9 X-Received: by 2002:a17:907:7e96:b0:a69:d82c:167f with SMTP id a640c23a62f3a-a6f47c7d729mr304405366b.15.1718264068888; Thu, 13 Jun 2024 00:34:28 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1718264068; cv=pass; d=google.com; s=arc-20160816; b=Tlr9/1PJ1EAvFffYtOk8AoHmQuMqXBTaVhJcKIgQGdvcq6CnWh/yLOfYvJmO/R4wbs kDtoGRvMjT72BC2UzresTYgmky8QSY2D5yfcSuxtYgVDhaWEAAOZKHuOeQZgny9II2bn 9TmpO7JwIXMS8OIwyHKKE2YerbtE0aAjdv2L7qfA/zKYAtiP/SGoGHu00i1II6mWNw35 UsxwFXx2KMMungTYcSMt8Vv1RpW5W/Psx7f0rSoOs5sf55b6Lmtvbt/AuaoZ3DjY5gI0 M8zbT5TsAJJBHBzSukWkG0DV797Aw/I0Btv2mEqROJkc4z4BgmknAHD744zCWFxHoPlH u6sA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:list-unsubscribe:list-subscribe :list-id:precedence:dkim-signature; bh=0C8KlVl3OW2PC2dRMHxIiA4eTV9+V+jwqso0xqeA27M=; fh=I2DqEqvfWyI1n2epiFUy5Mk1qQU8YeDcCkQZCQ34YtI=; b=jOlJ1Itj6M8gGZt7SCHcQLiPEH9MGhRWmoja+Vb5QA5JNyaUkmgA18qtJcWXCWS5US ZTCWgHXSHSBRpRu5eJDmGgWiu2W2i9Gklz9FmfMltrbdf9xISwCuiEmqSNE6HuAFSZMH TeIrErqj1p7fVRgmL0P4yhywvQPJXDjRzsNYWpDCjWIylQJLGE9dL735GK/NbLlehuqe 9ZWwtecMnM29mCVXb0wJe1KnNhD5VIwzR98WgN358gvJP7B1NwncTilOry5089T/8wje rAdLpLVF4G3H7oIaFzQXzFW+LEgNvDIEmzKI0xwyFAyc1xt3P6gQXDqDld8PZANQq1Fq D0Zg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=FxoiJ78c; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com dmarc=pass fromdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-212715-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-212715-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from am.mirrors.kernel.org (am.mirrors.kernel.org. [147.75.80.249]) by mx.google.com with ESMTPS id a640c23a62f3a-a6f56df761csi39424966b.600.2024.06.13.00.34.28 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 13 Jun 2024 00:34:28 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel+bounces-212715-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20230601 header.b=FxoiJ78c; arc=pass (i=1 spf=pass spfdomain=gmail.com dkim=pass dkdomain=gmail.com dmarc=pass fromdomain=gmail.com); spf=pass (google.com: domain of linux-kernel+bounces-212715-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) smtp.mailfrom="linux-kernel+bounces-212715-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by am.mirrors.kernel.org (Postfix) with ESMTPS id 6F9A61F21FF6 for ; Thu, 13 Jun 2024 07:34:28 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id ADACC13C8F3; Thu, 13 Jun 2024 07:33:05 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b="FxoiJ78c" Received: from mail-lj1-f178.google.com (mail-lj1-f178.google.com [209.85.208.178]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id B35C313D26B; Thu, 13 Jun 2024 07:33:02 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.208.178 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263984; cv=none; b=evd5/AEo9Bx9Wrfqyopn1bT0vI1CnlTIBjPFpR3BPMKFa64dTWjzm9GvBSdcXpHVRZHAJjjmz58Bvjy3iPbE/Xof+xldeacv3m3KCRrV+vfafdqy+rt+L3OvdVLOBETsxTGZ6UchFA2SuKhMzMhMArGEGiu1t53HEjlcNlCWtu4= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1718263984; c=relaxed/simple; bh=d0mZNWDpRk6kCJv92A4sILZo41s0K1Wp0msT2BRz2HA=; h=MIME-Version:References:In-Reply-To:From:Date:Message-ID:Subject: To:Cc:Content-Type; b=I58bCfCjT+1AKDlJPXYweCQOpC9gKLVtrK2TxWhkH1pRCLBkLnvZrGh0kFXuOhpO4TxYspdjuZVoLfhH1BoNxRj8AVYpOoWsyOrokEBK3DUeBE5wUYtx2dEnSAYwrEEjA8qQQVQ2+nN0S5xdizJDAlVwdVTeO/5CYqQg1yMFHz0= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com; spf=pass smtp.mailfrom=gmail.com; dkim=pass (2048-bit key) header.d=gmail.com header.i=@gmail.com header.b=FxoiJ78c; arc=none smtp.client-ip=209.85.208.178 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=gmail.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=gmail.com Received: by mail-lj1-f178.google.com with SMTP id 38308e7fff4ca-2ebec2f11b7so5904341fa.2; Thu, 13 Jun 2024 00:33:02 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1718263981; x=1718868781; darn=vger.kernel.org; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:from:to:cc:subject:date :message-id:reply-to; bh=0C8KlVl3OW2PC2dRMHxIiA4eTV9+V+jwqso0xqeA27M=; b=FxoiJ78ccKBM32cI59ZqY0lEatSjSimSpA+ESw6cUf5FAQ7PrdwZ5CDzNvSZc/0TTE sl5IQ6Yv+1VL4NY1kXpVeS+wuZ6u5lMlgtp7jQbq/bPa/1Y62pqFgh96GT03I012OKOu HsuDH71fzE54G+0xNSHjNIkBi5KBRnavHADZAlzOZQInlQREUvAQfx5m41Vth2qROF49 sSJktkIPfLDz3n8ecXgBhumePYi+Z9K50S1LT0qinO0Dg98WatfE1s6CBNbLyS6KGugL Wv663QHDF5aiFAi+WisYucRhyGpPmjdZnAmttXwmuWlFy2Tce0CChTW26S8OGX/qFLYg GHYw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1718263981; x=1718868781; h=content-transfer-encoding:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=0C8KlVl3OW2PC2dRMHxIiA4eTV9+V+jwqso0xqeA27M=; b=OU8I3vjy6Xhhg+dd8qsekbwr5Mpt9g9kMumiAR51ZZeaUdFiN05t9NhzrhnzFsxcoc AgivUStZ4mu1lLSL8Dg2exUs9DgEpkP+TnmKFCQkAh2KSXR2R/pSj6C7iYCofHyW4vo9 xpRN8BTh7yLsYmgfpSHvXlkbpp91v44CrY6EtPPNGLxCGK/jNgFLJkrxuaYF3f2T4Fgs 9uPH13UHgy7BDMCD3QpQtM/4kis/7iB1bYF7k8QcNzeCwvXQUf0SjoJerfMZHtD+ShCv vmHtPT8iYoOLC1TYG1CPXtAgdXT1I+8IfAIECp1d6eLcfyPbjtFUm25xH+dmp4u0L1iA WNSQ== X-Forwarded-Encrypted: i=1; AJvYcCXyGbiFldt7NLOhWtVSDRrYv3/6FuVO1eofpRoxIA/Klm3IVyaw2TZBPkKGfZ5qpaDttzEBDttWIjBg1OZICOeDMpRyAWspKR2MujfvPTE9WuMZDzGfWmVHaWReTBUoGheHHX35QN3PDfeL/tu2OuKP7BhXjlQlqg6bwUatB+N0KMxiJowwXYM4 X-Gm-Message-State: AOJu0YyDL0Dri34QQ74PrIbcfidcik0pltUSRxJyr/ySpHI85epsotEQ 100Am1djz60HHRkCrwMDqsxe9efmTU/YYiLZC05mb3EhfvEntwTGDdF9J7b07xSTRWUF5EuPa9+ RVugJRfvMnxbPX9xqR09tjvN9Q/fx/TvA X-Received: by 2002:a2e:878f:0:b0:2eb:ecba:444a with SMTP id 38308e7fff4ca-2ebfc9fac80mr21389571fa.23.1718263980494; Thu, 13 Jun 2024 00:33:00 -0700 (PDT) Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 References: <5b79732b-087c-411f-a477-9b837566673e@leemhuis.info> <20240527183139.42b6123c@rorschach.local.home> <20240528144743.149e351b@rorschach.local.home> <20240529144757.79d09eeb@rorschach.local.home> <20240529154824.2db8133a@rorschach.local.home> <20240530095953.0020dff9@rorschach.local.home> <20240612115612.2e5f4b34@rorschach.local.home> In-Reply-To: <20240612115612.2e5f4b34@rorschach.local.home> From: =?UTF-8?B?SWxra2EgTmF1bGFww6TDpA==?= Date: Thu, 13 Jun 2024 10:32:24 +0300 Message-ID: Subject: Re: Bug in Kernel 6.8.x, 6.9.x Causing Trace/Panic During Shutdown/Reboot To: Steven Rostedt Cc: "Linux regression tracking (Thorsten Leemhuis)" , Linux regressions mailing list , stable@vger.kernel.org, linux-kernel@vger.kernel.org, linux-trace-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable ok, so if you don't have any idea where this bug is after those debug patches, I'll try to find some time to bisect it as a last resort. Stay tuned. --Ilkka On Wed, Jun 12, 2024 at 6:56=E2=80=AFPM Steven Rostedt wrote: > > On Wed, 12 Jun 2024 15:36:22 +0200 > "Linux regression tracking (Thorsten Leemhuis)" wrote: > > > Hi, Thorsten here, the Linux kernel's regression tracker. Top-posting > > for once, to make this easily accessible to everyone. > > > > Ilkka or Steven, what happened to this? This thread looks stalled. I > > also was unsuccessful when looking for other threads related to this > > report or the culprit. Did it fall through the cracks or am I missing > > something here? > > Honesty, I have no idea where the bug is. I can't reproduce it. These > patches I sent would check all the places that add to the list to make > sure the proper trace_inode was being added, and the output shows that > they are all correct. Then suddenly, something that came from the > inode cache is calling the tracefs inode cache to free it, and that's > where the bug is happening. > > This really looks like another bug that the recent changes have made > more predominate. > > -- Steve > > > > > > Ciao, Thorsten (wearing his 'the Linux kernel's regression tracker' hat= ) > > -- > > Everything you wanna know about Linux kernel regression tracking: > > https://linux-regtracking.leemhuis.info/about/#tldr > > If I did something stupid, please tell me, as explained on that page. > > > > #regzbot poke > > > > On 02.06.24 09:32, Ilkka Naulap=C3=A4=C3=A4 wrote: > > > sorry longer delay, been a bit busy but here is the result from that > > > new patch. Only applied this patch so if the previous one is needed > > > also, let me know and I'll rerun it. > > > > > > --Ilkka > > > > > > On Thu, May 30, 2024 at 5:00=E2=80=AFPM Steven Rostedt wrote: > > >> > > >> On Thu, 30 May 2024 16:02:37 +0300 > > >> Ilkka Naulap=C3=A4=C3=A4 wrote: > > >> > > >>> applied your patch and here's the output. > > >>> > > >> > > >> Unfortunately, it doesn't give me any new information. I added one m= ore > > >> BUG on, want to try this? Otherwise, I'm pretty much at a lost. :-/ > > >> > > >> -- Steve > > >> > > >> diff --git a/fs/tracefs/inode.c b/fs/tracefs/inode.c > > >> index de5b72216b1a..a090495e78c9 100644 > > >> --- a/fs/tracefs/inode.c > > >> +++ b/fs/tracefs/inode.c > > >> @@ -39,13 +39,17 @@ static struct inode *tracefs_alloc_inode(struct = super_block *sb) > > >> return NULL; > > >> > > >> ti->flags =3D 0; > > >> + ti->magic =3D 20240823; > > >> > > >> return &ti->vfs_inode; > > >> } > > >> > > >> static void tracefs_free_inode(struct inode *inode) > > >> { > > >> - kmem_cache_free(tracefs_inode_cachep, get_tracefs(inode)); > > >> + struct tracefs_inode *ti =3D get_tracefs(inode); > > >> + > > >> + BUG_ON(ti->magic !=3D 20240823); > > >> + kmem_cache_free(tracefs_inode_cachep, ti); > > >> } > > >> > > >> static ssize_t default_read_file(struct file *file, char __user *bu= f, > > >> @@ -147,16 +151,6 @@ static const struct inode_operations tracefs_di= r_inode_operations =3D { > > >> .rmdir =3D tracefs_syscall_rmdir, > > >> }; > > >> > > >> -struct inode *tracefs_get_inode(struct super_block *sb) > > >> -{ > > >> - struct inode *inode =3D new_inode(sb); > > >> - if (inode) { > > >> - inode->i_ino =3D get_next_ino(); > > >> - inode->i_atime =3D inode->i_mtime =3D inode_set_ctim= e_current(inode); > > >> - } > > >> - return inode; > > >> -} > > >> - > > >> struct tracefs_mount_opts { > > >> kuid_t uid; > > >> kgid_t gid; > > >> @@ -384,6 +378,7 @@ static void tracefs_dentry_iput(struct dentry *d= entry, struct inode *inode) > > >> return; > > >> > > >> ti =3D get_tracefs(inode); > > >> + BUG_ON(ti->magic !=3D 20240823); > > >> if (ti && ti->flags & TRACEFS_EVENT_INODE) > > >> eventfs_set_ef_status_free(dentry); > > >> iput(inode); > > >> @@ -568,6 +563,18 @@ struct dentry *eventfs_end_creating(struct dent= ry *dentry) > > >> return dentry; > > >> } > > >> > > >> +struct inode *tracefs_get_inode(struct super_block *sb) > > >> +{ > > >> + struct inode *inode =3D new_inode(sb); > > >> + > > >> + BUG_ON(sb->s_op !=3D &tracefs_super_operations); > > >> + if (inode) { > > >> + inode->i_ino =3D get_next_ino(); > > >> + inode->i_atime =3D inode->i_mtime =3D inode_set_ctim= e_current(inode); > > >> + } > > >> + return inode; > > >> +} > > >> + > > >> /** > > >> * tracefs_create_file - create a file in the tracefs filesystem > > >> * @name: a pointer to a string containing the name of the file to = create. > > >> diff --git a/fs/tracefs/internal.h b/fs/tracefs/internal.h > > >> index 69c2b1d87c46..9059b8b11bb6 100644 > > >> --- a/fs/tracefs/internal.h > > >> +++ b/fs/tracefs/internal.h > > >> @@ -9,12 +9,15 @@ enum { > > >> struct tracefs_inode { > > >> unsigned long flags; > > >> void *private; > > >> + unsigned long magic; > > >> struct inode vfs_inode; > > >> }; > > >> > > >> static inline struct tracefs_inode *get_tracefs(const struct inode = *inode) > > >> { > > >> - return container_of(inode, struct tracefs_inode, vfs_inode); > > >> + struct tracefs_inode *ti =3D container_of(inode, struct trac= efs_inode, vfs_inode); > > >> + BUG_ON(ti->magic !=3D 20240823); > > >> + return ti; > > >> } > > >> > > >> struct dentry *tracefs_start_creating(const char *name, struct dent= ry *parent); >