Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Tue, 17 Jan 2023 15:19:58 +0800
From:   Oliver Sang <oliver.sang@intel.com>
To:     Vlastimil Babka <vbabka@suse.cz>
CC:     Hyeonggon Yoo <42.hyeyoo@gmail.com>, <oe-lkp@lists.linux.dev>,
        <lkp@intel.com>, Mike Rapoport <rppt@linux.ibm.com>,
        Christoph Lameter <cl@linux.com>,
        <linux-kernel@vger.kernel.org>, <linux-mm@kvack.org>,
        Matthew Wilcox <willy@infradead.org>
Subject: Re: [linus:master] [mm, slub] 0af8489b02:
 kernel_BUG_at_include/linux/mm.h
Message-ID: <Y8ZMHi4IGP+v7biC@xsang-OptiPlex-9020>
References: <41276905-b8a5-76ae-8a17-a8ec6558e988@suse.cz>
 <Y7Qxucg5le7WOzr7@xsang-OptiPlex-9020>
 <Y7VBFLHY/PMbb4XS@hyeyoo>
 <Y7Yr3kEkDEd51xns@xsang-OptiPlex-9020>
 <3f7fa3b3-9623-5c4c-94b1-a41dea6eaaf2@suse.cz>
 <Y7weKyxIY+QFYq6j@xsang-OptiPlex-9020>
 <Y71t89JMgtRoNHM/@xsang-OptiPlex-9020>
 <d024b846-9282-4ecd-eafd-2e5571932a72@suse.cz>
 <Y7+7KUyuo/9HeCF2@xsang-OptiPlex-9020>
 <bbf4fc3e-a9c2-83aa-d50a-8f7cb5b2bf4b@suse.cz>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <bbf4fc3e-a9c2-83aa-d50a-8f7cb5b2bf4b@suse.cz>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?XVh4IjefmTmIw/WgtPIrQg5IWPccwD3udmn5X/nVfB3/D4s2n7UAJkc8rM+8?=
 =?us-ascii?Q?Q8zPDxs+euNBODOF2CS1RHYTEk2feP4Y04np7LGKJ/7WN9PteWB3kFHz+hsT?=
 =?us-ascii?Q?4pHTsVb4IJtL/Bl4hTYSHY6zYYcqjdvlg5//SCPLZX5GyYsC4URhC4kWruio?=
 =?us-ascii?Q?isUXxozwF84nTjIGt6vWk4plfVaU41eyY0evwBkk79VMatNAbOLng6qSWpu2?=
 =?us-ascii?Q?ACB7qz+48aw5ijNMJC4lmfHH7D8miy1fLKP02vRC7FiJC3j+N9kKaua0RJ7F?=
 =?us-ascii?Q?gLPXm0TviWzjfE/jU3XaO7Zb0nzRLwE/Prz6DDQ9tqcERoZ0ZusoCtmGkAqx?=
 =?us-ascii?Q?sYI8kuLBdVR8Bv8TkcGmBD+OZKWnzCKIxfDUJkY2EIFCixIfVfSNGQjjM+mX?=
 =?us-ascii?Q?DZuKVKyHHfUra0NHVsBvv1UJYVBUF79Ha4fG7di7buE68JtLbLcDBhzwUWtk?=
 =?us-ascii?Q?wtD5B6CpjzMVDXO9+KIFZgncuCtVaU9F3MztwG0gxIv9ZZEfMv8PS5bw1oXt?=
 =?us-ascii?Q?l2dGstVSZiTQ0sE/2emQqR4lZJa44pPZ9eR5/zLvHCijdutzuF8hUR/CwRto?=
 =?us-ascii?Q?bEZR3Mqxp1ybgXd/7N1zeQzdshRzE81stcRCOsLB6Pmeo78+kPR6ksNzBsJT?=
 =?us-ascii?Q?MX/ewl3c8fB/7XUD9YLmeyF5Mdx6dxAPcimzN6sqciirpvRGb/Qsot2oifUi?=
 =?us-ascii?Q?DxtawEJwLouoHrUfT9jFL07LvfQ2C9XZrD3kn1lRgrujCEADhVzWXjdbYUNL?=
 =?us-ascii?Q?Awy7T74gM3cAmX5fY6LJKR4CcBSPiygQ7TSX4x/BEcD3Als+LzALHJG1GEk6?=
 =?us-ascii?Q?iQDkdw1KmxU9WHRJdowoVtaRD9/ZWXOaV8wZPKzmS/nt4tcVnON4PHvIG3pB?=
 =?us-ascii?Q?XGZr8dS9bMJrLtWKjOwuEzqnWpNs3JisWfuQovAqetYY2ZGMvW/d2bvKbocu?=
 =?us-ascii?Q?+aSIS/jI+y17TYATCJnzI+atRinb8cGlWh/Y903zf5Hu0FrFtfQJWS/KPwXS?=
 =?us-ascii?Q?F1yulxSy6Gc9OlEVgX789Djk0eenIIa4UW43Pqu+LNht3PoZEVtqcHXbkuB1?=
 =?us-ascii?Q?LWhhMXegWwjjCBaOYimMVwLZGdJh3MWH0Bd89nXOrL3oWcnGp7cPHK98W7Bi?=
 =?us-ascii?Q?stbK2/GN0dqSYNhmzGJp7MHq9YglD7ukJcOQLKCxiIL7Le8xAXbs4d1SOW3y?=
 =?us-ascii?Q?m+3tk6zKT5RTt93yG/NTaa8urHwHBh80K15eXskknJH345pV/YrvW61B+Vs0?=
 =?us-ascii?Q?nr1J5XJiwGWzv52RpxGSs6PWKMZaFkI+ZgH7rFBgPRH9KMehaKQkQMpthXaV?=
 =?us-ascii?Q?Dvi+joTPqApiOE8i/UskQfFrdDFwkHAv+shnGgkKyu1Dzx8Y2kq3S19sd/Qx?=
 =?us-ascii?Q?J2ZojFIafVBOCdn5w+qegLpaC+DIFjBrsZI9BX37njJFIitUfbSowf2fphzu?=
 =?us-ascii?Q?E9pmDK+9Vl65ABn8wdoyvao3xEDtwA4Ff75/UPeZO0HVKhuZ8SfChToFXHHe?=
 =?us-ascii?Q?Zi3wUcqSz2vWFR4CNVfIFpuTOr/pM4npGS5NHbIc41PnrI6JCGNtq5hHQ/RR?=
 =?us-ascii?Q?JdKcliyme8OyIjjC65XK8ofl8OBv7bucStwgHdwW4SRmZk6SCYwg3FHSyMpQ?=
 =?us-ascii?Q?5A=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: f559544e-7d75-4f8c-7d3b-08daf85b4380
X-MS-Exchange-CrossTenant-AuthSource: PH8PR11MB6779.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Jan 2023 07:20:08.6427
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: pjt0thWS1Ar85kHDIEduSuciI3h0dwTojQzeYBqf/RQNzj51PyeFYGzzc+utWhw3tcy9NWdiXNPeDVKdD28DbA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BY1PR11MB8008
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,
        SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi, Vlastimil,

On Thu, Jan 12, 2023 at 08:56:59AM +0100, Vlastimil Babka wrote:
> 
> Actually no, by "obscure" means with CONFIG_SLUB_DEBUG it wouldn't happen
> anymore. But this is the opposite, it seems to happen a lot. I would have
> preferred that slub debugging catches some slab misuse, but this seems
> useful too. With such fail rates you can perhaps try ealier kernels than 6.0
> and eventually find the truly clean and first bad release and bisect?

Thanks a lot for guidance!

yeah, we reached back to until v5.14-rc1 which still has similar issue,
and v5.13 is clean. new bisection was triggered then we got '7118fc2906'

this was already reported as
"[linus:master] [hugetlb]  7118fc2906: kernel_BUG_at_lib/list_debug.c"
at https://lore.kernel.org/all/202301170941.49728982-oliver.sang@intel.com/
and I add you, Hyeonggon, Feng and Fengwei there.

hope that would be helpful.