Received: by 2002:a05:6358:111d:b0:dc:6189:e246 with SMTP id f29csp145584rwi; Wed, 2 Nov 2022 10:14:35 -0700 (PDT) X-Google-Smtp-Source: AMsMyM4y4f3qvhMZmf9oTkaAg/1auQjyAEQDhYMAq4gNqeN8ED1H0qsJzqG+bXgkcOePPODWe0nK X-Received: by 2002:a05:6402:254f:b0:45d:3044:d679 with SMTP id l15-20020a056402254f00b0045d3044d679mr26651757edb.137.1667409275052; Wed, 02 Nov 2022 10:14:35 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1667409275; cv=none; d=google.com; s=arc-20160816; b=YG7hNQ7eCpGGAfXQkTVDuLhLXvfcFrzIV5PH7DIl0r8Zs4WyOQKr6i1WaLCwFd5ih7 Do6vMC1ZAs5nRarvIOIU2W4FucJiG6/z0JP7Sc5sCbudCxuDL9OeByyHGV+YWgz5HsPU ZKP9v93HQ0ulOIIGmeprsPEs+HEJ/rX0pQ07cVbbneBjAX2XEKKNXGk2R62AJXpp22SF 5hwZmzCshpVs9slWfRT4uqVFyg1HyokNduXkHXZ24PYcnLnt6MJGJSSdrjbZSdTFN5SX FWRIna/n1aDBLLwDizSMt+qO83n1Cbo3gxgqb4MgPvproci0vGYiUHR1bRIdJEQy0YQo j6rQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:in-reply-to:reply-to :content-language:references:cc:to:subject:from:mime-version:date :message-id:dkim-signature; bh=tRu7jz2k09j/r4Z17Q9gh7+UJotFgN5pS21DDeudeqI=; b=hZrkBx/xOGBUxYMhQb7EzMRjmdxvVw5phbVwslmEK1zfIyIkCJ8/XOEgQGc3Bimnvc jdbuzceSyhzwkAI+EMFg+NJYcoiU1hmd1kWuhhrIR5Nh/aZBaS+htG463PvZum8sAGbg brsD/VndetMAXNa1DyvB1NW5TScTYl9fN2ix5Qe+Vrk5Hk9R7VwQi1JCWPQtqRY38JyS 0QkHUOy4cVBP3IUVJY0iEAesCnmzTEyV+GCZ32ZuUyC4Mj9gaiTJ2A8ivMT44w62Z52C 1n6NPXXp+0UvPvj9jg2Qw4GDtDKOe4ThqmkA7DcA0fvc9w9PmaVlQ4NoabvzD4qyWetT eRhg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@posteo.eu header.s=2017 header.b=hv5PT0qq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=posteo.eu Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id h20-20020a170906111400b0078dad5e8d65si12232332eja.68.2022.11.02.10.13.47; Wed, 02 Nov 2022 10:14:35 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@posteo.eu header.s=2017 header.b=hv5PT0qq; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=posteo.eu Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230089AbiKBP6E (ORCPT + 98 others); Wed, 2 Nov 2022 11:58:04 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36796 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229551AbiKBP6C (ORCPT ); Wed, 2 Nov 2022 11:58:02 -0400 Received: from mout01.posteo.de (mout01.posteo.de [185.67.36.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3B22B2A70A for ; Wed, 2 Nov 2022 08:57:59 -0700 (PDT) Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id CFC03240026 for ; Wed, 2 Nov 2022 16:57:57 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.eu; s=2017; t=1667404677; bh=TXw9bmFqmvwkICnHKRf9HU/0WQCtu3aGxJIdAtZv4Sg=; h=Date:From:Subject:To:Cc:From; b=hv5PT0qqiboAwsPXiSRY1PJjIYv9QBO5WxdsKDFMEb2wJzzz46swXQaoFjpqZ53Qi Ch5AWS1A0E9u7T/CzCGCTZu/NnBWtx3Uh2GJwD1dYrYskxL6/eouOw5NG7aWf2ebvd 9Jjrcg2keTmBN1HQpoPO8jlVEYcJuM+t1XhKiVpj+n0vkHydR69x53N2pGMDtT0dqx D1jccy25WEmoMR3tj147D0eEEWhBo/N0v5TrUW7DKQMjwrlxbbiht7tcWDFB0/rZSb 7lGekF8mUZDY0my/gd1x8H47ZOhe3hMz4I8/vW5wllAqzfL4tqunGUvPMTsaFY94SO RWjQLKh7tfLOQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4N2WkY0Blhz6tnX; Wed, 2 Nov 2022 16:57:56 +0100 (CET) Message-ID: <25d7e982-f959-d86c-a6e7-5efd1db1e75c@posteo.de> Date: Wed, 2 Nov 2022 15:57:56 +0000 MIME-Version: 1.0 From: postix@posteo.eu Subject: Re: Fwd: [REGRESSION 6.0.x / 6.1.x] NULL dereferencing at tracing To: rostedt@goodmis.org Cc: linux-kernel@vger.kernel.org, regressions@lists.linux.dev, tiwai@suse.de References: <71829e56-a13f-0462-37a7-a4d64c16f561@posteo.de> Content-Language: en-US, de-DE Reply-To: 20221031144850.5522b036@rorschach.local.home In-Reply-To: <71829e56-a13f-0462-37a7-a4d64c16f561@posteo.de> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_NONE, SPF_HELO_NONE,SPF_PASS autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Hello everyone, I have added lot's of debug printk's to see what's happening and I found that the "cpu" counter, which is used to access the buffer's array elements (cpu_buffer = buffer->buffers[cpu]) in the ring_buffer_wake_waiters function, exceeds the maximum number of total of total cores, namely in my case 24, which means, it should only run from 0..23. However, upon debugging, it runs up to 31, and thus causing a NULL pointer dereference (&cpu_buffer->irq_work). After adding a return statement in case cpu > 24, the bug is no longer reproducible. You can find the diff between v6.1-rc2 and the patched version with added debug log in [1]. The corresponding dmesg output can be found in [2]. I hope this gives you a good hint to find the root cause! [1] https://paste.opensuse.org/e60601aa [2] https://paste.opensuse.org/bf1398ce