Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 3 Nov 2022 16:16:57 +0800
From:   Feng Tang <feng.tang@intel.com>
To:     John Thomson <lists@johnthomson.fastmail.com.au>
CC:     Hyeonggon Yoo <42.hyeyoo@gmail.com>,
        Vlastimil Babka <vbabka@suse.cz>,
        Andrew Morton <akpm@linux-foundation.org>,
        Christoph Lameter <cl@linux.com>,
        Pekka Enberg <penberg@kernel.org>,
        David Rientjes <rientjes@google.com>,
        Joonsoo Kim <iamjoonsoo.kim@lge.com>,
        Roman Gushchin <roman.gushchin@linux.dev>,
        Dmitry Vyukov <dvyukov@google.com>,
        "Jonathan Corbet" <corbet@lwn.net>,
        Andrey Konovalov <andreyknvl@gmail.com>,
        "Hansen, Dave" <dave.hansen@intel.com>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "kasan-dev@googlegroups.com" <kasan-dev@googlegroups.com>,
        Robin Murphy <robin.murphy@arm.com>,
        John Garry <john.garry@huawei.com>,
        Kefeng Wang <wangkefeng.wang@huawei.com>,
        Thomas Bogendoerfer <tsbogend@alpha.franken.de>,
        John Crispin <john@phrozen.org>,
        "Matthias Brugger" <matthias.bgg@gmail.com>,
        "linux-mips@vger.kernel.org" <linux-mips@vger.kernel.org>
Subject: Re: [PATCH v6 1/4] mm/slub: enable debugging memory wasting of
 kmalloc
Message-ID: <Y2N4+dqhezRgk87k@feng-clx>
References: <53b53476-bb1e-402e-9f65-fd7f0ecf94c2@app.fastmail.com>
 <Y2DngwUc7cLB0dG7@hyeyoo>
 <29271a2b-cf19-4af9-bfe5-5bcff8a23fda@app.fastmail.com>
 <Y2D4D52h5VVa8QpE@hyeyoo>
 <Y2ElURkvmGD5csMc@feng-clx>
 <70002fbe-34ec-468e-af67-97e4bf97819b@app.fastmail.com>
 <Y2IJSR6NLVyVTsDY@feng-clx>
 <Y2IZNqpABkdxxPjv@hyeyoo>
 <Y2NrRt5FF+zi4Vf1@feng-clx>
 <f479b9cc-1301-410c-a36e-80c365964566@app.fastmail.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <f479b9cc-1301-410c-a36e-80c365964566@app.fastmail.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?aL5dR+7Ne4BPbyJt55oK7YR4BHFYscMiofGoALtZeLbbEbuzWJLa3AgreId2?=
 =?us-ascii?Q?eCymeONfB3OSaorDiUlkh4e3VF0Kl/XiwerKNapDpUICHKUvD/qh4g+3froq?=
 =?us-ascii?Q?ekC1GB1ERrkor1XR711lWpAGoDV0kI5nyFN+hBlAvFoQzbT7O6NWZHruMdNw?=
 =?us-ascii?Q?PlP7/TrGAFSzOJ2OtH8tDJaIdsviKxk75CZFolCAeIH/U7cAgyLyry60jYVH?=
 =?us-ascii?Q?03Aujvm5AkZ0ITw71rkQwgnjNIr5rklMFXR2NXzc6foiclMz+T1ipspp/Wbn?=
 =?us-ascii?Q?kZVqbdyTj3ntJ5uJel31Fot0MwPd2YClbpgLO4JfDQ087vU3sD6NkcK/tkUM?=
 =?us-ascii?Q?JrzdneD6qSzHAfINzrELVIDKyM8ThZzAvyrpi06XoKvXsIk3WhcNJYEDdMig?=
 =?us-ascii?Q?RV6l4vkRffBlpBOK08SsVYneVgAGXlmL8KYbm1xs+/Z6MvM346Zk0UOlJPz8?=
 =?us-ascii?Q?GYQLF6QSqg6WAGz4UXi6UPY7sO+NGE9VA+e+4//CSXy6ehqfL2TIkHQtIoNu?=
 =?us-ascii?Q?t6ha1TFJXoClRLn1vxCJ8LpPhhKIbbHIQg8xnUDoxUQk61rxs7/pzyupLBQt?=
 =?us-ascii?Q?Q0UXvPDEBOvXDCO5xlKlMI8TJV7SeKHEig2jqbONbH7Xg30E28GvvKVwpkD/?=
 =?us-ascii?Q?iTgritIEVNkthWdNdPtcc5eGNC6jF8cCVqBBE5NMnctLbu0y9rz0K1S46MTE?=
 =?us-ascii?Q?BH/xrVcAHVYaR48QE61SNpRoe+MyIXJ8mjXEteQvVAfzZnCc5y5DCp1Nngpn?=
 =?us-ascii?Q?zBAqWmXk1i9Xz9l/oJBM2oDEwDFI5O7u/M/tl8D0qPp1UOkCgN/sREIbBQc4?=
 =?us-ascii?Q?6h5HFIRRu7pPS2hMd+7X+jo+YukX9ewALfWsasz9VhNxOu2Sc6YNnFpsklOd?=
 =?us-ascii?Q?A0Ostbo1sTiAOINjUl0fBnVroKLm+0N2x7uSSKQKJoVfrnMxzdn4yZcziOcw?=
 =?us-ascii?Q?ZrRx1W2kpyEdjVQZwq8DiBscfD+k3ay8x0SsKqzrrUHH911KOB2x/3Y5BkXR?=
 =?us-ascii?Q?aOyHXbErgfaagWvapWP/Y5Av5LMalgVEuFCPh95XcProYr5m0YfeZsdgjK0V?=
 =?us-ascii?Q?OYlwC4AUHdUbZxmuCOYihhGND3DYReQdGEG7cEkQ1zXof0RPwZl9Gug9MYLb?=
 =?us-ascii?Q?uaT0OdEvFnTo4HfDwu4CuqTrRBKCTwN8Y9SWbKhWjBSeaMaCdkW3xfwrKqIC?=
 =?us-ascii?Q?RABVMho3wA7N4cEYBRgwOU31qrdT7eKmJUTxDVtpVN++Ma8s1d58c8TxJvM4?=
 =?us-ascii?Q?a1bUKadUF5sKMhn0odL692b4ap6GxYaz+mf66/w++S8JfQlTJLVaWoREh3/8?=
 =?us-ascii?Q?7q1c9amvX4IBtrwPXISnhyTst/37hIrJm7MNYqGUM0zHUKFr+RmyYwvBQ3k4?=
 =?us-ascii?Q?0i4ExV/NO3JRq4trmzkBvo8J7p/LwDtJOtb/hTmYbC5lqW/xlA794qjiVt01?=
 =?us-ascii?Q?e1rLEAgI1040VJ1ZacLyRHt4cUfGbQ8Y3fSCsC7j9GKvfHjT+i8rnEtA3LpH?=
 =?us-ascii?Q?rxfXeGl5nVCoLioJW8cePHEtUUp8O1jCnciBaMk6epbEjQRe2bNJW8vrUsl6?=
 =?us-ascii?Q?RsqHkjAnfNPYxkNAVFFltvB8bKGFGU8sunI3HTL7?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 9fc455c7-354c-4c5f-a791-08dabd743a72
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6304.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Nov 2022 08:20:12.2981
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9GZzhDotdqARZam/1HO7zJbPp6Vv//sHUyuua21vW6/sz6IVRKS1c/krP/QTHgGyYKJNfzzTgVedZznrziKeFA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR11MB7565
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-8.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,
        RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Nov 03, 2022 at 07:45:49AM +0000, John Thomson wrote:
> On Thu, 3 Nov 2022, at 07:18, Feng Tang wrote:
> > On Wed, Nov 02, 2022 at 04:16:06PM +0900, Hyeonggon Yoo wrote:
> >> On Wed, Nov 02, 2022 at 02:08:09PM +0800, Feng Tang wrote:
> > [...]
> >> > > transfer started ......................................... transfer ok, time=2.11s
> >> > > setting up elf image... OK
> >> > > jumping to kernel code
> >> > > zimage at:     80B842A0 810B4BC0
> >> > > 
> >> > > Uncompressing Linux at load address 80001000
> >> > > 
> >> > > Copy device tree to address  80B80EE0
> >> > > 
> >> > > Now, booting the kernel...
> >> > > 
> >> > > [    0.000000] Linux version 6.1.0-rc3+ (john@john) (mipsel-buildroot-linux-gnu-gcc.br_real (Buildroot 2021.11-4428-g6b6741b) 12.2.0, GNU ld (GNU Binutils) 2.39) #73 SMP Wed Nov  2 05:10:01 AEST 2022
> >> > > [    0.000000] ------------[ cut here ]------------
> >> > > [    0.000000] WARNING: CPU: 0 PID: 0 at mm/slub.c:3416 kmem_cache_alloc+0x5a4/0x5e8
> >> > > [    0.000000] Modules linked in:
> >> > > [    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 6.1.0-rc3+ #73
> >> > > [    0.000000] Stack : 810fff78 80084d98 00000000 00000004 00000000 00000000 80889d04 80c90000
> >> > > [    0.000000]         80920000 807bd328 8089d368 80923bd3 00000000 00000001 80889cb0 00000000
> >> > > [    0.000000]         00000000 00000000 807bd328 8084bcb1 00000002 00000002 00000001 6d6f4320
> >> > > [    0.000000]         00000000 80c97d3d 80c97d68 fffffffc 807bd328 00000000 00000000 00000000
> >> > > [    0.000000]         00000000 a0000000 80910000 8110a0b4 00000000 00000020 80010000 80010000
> >> > > [    0.000000]         ...
> >> > > [    0.000000] Call Trace:
> >> > > [    0.000000] [<80008260>] show_stack+0x28/0xf0
> >> > > [    0.000000] [<8070c958>] dump_stack_lvl+0x60/0x80
> >> > > [    0.000000] [<8002e184>] __warn+0xc4/0xf8
> >> > > [    0.000000] [<8002e210>] warn_slowpath_fmt+0x58/0xa4
> >> > > [    0.000000] [<801c0fac>] kmem_cache_alloc+0x5a4/0x5e8
> >> > > [    0.000000] [<8092856c>] prom_soc_init+0x1fc/0x2b4
> >> > > [    0.000000] [<80928060>] prom_init+0x44/0xf0
> >> > > [    0.000000] [<80929214>] setup_arch+0x4c/0x6a8
> >> > > [    0.000000] [<809257e0>] start_kernel+0x88/0x7c0
> >> > > [    0.000000] 
> >> > > [    0.000000] ---[ end trace 0000000000000000 ]---
> >> > > [    0.000000] SoC Type: MediaTek MT7621 ver:1 eco:3
> >> > > [    0.000000] printk: bootconsole [early0] enabled
> >> > > 
> >> > > Thank you for working through this with me.
> >> > > I will try to address the root cause in mt7621.c.
> >> > > It looks like other arch/** soc_device_register users use postcore_initcall, device_initcall,
> >> > > or the ARM DT_MACHINE_START .init_machine. A quick hack to use postcore_initcall in mt7621
> >> > > avoided this zero ptr kmem_cache passed to kmem_cache_alloc_lru.
> >> > 
> >> > If IIUC, the prom_soc_init() is only called once in kernel, can the
> >> > 'soc_dev_attr' just be defined as a global data structure instead
> >> > of calling kzalloc(), as its size is small only containing 7 pointers.
> >> 
> >> But soc_device_registers() too uses kmalloc. I think calling it
> >> after slab initialization will be best solution - if that is correct.
> >
> > Yes, you are right, there is other kmalloc() down the call chain.
> >
> > Hi John,
> >
> > Will you verify and submit a patch for your proposal of deferring
> > calling prom_soc_init()? thanks
> >
> > - Feng
> 
> Hi Feng,
> 
> My proposed mt7621.c changes are RFC here:
> https://lore.kernel.org/lkml/20221103050538.1930758-1-git@johnthomson.fastmail.com.au/

Great!

> That series lets me boot the v6.1-rc3 kernel. I have only tried it with my config (as sent earlier). If there are other suspect config settings that I should test, please let me know?
> I used device_initcall, but postcore_initcall also works fine.

I'm not sure which order is better, due to lack of mips platform
knowledge.

> I rephrased Vlastimil's explanation and used it in patch 3 description.
> I have not referenced a Fixes tag yet (unsure which/if any I should use)

With older version, the kernel boots fine with soc_dev_init() not
being actually called, and I don't know if they also need to get
this called.

Thanks,
Feng

> Cheers,
> -- 
>   John Thomson
>