Received: by 2002:a05:6358:16cc:b0:ea:6187:17c9 with SMTP id r12csp7503627rwl; Tue, 10 Jan 2023 01:27:10 -0800 (PST) X-Google-Smtp-Source: AMrXdXtWjihTNaZHOrWlRrmJxhXlneKM+IPclXt94Ezbz6XR0g02jltGqA/89OLBjB/s3Lm8VCaY X-Received: by 2002:a05:6402:1486:b0:482:ab74:6432 with SMTP id e6-20020a056402148600b00482ab746432mr51959376edv.20.1673342829871; Tue, 10 Jan 2023 01:27:09 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1673342829; cv=none; d=google.com; s=arc-20160816; b=JosPWi0sZTiX5288Ekye8IE051c3lFTc3y6cfOlki3zYqkHNCqtBDyggd+oslWU1x+ QLPfTtW2i5UUoHBJE45vySbrdP3x+dY/ulhrmeKZ/TbbSzVaJSlHxlnTI7GhddkFB3ma jiuZhyEDMns3+iR/nwOUdpNLKnOgvDeQUPchPjPIjmRUnwQRZKIZ17fnH2jOO4EN4/Bf SP6/C1EyU11fytsSzn6Z0A4BUXqv5z7aP3Jw9bThlhvl/7ukNatUI7mN1G0goSdUERMx eYxYb/cySIPvVgh5DQ1l0KuRb6CF0VyP63KEy8QJ+fni17qBjD5PIbN6jUVTCQhf7ad3 wD3A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:subject:message-id:date:from:in-reply-to :references:mime-version:dkim-signature; bh=19QYjJTilGk/RhxRe6+HhMCtJmIsJ5ED63/iAKX6oJs=; b=hAT9J+qbQNJLTTUHgkulbjesHDPMn9FxynbJ6voL5QPGRsACUW9Y0QlY9wa4eHERwn 7jqxHthHPeJA5U21aYIdF+qzW/71rLE+EzeAjwDbkeLUHpj51hX3ke1Tb7JqGjVodyEr ABys5COH+7+FxOuOdWpjTdu778J+S17kEylLhbYGLvtmEdyQjv0yJ1263u8xmojleajR DtWog2iZtcfqHdhrXpMc68hUz21LXAmXTWLCvRwqlA7ikuYQ9Tx0/S4/p/Mj6QRyTX0G W4ySJ85xJU46HGYCVwYfGqvc3KKkrecYIaRghuRerjWaDRGpWVJP9JqAN9sG6yYLYhkg MLsQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=lbCmyLed; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id eb11-20020a0564020d0b00b0047866149cdasi14265677edb.388.2023.01.10.01.26.57; Tue, 10 Jan 2023 01:27:09 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=lbCmyLed; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S230526AbjAJI4a (ORCPT + 53 others); Tue, 10 Jan 2023 03:56:30 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:42520 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S237838AbjAJIz7 (ORCPT ); Tue, 10 Jan 2023 03:55:59 -0500 Received: from mail-yb1-xb32.google.com (mail-yb1-xb32.google.com [IPv6:2607:f8b0:4864:20::b32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 6A34559FAF for ; Tue, 10 Jan 2023 00:53:11 -0800 (PST) Received: by mail-yb1-xb32.google.com with SMTP id e76so11082807ybh.11 for ; Tue, 10 Jan 2023 00:53:11 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:from:to:cc:subject:date:message-id:reply-to; bh=19QYjJTilGk/RhxRe6+HhMCtJmIsJ5ED63/iAKX6oJs=; b=lbCmyLed2VvMiWlAcFE73mcQDp5etVdnVHUJB0fOIywFthZ8lU7tD2PGNO2HC3zSAc dWKpNjNnenpRB/dYUu/Bi9UKnr53swijKGeB+DGMJw/CtYP9CyD/TXtTQ84qKtpOqa1B we5SLgjXbaeHfSmWsG3ddc7582JJaXXNPM0ZVcUZvBZvkBf1GjtFGD4jByY9MTuwdTZQ LqlMAkCjgFRNQ11MujIamnA4f5V112faxZSPfRUJ3D5SWQ8SxH5+y7WVLOCty9zTEV4H 9x6AtFdn5fE2ZaudYTtWp4NSrDa4id6ZpWxndwXOdsPPP03SkrZFtgTmG+ErjTOg9i05 S0iw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=cc:to:subject:message-id:date:from:in-reply-to:references :mime-version:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to; bh=19QYjJTilGk/RhxRe6+HhMCtJmIsJ5ED63/iAKX6oJs=; b=grnvTEwko7724Ahw6XtgBAKtCQroTHxQEgp9Y9DFa/N9ZvLzvg7kELvLOPN/PVydNU RIetu5Oc5itAUNnceIjCD9+tp4WSX8ofCkgGSkOEsKKy7ELvKsH81X1185leu9HSGdxB btpksYZGiuA3j5n5Q1XA96HDvavFgtcoRdx1YeWBBO3xbOZrzh/P/LgQsSwmH1eQWfMF FzTgScV602Z+k8tgIjZIgH2RXdV5H4IvETcp656F8LG0nkGgQ3EWYBId2Q2UaDQ5ejsp du2k4bxWPKT+JvVsOpSi+EY8NC7yNEjrsUTJLJYKAUdaqrBIeJZP7Tj2eI2eNcjCGIcd Rbwg== X-Gm-Message-State: AFqh2kp2vg2B+BvmhKEk6XCXfXwqih2nel9wRRoj5JmYiVDB41XPqpIC KnmQs1eQFhi/ELZoXnOzyT4S9S64TyOwTp/u0Kr0uA== X-Received: by 2002:a5b:b47:0:b0:6fe:1625:f1f5 with SMTP id b7-20020a5b0b47000000b006fe1625f1f5mr6647952ybr.549.1673340786873; Tue, 10 Jan 2023 00:53:06 -0800 (PST) MIME-Version: 1.0 References: <20220701142310.2188015-1-glider@google.com> <20220701142310.2188015-11-glider@google.com> <63b74a6e6a909_c81f0294a5@dwillia2-xfh.jf.intel.com.notmuch> <63bc8fec4744a_5178e29467@dwillia2-xfh.jf.intel.com.notmuch> <63bd0be8945a0_5178e29414@dwillia2-xfh.jf.intel.com.notmuch> In-Reply-To: From: Alexander Potapenko Date: Tue, 10 Jan 2023 09:52:30 +0100 Message-ID: Subject: Re: [PATCH v4 10/45] libnvdimm/pfn_dev: increase MAX_STRUCT_PAGE_SIZE To: Dan Williams Cc: Greg Kroah-Hartman , Marco Elver , Alexander Viro , Alexei Starovoitov , Andrew Morton , Andrey Konovalov , Andy Lutomirski , Arnd Bergmann , Borislav Petkov , Christoph Hellwig , Christoph Lameter , David Rientjes , Dmitry Vyukov , Eric Dumazet , Herbert Xu , Ilya Leoshkevich , Ingo Molnar , Jens Axboe , Joonsoo Kim , Kees Cook , Mark Rutland , Matthew Wilcox , "Michael S. Tsirkin" , Pekka Enberg , Peter Zijlstra , Petr Mladek , Steven Rostedt , Thomas Gleixner , Vasily Gorbik , Vegard Nossum , Vlastimil Babka , kasan-dev , Linux Memory Management List , Linux-Arch , LKML Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-17.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org > > > > > > > > > > > -- >8 -- > > > > >From 693563817dea3fd8f293f9b69ec78066ab1d96d2 Mon Sep 17 00:00:00 2001 > > > > From: Dan Williams > > > > Date: Thu, 5 Jan 2023 13:27:34 -0800 > > > > Subject: [PATCH] nvdimm: Support sizeof(struct page) > MAX_STRUCT_PAGE_SIZE > > > > > > > > Commit 6e9f05dc66f9 ("libnvdimm/pfn_dev: increase MAX_STRUCT_PAGE_SIZE") > > > > > > > > ...updated MAX_STRUCT_PAGE_SIZE to account for sizeof(struct page) > > > > potentially doubling in the case of CONFIG_KMSAN=y. Unfortunately this > > > > doubles the amount of capacity stolen from user addressable capacity for > > > > everyone, regardless of whether they are using the debug option. Revert > > > > that change, mandate that MAX_STRUCT_PAGE_SIZE never exceed 64, but > > > > allow for debug scenarios to proceed with creating debug sized page maps > > > > with a new 'libnvdimm.page_struct_override' module parameter. > > > > > > > > Note that this only applies to cases where the page map is permanent, > > > > i.e. stored in a reservation of the pmem itself ("--map=dev" in "ndctl > > > > create-namespace" terms). For the "--map=mem" case, since the allocation > > > > is ephemeral for the lifespan of the namespace, there are no explicit > > > > restriction. However, the implicit restriction, of having enough > > > > available "System RAM" to store the page map for the typically large > > > > pmem, still applies. > > > > > > > > Fixes: 6e9f05dc66f9 ("libnvdimm/pfn_dev: increase MAX_STRUCT_PAGE_SIZE") > > > > Cc: > > > > Cc: Alexander Potapenko > > > > Cc: Marco Elver > > > > Reported-by: Jeff Moyer > > > > --- > > > > drivers/nvdimm/nd.h | 2 +- > > > > drivers/nvdimm/pfn_devs.c | 45 ++++++++++++++++++++++++++------------- > > > > 2 files changed, 31 insertions(+), 16 deletions(-) > > > > > > > > diff --git a/drivers/nvdimm/nd.h b/drivers/nvdimm/nd.h > > > > index 85ca5b4da3cf..ec5219680092 100644 > > > > --- a/drivers/nvdimm/nd.h > > > > +++ b/drivers/nvdimm/nd.h > > > > @@ -652,7 +652,7 @@ void devm_namespace_disable(struct device *dev, > > > > struct nd_namespace_common *ndns); > > > > #if IS_ENABLED(CONFIG_ND_CLAIM) > > > > /* max struct page size independent of kernel config */ > > > > -#define MAX_STRUCT_PAGE_SIZE 128 > > > > +#define MAX_STRUCT_PAGE_SIZE 64 > > > > int nvdimm_setup_pfn(struct nd_pfn *nd_pfn, struct dev_pagemap *pgmap); > > > > #else > > > > static inline int nvdimm_setup_pfn(struct nd_pfn *nd_pfn, > > > > diff --git a/drivers/nvdimm/pfn_devs.c b/drivers/nvdimm/pfn_devs.c > > > > index 61af072ac98f..978d63559c0e 100644 > > > > --- a/drivers/nvdimm/pfn_devs.c > > > > +++ b/drivers/nvdimm/pfn_devs.c > > > > @@ -13,6 +13,11 @@ > > > > #include "pfn.h" > > > > #include "nd.h" > > > > > > > > +static bool page_struct_override; > > > > +module_param(page_struct_override, bool, 0644); > > > > +MODULE_PARM_DESC(page_struct_override, > > > > + "Force namespace creation in the presence of mm-debug."); > > > > > > I can't figure out from this description what this is for so perhaps it > > > should be either removed and made dynamic (if you know you want to debug > > > the mm core, why not turn it on then?) or made more obvious what is > > > happening? > > > > I'll kill it and update the KMSAN Documentation that KMSAN has > > interactions with the NVDIMM subsystem that may cause some namespaces to > > fail to enable. That Documentation needs to be a part of this patch > > regardless as that would be the default behavior of this module > > parameter. > > > > Unfortunately, it can not be dynamically enabled because the size of > > 'struct page' is unfortunately recorded in the metadata of the device. > > Recall this is for supporting platform configurations where the capacity > > of the persistent memory exceeds or consumes too much of System RAM. > > Consider 4TB of PMEM consumes 64GB of space just for 'struct page'. So, > > NVDIMM subsystem has a mode to store that page array in a reservation on > > the PMEM device itself. > > Sorry, I might be missing something, but why cannot we have > > #ifdef CONFIG_KMSAN > #define MAX_STRUCT_PAGE_SIZE 128 By the way, KMSAN only adds 16 bytes to struct page - would it help to reduce MAX_STRUCT_PAGE_SIZE to 80 bytes? > #else > #define MAX_STRUCT_PAGE_SIZE 64 > #endif >