Received-SPF: pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 21 Sep 2022 11:03:25 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Dan Williams <dan.j.williams@intel.com>
Cc:     akpm@linux-foundation.org, Matthew Wilcox <willy@infradead.org>,
        Jan Kara <jack@suse.cz>, "Darrick J. Wong" <djwong@kernel.org>,
        Christoph Hellwig <hch@lst.de>,
        John Hubbard <jhubbard@nvidia.com>,
        linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev,
        linux-xfs@vger.kernel.org, linux-mm@kvack.org,
        linux-ext4@vger.kernel.org
Subject: Re: [PATCH v2 10/18] fsdax: Manage pgmap references at entry
 insertion and deletion
Message-ID: <YysZrdF/BSQhjWZs@nvidia.com>
References: <166329930818.2786261.6086109734008025807.stgit@dwillia2-xfh.jf.intel.com>
 <166329936739.2786261.14035402420254589047.stgit@dwillia2-xfh.jf.intel.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <166329936739.2786261.14035402420254589047.stgit@dwillia2-xfh.jf.intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?k9NNNFW/EAQBgUM+DK1oZp/HQjNtZEdxNvhTSwp60nPL0OShlIftoM16BFFE?=
 =?us-ascii?Q?TxnxyThJ36JOTekmALC4ppTqvR5oEDEKHFXCT6jQO1eOpyq9iSW/d4ouw8Fe?=
 =?us-ascii?Q?0PAmhJ9H7vR5mS6WTzueijsBJmt0t8t+VH6Lbhx1oMXNp+RTZWVq0BPhSZhc?=
 =?us-ascii?Q?dGmZGlYYnJ3oxyh0fETembdM2v1acxKrI+a2jP74fre+AjH2nWc63uWDyMzt?=
 =?us-ascii?Q?Vr+mvRnuWECH29dwG5buGeL2N11ra6GgkKL3VhyVyPLm1ahr+QUuklLGVaz5?=
 =?us-ascii?Q?DhBE9GyZC5d/kAJjW+fI/kBzZwbux/GOL/+rwtuwTmWyh24Qroyz9OFkyKEK?=
 =?us-ascii?Q?V5iP7xz9H6cU6ed1n5Yi2HPKGrFT53ufOiv+EoEmFQr2HYcQgID3xTHQKQg+?=
 =?us-ascii?Q?7BP07zl0DMDl2JG2AFfU7MUrHdCkGEDc0hEQfDGQhzHJ6Tq3X7LMPCRkJ4BS?=
 =?us-ascii?Q?0lhOQ/0cyYBEE17TOXdq1NuP4v4QCS2suzc6oo6zG28GWkh2VlhwVgtJmXZf?=
 =?us-ascii?Q?VsaCUFKRva+CMj/QFI0pb78yLb7myLfdsGEykpGh98MFFqCwHfBOr2Xo8z8H?=
 =?us-ascii?Q?d/F4Zzw+CAanTuriFCGh0TgVGZwuNU84qPgtE2drXp6VhxMl22KWyCLOS75t?=
 =?us-ascii?Q?R3ngl1cJUW4XZv39f8uZdu4o6eb/6B1C/weAdUx7rfv6nnIqiY/eUUsFyYLd?=
 =?us-ascii?Q?FKxgLLQpLJSPgKe9TJQZgvyV8o6ig1DbwKbu5iPIQKYCJD5eJ6ziJainoQUH?=
 =?us-ascii?Q?E8aO0YtDfhpOq8+MGGW2xGVCtLI2iUZ2jKULPcUobhlvLIfHNA/jQMbEkIKl?=
 =?us-ascii?Q?gLDTy8EC7t2jwE/oTY5oETy9mvTmBGKh4HpjTtcB+3CRzSb9BUbunixBXt0w?=
 =?us-ascii?Q?lQh8xciOn8SPFm59VVPYXW7Mf4ZJBN0JdGr24q/88+uiR1gHlvtTYy63RkH3?=
 =?us-ascii?Q?1SsFIlgZK9lcOsTmsiskbbJF5AmerKd1c7MFFeilscSYMGARLtiKAw5N9b7l?=
 =?us-ascii?Q?j9swqAF7+AMuIcFZU9mV8mZoTIOkg6GO7hnHjMXq/+BvASIW2iiIF5BOEMHH?=
 =?us-ascii?Q?m+55PcOf1U1YTI5zBs7vLcDqFUkNpKYcSSsI9Gyt9bMGhVL1zbvNf5g/riWJ?=
 =?us-ascii?Q?BBLNrtCO9AzZJFJXr4n/krBYxx9EQEqky5igL1xi2AkqlgofaTLkWdR7wRSf?=
 =?us-ascii?Q?xEeNZcVy9tPAOWOI3wXEuaA/I4VdrbkgJg0jZvfzCqAhM+/wCJHzl0J4vodE?=
 =?us-ascii?Q?bAbjSDVOZuGJiR8BCi4SA6L8DX2pMA5Qvi59AJt6kgWcH+AfKm7WnRYtuwbB?=
 =?us-ascii?Q?wUmx1H+K0XqJ4J+twHUjCVYsoWsIlKsDC8KWwdI6CJTBMCfkeHi0Z3qC2UaU?=
 =?us-ascii?Q?KziZWfY0H+XlGiLnKyFCculZToSoNuwb5GRpPT0yOD/8XOedwX5KgM5f3xy0?=
 =?us-ascii?Q?5LsiJBB/K9rqbS0Dq7CHC4++i7w1rRTfBvrBQzrejdm6SgwqhnmaOf4pT6cw?=
 =?us-ascii?Q?y0F95elXbGSdZh45plpHGDm4q89adjNA9PgnZ4LsxwStHbsi1wCOB5hZYIOA?=
 =?us-ascii?Q?iVasPDqmkXygYMG7r+dzNQAtmf5lv1NR9rBg3o7j?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ef07b360-34d7-466a-997b-08da9bda0dd9
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 21 Sep 2022 14:03:26.5618
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 5o2QvWsQVMxBoGhUx4ypqUUMg+akEqEACasop4laEQ1pKWsE2nf1wb98da+kE7LC
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7443
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE
        autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-ext4.vger.kernel.org>
X-Mailing-List: linux-ext4@vger.kernel.org

On Thu, Sep 15, 2022 at 08:36:07PM -0700, Dan Williams wrote:
> The percpu_ref in 'struct dev_pagemap' is used to coordinate active
> mappings of device-memory with the device-removal / unbind path. It
> enables the semantic that initiating device-removal (or
> device-driver-unbind) blocks new mapping and DMA attempts, and waits for
> mapping revocation or inflight DMA to complete.

This seems strange to me

The pagemap should be ref'd as long as the filesystem is mounted over
the dax. The ref should be incrd when the filesystem is mounted and
decrd when it is unmounted.

When the filesystem unmounts it should zap all the mappings (actually
I don't think you can even unmount a filesystem while mappings are
open) and wait for all page references to go to zero, then put the
final pagemap back.

The rule is nothing can touch page->pgmap while page->refcount == 0,
and if page->refcount != 0 then page->pgmap must be valid, without any
refcounting on the page map itself.

So, why do we need pgmap refcounting all over the place? It seems like
it only existed before because of the abuse of the page->refcount?

Jason