Received-SPF: pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 21 Sep 2022 21:25:25 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Dan Williams <dan.j.williams@intel.com>
Cc:     akpm@linux-foundation.org, Matthew Wilcox <willy@infradead.org>,
        Jan Kara <jack@suse.cz>, "Darrick J. Wong" <djwong@kernel.org>,
        Christoph Hellwig <hch@lst.de>,
        John Hubbard <jhubbard@nvidia.com>,
        linux-fsdevel@vger.kernel.org, nvdimm@lists.linux.dev,
        linux-xfs@vger.kernel.org, linux-mm@kvack.org,
        linux-ext4@vger.kernel.org
Subject: Re: [PATCH v2 10/18] fsdax: Manage pgmap references at entry
 insertion and deletion
Message-ID: <YyurdXnW7SyEndHV@nvidia.com>
References: <166329930818.2786261.6086109734008025807.stgit@dwillia2-xfh.jf.intel.com>
 <166329936739.2786261.14035402420254589047.stgit@dwillia2-xfh.jf.intel.com>
 <YysZrdF/BSQhjWZs@nvidia.com>
 <632b2b4edd803_66d1a2941a@dwillia2-xfh.jf.intel.com.notmuch>
 <632b8470d34a6_34962946d@dwillia2-xfh.jf.intel.com.notmuch>
 <YyuLLsindwo0prz4@nvidia.com>
 <632ba8eaa5aea_349629422@dwillia2-xfh.jf.intel.com.notmuch>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <632ba8eaa5aea_349629422@dwillia2-xfh.jf.intel.com.notmuch>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?ISHAsAGIzJAN3SF6xX11s2obAZIvEzDnFewalLCD4wXJuw0NpJXpcr/JaSqg?=
 =?us-ascii?Q?uC0xrl08eOWR61nmDjEFOuv1dNBxExMfpZUzNhDaYTOgggAUWfAaluOOEbzu?=
 =?us-ascii?Q?eyUj77AB0kNunnjZ0T89PBx770c546QzW9i0Uv8LBnJjCefJjvcPEAxLMXVj?=
 =?us-ascii?Q?lHsxBqaD58iDes5QUrnifSBdPfQoyBs5lHD47WHJ+0nMPbYZFmolzrlPXjIX?=
 =?us-ascii?Q?MqeiwmDhG+Y1WQekLTlM76Ln7IOW/+8eBcSW6QroYPaIxwIPMyHJtG/TSD6X?=
 =?us-ascii?Q?etUcG3rU16ZGPC9P6LUiICz73Q1/s7/Jpn61UzmwPKe2RbngnUaS8rUGqOM/?=
 =?us-ascii?Q?Gwi1LIlgA1WZSIZSGiLi6nnMjN6BUw2huhLLbHVbYQL5RaKFxZucM0ytYDdf?=
 =?us-ascii?Q?+rVl3p7k2gzhs6Z4h9UtJsZOfcb2xSSxYzEBgIf3cTBvUvyeyFWbFrTrOGtG?=
 =?us-ascii?Q?ojCQKEaX24UQOjIqOq6z1fs0lNUx4oKxTEBlmfomwoXQ/usNW1PmmzvDhFte?=
 =?us-ascii?Q?8rjOfYBQiml2qwoIvuWyMeb8g/2GXn9cSLhIUDiU4hUQFk7vGOwdo0FdtNxy?=
 =?us-ascii?Q?PDgJTVlYSWYTvAonHKNy5dVYOPf3YZGz3OQvQPcUPKW7ifhl/PgYteMjDdyh?=
 =?us-ascii?Q?zHx7YTwIVy4RuN710VC4Ar8wlyQZEvVln0GNeowycBrBIChK0nwWmz5JnI+o?=
 =?us-ascii?Q?5RjoiC6jGWN/oQ5tAx4/MuibqPkaYj+BnB7hji2oAHtm1O7u91etUxmRL/UR?=
 =?us-ascii?Q?5vkVBrlmwXFFcKmunvjX8ICXwV5SNkmpC21zAD/zwx/uhF1ge91haxm/uw/i?=
 =?us-ascii?Q?J8tqb1hBaSXVydeXhFoyXJHDiGLFfctL7KXuvvI5EFY/qQX4bT+UivXKhkBg?=
 =?us-ascii?Q?rdGPhuYrwWENamGzkimjRjtL/317HicjSdnMb9Py8FQt3po+h3DX2EBdCMCp?=
 =?us-ascii?Q?oz7fVanpnIPnk80qV0PAJtimNH7NaMGjQue3jAj32tNv2sSs7CaIkBByCAie?=
 =?us-ascii?Q?ONVkp6yUCOWyOD6SeQfHV+Qrs7wbTpjAP5edU3WVv7G13JW8Spf4niHiFbpU?=
 =?us-ascii?Q?czktB0Wyy+2fAEgGJu27i/LqW+9ec9d9cEH5qB4nT7NI2FPo+W5B38lPAZXE?=
 =?us-ascii?Q?fpEVxGmIZmPisPRwttxcvKdmW9MQfg9gXdM2rZECV6NrQWUSF7ZJmb3aRIP6?=
 =?us-ascii?Q?q0ci3AyvRTWOIVQlCNS0XYiLL1UUtAwadxC6aaUeTcQHZ4mFOoQaniSz7LA9?=
 =?us-ascii?Q?LMP+5TDvtIQypk14k4Rq7DxTsoeNgN49RROZG82KAZIeEYQTeGC/rOp5elS/?=
 =?us-ascii?Q?E+Nz2PSjAvfkNEbEq7djUF++iPG6n2yXQewjjtBhPG7zKt+W4SSAkkJuv4y1?=
 =?us-ascii?Q?rPTD+i1lMnvj/WojYfDkVZSgbs8NguI0KFEF/8AW0v+02S3jhSk4BG494qz4?=
 =?us-ascii?Q?Z448P+j1RuPv/SPH2sygeqBfzfwJYGmMTB1rAm+5rvVD0LRujJG9JeE7PRfs?=
 =?us-ascii?Q?qAfVtOvoahV+4/fEYzXFjrIkYMqkje2VObnMeXpb4xONjmKHuLhBJaARcTh6?=
 =?us-ascii?Q?gWKiS62CnhlbnViC1CuL/hIfKHU0PguCkEV7CmXa?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 718fa14e-969f-4637-1907-08da9c30f247
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2022 00:25:26.4705
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: dybTyZIkA1CcSi+t+cqsHCM+G/EeruGWtOa/A/NG2SVHHSha0xgq5HBLW7pZHzWH
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7491
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE
        autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-ext4.vger.kernel.org>
X-Mailing-List: linux-ext4@vger.kernel.org

On Wed, Sep 21, 2022 at 05:14:34PM -0700, Dan Williams wrote:

> > Indeed, you could reasonably put such a liveness test at the moment
> > every driver takes a 0 refcount struct page and turns it into a 1
> > refcount struct page.
> 
> I could do it with a flag, but the reason to have pgmap->ref managed at
> the page->_refcount 0 -> 1 and 1 -> 0 transitions is so at the end of
> time memunmap_pages() can look at the one counter rather than scanning
> and rescanning all the pages to see when they go to final idle.

That makes some sense too, but the logical way to do that is to put some
counter along the page_free() path, and establish a 'make a page not
free' path that does the other side.

ie it should not be in DAX code, it should be all in common pgmap
code. The pgmap should never be freed while any page->refcount != 0
and that should be an intrinsic property of pgmap, not relying on
external parties.

Though I suspect if we were to look at performance it is probably
better to scan the memory on the unlikely case of pgmap removal than
to put more code in hot paths to keep track of refcounts.. It doesn't
need rescanning, just one sweep where it waits on every non-zero page
to become zero.

Jason