Received-SPF: pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 11 Feb 2022 12:56:24 -0400
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     David Hildenbrand <david@redhat.com>
Cc:     Alex Sierra <alex.sierra@amd.com>, akpm@linux-foundation.org,
        Felix.Kuehling@amd.com, linux-mm@kvack.org, rcampbell@nvidia.com,
        linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org,
        amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org,
        hch@lst.de, jglisse@redhat.com, apopple@nvidia.com,
        willy@infradead.org
Subject: Re: [PATCH v6 01/10] mm: add zone device coherent type memory support
Message-ID: <20220211165624.GP4160@nvidia.com>
References: <20220201154901.7921-1-alex.sierra@amd.com>
 <20220201154901.7921-2-alex.sierra@amd.com>
 <beb38138-2266-1ff8-cc82-8fe914bed862@redhat.com>
 <20220211164537.GO4160@nvidia.com>
 <6a8df47e-96d0-ffaf-247a-acc504e2532b@redhat.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <6a8df47e-96d0-ffaf-247a-acc504e2532b@redhat.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?4jQWN3+Q5BSU6bBLY7MFuwREZY4ESOKs3pG7gGWIBfMqLl4hgieHYS3zenSl?=
 =?us-ascii?Q?/QCjtbVDeFmFdXQtyYwRvwqsL+coWAkYs8yS0gaiOi7izPm8/mb0UX7ThmGu?=
 =?us-ascii?Q?AtGi64RDBp9GfYXjcYSWbhraSi54jdG9O2RiRr0KcS5RceyZTwnPmR2L2mPu?=
 =?us-ascii?Q?kG/qQlHZyV/hOxnn0vvKeITdk/+nvSWcKk211AIWegcB33vQ6lSsB9C3P9Za?=
 =?us-ascii?Q?3hYr/udVk3FEjKX38ZLSgNZvg8M/R7+zEecRy1fgY7k0SgYD+4pXc+yNmycJ?=
 =?us-ascii?Q?HhEOc/ryVc65XQrL8+GjyIEytUdJZij00WxMVs+Q+bMGacgQV+socaQynGn/?=
 =?us-ascii?Q?NoyMwyJ/Wf85Uy7oY3EHyBDC3AqyjcW7KGlVEnRLMzZ0U0A8Y45mf+o0NGFW?=
 =?us-ascii?Q?hqEZ+/dhjpqlaYBO80L5wvWK70GqjGJcXywdqe2hDhu9EisjAJysxWvNwL7m?=
 =?us-ascii?Q?Rt46kuJCWvkoGyV1EvR9dcEQ5jy6GkQpyaQZYfBvishfTGPD2qzkDegoHfSm?=
 =?us-ascii?Q?6D6h/gCFyWn7wuAJkzDukdvG9gFi0kv2Aka3xc7q+0cA2Ozkz1FkLmP8jjR2?=
 =?us-ascii?Q?HyXlyGCl0SuIrFhpRPkyb9+ApeJHeSQAHGFyZfMNr8Yu6T+UsHJPle0ORmnS?=
 =?us-ascii?Q?HxOTLKoeqpZfultLjSTMFF8jMypD/FI/2E3kC7pDejnqMbWCtf0nMuMujFdD?=
 =?us-ascii?Q?yYVivSKXhgWULXmyljZTzMnwaJ8W+gdOUSBl9qfQQcPArRmowYEzDdT39hwO?=
 =?us-ascii?Q?NIaRVEGaWBTJ7+UntxSNGc//0uWPTfg4q519zjYjFXxvNt658tYVQ9NMkTxx?=
 =?us-ascii?Q?ZQs1ozBwQgTST8NK+iSrbKTQbsR1c39v08NiMcFFhfVsisUfIkMh0JyJG32t?=
 =?us-ascii?Q?nqozPrFgVjP9xQKWPi+XX+1oGy1h2ZWA6hHVCikkDf8tPSjqoCXSTGEFBu7d?=
 =?us-ascii?Q?hy/uaUf+SmFeP4MPp383gGnwgl2d0Loi+3xIc2R68l2iP7+jnbpw+QMqPU/o?=
 =?us-ascii?Q?Zszp1U8Y7bc2lqPulZiygMSCft0TrMkP6se9p9rvusBhVGd0YVjR6whFkr4G?=
 =?us-ascii?Q?kxRSS2NbT8gE1ghUDdPUTQRh5c4qd9WBkkvFGoFHm3Bo4FjRRjS8fQ88ZAmL?=
 =?us-ascii?Q?2MLD+oC80CFvPTxOoPxLFnYGak29cudRkunw5yFVkYkzSAxncC01oN4fpI93?=
 =?us-ascii?Q?Vjk9gKLDAD4iloDq9FyOezWqD3qkCZ2K5n4HG/ZRJrUPIQHMhvmnF4IYw5cs?=
 =?us-ascii?Q?UiClddInNWjDDFWr+5UJfmZ5OJqzKN5NHrHb09h7/6Mu8wWULrMu71J7a4x+?=
 =?us-ascii?Q?m2/cD1AbvBx/ri+p8IFRKCu8Anq+Gr240J6GtZU4Hm1DTWZ1w9blKLLS3maa?=
 =?us-ascii?Q?eX/XostVmah32dAw1H9Utw0n+5tIZDvAwosxoWGauDxuius9HZQvsH6nIf8o?=
 =?us-ascii?Q?BTTPTcm69MeisLrM/QmZAK71PcQCkvV2JXSgMRyZgkVnQ370fu4B1mIUuoAC?=
 =?us-ascii?Q?YO1NyQHpt1DeWGUaxk7iS1zM8BptNiZk4DXZQ5gHdOjfZ0/8m9KvtvwsxrVX?=
 =?us-ascii?Q?ZgTbozP/pQi0lQwNykM=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ec207d12-32b8-4992-63be-08d9ed7f70a2
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Feb 2022 16:56:25.7361
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: vhJ2aleoWcKILV1Kyk4ZJfO+yduo2mMiZcVlQBbRCJd3gz2K71CpD3fmhEL1uCRh
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR1201MB0272
X-Spam-Status: No, score=-1.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-ext4.vger.kernel.org>
X-Mailing-List: linux-ext4@vger.kernel.org

On Fri, Feb 11, 2022 at 05:49:08PM +0100, David Hildenbrand wrote:
> On 11.02.22 17:45, Jason Gunthorpe wrote:
> > On Fri, Feb 11, 2022 at 05:15:25PM +0100, David Hildenbrand wrote:
> > 
> >> ... I'm pretty sure we cannot FOLL_PIN DEVICE_PRIVATE pages
> > 
> > Currently the only way to get a DEVICE_PRIVATE page out of the page
> > tables is via hmm_range_fault() and that doesn't manipulate any ref
> > counts.
> 
> Thanks for clarifying Jason! ... and AFAIU, device exclusive entries are
> essentially just pointers at ordinary PageAnon() pages. So with DEVICE
> COHERENT we'll have the first PageAnon() ZONE_DEVICE pages mapped as
> present in the page tables where GUP could FOLL_PIN them.

This is my understanding

Though you probably understand what PageAnon means alot better than I
do.. I wonder if it really makes sense to talk about that together
with ZONE_DEVICE which has alot in common with filesystem originated
pages too.

I'm not sure what AMDs plan is here, is there an expecation that a GPU
driver will somehow stuff these pages into an existing anonymous
memory VMA or do they always come from a driver originated VMA?

Jason