Received-SPF: pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
References: <20220505213438.25064-1-alex.sierra@amd.com>
 <20220505213438.25064-5-alex.sierra@amd.com>
 <SN6PR12MB27173F2F37294D6DDBC3457CFDC29@SN6PR12MB2717.namprd12.prod.outlook.com>
User-agent: mu4e 1.6.9; emacs 27.1
From:   Alistair Popple <apopple@nvidia.com>
To:     "Sierra Guiza, Alejandro (Alex)" <Alex.Sierra@amd.com>
Cc:     "jgg@nvidia.com" <jgg@nvidia.com>,
        "rcampbell@nvidia.com" <rcampbell@nvidia.com>,
        "willy@infradead.org" <willy@infradead.org>,
        "david@redhat.com" <david@redhat.com>,
        "Kuehling, Felix" <Felix.Kuehling@amd.com>,
        "amd-gfx@lists.freedesktop.org" <amd-gfx@lists.freedesktop.org>,
        "linux-xfs@vger.kernel.org" <linux-xfs@vger.kernel.org>,
        "linux-mm@kvack.org" <linux-mm@kvack.org>,
        "jglisse@redhat.com" <jglisse@redhat.com>,
        "dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org>,
        "akpm@linux-foundation.org" <akpm@linux-foundation.org>,
        "linux-ext4@vger.kernel.org" <linux-ext4@vger.kernel.org>,
        "hch@lst.de" <hch@lst.de>
Subject: Re: [PATCH v1 04/15] mm: add device coherent checker to remove
 migration pte
Date:   Thu, 12 May 2022 12:39:16 +1000
In-reply-to: <SN6PR12MB27173F2F37294D6DDBC3457CFDC29@SN6PR12MB2717.namprd12.prod.outlook.com>
Message-ID: <87fslfqwyo.fsf@nvdebian.thelocal>
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?P+O5kbRwh0tJFrP7gNfNVaD/xRBVVkGEHYZjDJPhxgqRkuQ3IrTj9QY2UkC3?=
 =?us-ascii?Q?ZKuqN4kNiSiNet0yMJBBGapBnIj0LUHHBcEBHvO3kufYljOqyjpHM5zDNJf6?=
 =?us-ascii?Q?sx3nU0wsw3A4Zfu6pYeAQctjqxk7Y4ColL6qTvbSaYGk7P1iNc565qVjBLV+?=
 =?us-ascii?Q?hjsQ2bgeX9u25bTAFOXEudAjtvJqMjDvwab5q4+N1jbyyKxXI7SiZhplAWOQ?=
 =?us-ascii?Q?1F5rzp3jSuJxjFwY160MS2Qu/oCylmkAM3gTFU0OdYb1qzIyPY5thmdUK8fM?=
 =?us-ascii?Q?zaTXoO9uV5aeJp2D17KBTQ53dHmqTlDMOFejuMrzivyiBG14PO7g79CkDJcD?=
 =?us-ascii?Q?THYOIHWuDRwwBEnvT66wYbhkl4ngwGij9Pv8ilu3XKWBLjgUR4QGH03IitqB?=
 =?us-ascii?Q?dV0ODFNpBItkz8DHCngsfFs5QCTKKbpLtr9vF1d+ABNaQbK16HZLO9bcvwH5?=
 =?us-ascii?Q?+eWBCzM0SkNpACRV2BbXIjw5oqldkwHPRNgUVr1OcDWPClVkbsr5AnYAW4VC?=
 =?us-ascii?Q?q+K+NI/K3+S8UkHRiWAALLKyqdCKvYErtRgsQ9yVsbKsDNUOeN7GFnFqod0J?=
 =?us-ascii?Q?gXKXsiUYCINDYK21Esa+lbkNgWX2k2dovh9e15Tcw+YlTrbv6oamyuFOt1oO?=
 =?us-ascii?Q?Z3A5CUsVshnUKzQc+zYFVOSCD15L57E9RvtW35hhZTqXSgizVfLyVO6qBeBV?=
 =?us-ascii?Q?daQauYhBEcPvmetHI99O6hCA5/rwShWqYzIIk4YfzBw+3IeCeF6+6kFAIMNq?=
 =?us-ascii?Q?p1MOlsllQUWsF3xq4EPXhCsoBm8SSA/HLFUXQrFng0BHlb4C5P8uPaAsmCg6?=
 =?us-ascii?Q?sTbnLPqhUQGVfUv0BHzfFdjYenbtbrMgHZrRyAgyPHA/XCxEEClVrzXKX3Lf?=
 =?us-ascii?Q?Y7gc90G0CXbFgYAuoRij/RMRjKlbJNapxpGhGUwvxkRVggrbM06TIu5f/rtc?=
 =?us-ascii?Q?glQJZlnTvAI5JE8K2fy0oKIzwb17dNitOBcWVuZTKwwoQU2wg+X4Vp9iTYVY?=
 =?us-ascii?Q?EysTTnNsnZM3uN6G61zdSQ6iXzqvreW/1wmGd6FeQLKvRxxIZK8qhItlqXpN?=
 =?us-ascii?Q?CgrU2/WbDYh3XE8M+RRbtfS0F8/Al8rEaZCVGChfvuhVnRbV8mwo9pJPBgsT?=
 =?us-ascii?Q?CxjyhgRtYNz3QaXDD7DR9n40VpNovGu67fssyXjSnuofKETWiBDBmdlfs9De?=
 =?us-ascii?Q?zG3r6mxVzy6iCsnVT/OLJmESXZykLHo8F2QGHAexFA6njIeNriKv7ou0faCZ?=
 =?us-ascii?Q?lhFRjYP2+aNX2OOO8a2qLED7W87qHBf9ZP8GJZ7LuZxV2tA/EXrLSCXtCu/B?=
 =?us-ascii?Q?rC4jCw6mH9RnkMQ9Ls/3UTTFb1jU6mO6M2NmCDq4RLV/ZcP2tTHjL59frp5A?=
 =?us-ascii?Q?/Ly1zY7KE6311RKtXjuqMt5ydTVUFyvrLs6sNT8X0KTmLjA7dE/Eut99Yi11?=
 =?us-ascii?Q?RCA1C6MqNZcpGfNza2IiqdORKPFfxBagpIVUrm1QPCuQnKvYMB7LHrZEYgNI?=
 =?us-ascii?Q?RSUfp9Jfv5fJ88NKkB8VmzjdrsPRu9AP8tBX/+9UqZ14GOmRbmYiVuYAfuPM?=
 =?us-ascii?Q?GGWDzT014xZxg8IF3J1YQ7S1NXPGWRXmM1LxEXm9y+K9HVszvttRkPPnUThP?=
 =?us-ascii?Q?1yfEjkJADdaTw00DL1wIQW3GJAkXcm/vYM7XIuIgaEXmTUf2e4dPdzGQK0dj?=
 =?us-ascii?Q?wxZy9t1s2PJCIQs9zm8gk+6RsyiPkmtNXoVC4OlRAFW/FGw11eoS0Os4WOZg?=
 =?us-ascii?Q?0BvEjoRmLg=3D=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 73d153d8-e246-41f9-25a6-08da33c32c47
X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 12 May 2022 02:57:38.4908
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6qDQ2HV38/aF3MMNFuDDqDNoSUEBtv9J2drnDa+Z7zhbMM1neZw5lh1hApa91kri7xq57Kr2HVfQCoBj7Vsr5A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB3836
X-Spam-Status: No, score=-1.7 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-ext4.vger.kernel.org>
X-Mailing-List: linux-ext4@vger.kernel.org


"Sierra Guiza, Alejandro (Alex)" <Alex.Sierra@amd.com> writes:

> @apopple@nvidia.com Could you please check this patch? It's somehow related to migrate_device_page() for long term device coherent pages.
>
> Regards,
> Alex Sierra
>> -----Original Message-----
>> From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> On Behalf Of Alex
>> Sierra
>> Sent: Thursday, May 5, 2022 4:34 PM
>> To: jgg@nvidia.com
>> Cc: rcampbell@nvidia.com; willy@infradead.org; david@redhat.com;
>> Kuehling, Felix <Felix.Kuehling@amd.com>; apopple@nvidia.com; amd-
>> gfx@lists.freedesktop.org; linux-xfs@vger.kernel.org; linux-mm@kvack.org;
>> jglisse@redhat.com; dri-devel@lists.freedesktop.org; akpm@linux-
>> foundation.org; linux-ext4@vger.kernel.org; hch@lst.de
>> Subject: [PATCH v1 04/15] mm: add device coherent checker to remove
>> migration pte
>>
>> During remove_migration_pte(), entries for device coherent type pages that
>> were not created through special migration ptes, ignore _PAGE_RW flag. This
>> path can be found at migrate_device_page(), where valid vma is not
>> required. In this case, migrate_vma_collect_pmd() is not called and special
>> migration ptes are not set.

It's true that we don't call migrate_vma_collect_pmd() for
migrate_device_page(), but this doesn't imply migration entries are not
created. We still call migrate_vma_unmap() which calls try_to_migrate()
to install migration entries.

When we have a vma migrate_vma_collect_pmd() is a fast path for the
common case a page is only mapped once. So migrate_vma_collect_pmd()
should fairly closely match try_to_migrate_one(). I did experiment
locally with removing the fast path to simplify the code, but it does
provide a meaningful performance improvement so I abandoned it.

I think you're running into the problem addressed by
https://lkml.kernel.org/r/20211018045247.3128058-1-apopple@nvidia.com
but for DEVICE_COHERENT pages.

Based on that I think the approach below is wrong. You should update
try_to_migrate_one() to deal with DEVICE_COHERENT pages. It would make
sense to do that as part of patch 1 in this series.

The problem is that try_to_migrate_one() assumes folio_is_zone_device()
implies it is a DEVICE_PRIVATE page due to the check in
try_to_migrate().

>> Signed-off-by: Alex Sierra <alex.sierra@amd.com>
>> ---
>>  mm/migrate.c | 3 ++-
>>  1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/mm/migrate.c b/mm/migrate.c index
>> 6c31ee1e1c9b..e18ddee56f37 100644
>> --- a/mm/migrate.c
>> +++ b/mm/migrate.c
>> @@ -206,7 +206,8 @@ static bool remove_migration_pte(struct folio *folio,
>>  		 * Recheck VMA as permissions can change since migration
>> started
>>  		 */
>>  		entry = pte_to_swp_entry(*pvmw.pte);
>> -		if (is_writable_migration_entry(entry))
>> +		if (is_writable_migration_entry(entry) ||
>> +		    is_device_coherent_page(pfn_to_page(pvmw.pfn)))
>>  			pte = maybe_mkwrite(pte, vma);
>>  		else if (pte_swp_uffd_wp(*pvmw.pte))
>>  			pte = pte_mkuffd_wp(pte);
>> --
>> 2.32.0