Received-SPF: pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
References: <20220629035426.20013-1-alex.sierra@amd.com>
 <20220629035426.20013-5-alex.sierra@amd.com>
 <956b1c51-b8f1-0480-81ca-5d03b45110f7@redhat.com>
User-agent: mu4e 1.6.9; emacs 27.1
From:   Alistair Popple <apopple@nvidia.com>
To:     David Hildenbrand <david@redhat.com>
Cc:     Alex Sierra <alex.sierra@amd.com>, jgg@nvidia.com,
        Felix.Kuehling@amd.com, linux-mm@kvack.org, rcampbell@nvidia.com,
        linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org,
        amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org,
        hch@lst.de, jglisse@redhat.com, willy@infradead.org,
        akpm@linux-foundation.org
Subject: Re: [PATCH v7 04/14] mm: add device coherent vma selection for
 memory migration
Date:   Thu, 30 Jun 2022 21:44:55 +1000
In-reply-to: <956b1c51-b8f1-0480-81ca-5d03b45110f7@redhat.com>
Message-ID: <878rpe73t3.fsf@nvdebian.thelocal>
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?EL6XHlkOJJ/gAiAnT6V4XbS3cAc9nV1M4KKJlmggBs8nR19K8clQSRAwVFNK?=
 =?us-ascii?Q?n5E9TkGAwJ+d8UPZLFbgDZ+DOavgT89/YkjpXxKx/aBlrdCi1hPGrEWAJJdP?=
 =?us-ascii?Q?lIBcrn0X7rQKj1D6cxZV6HRAJMsZ3eogMfwV51Z0x5pi5MmosoRiLqjmO0Nq?=
 =?us-ascii?Q?HknVe5+Exk/SghHtOuLkr5v42+iYeYUnzDgT0RQBN3N8KzAJMBDii/YoyRnW?=
 =?us-ascii?Q?UFkZfNMdBFAVKglMTogWrMwmi9OEEpVsEzcyLqiE6+1w0QV60EFgvuMP04zH?=
 =?us-ascii?Q?GOyadHqv7WBwHzbn4tb7qfM4tihHX1LO6Kss5+4UKCtmAhuKwE/h+q9LcQxi?=
 =?us-ascii?Q?rcv9YjAJ0gQxg7NASPeHm08UsuQG19nvlkt/hwEezLF+5iOcW52hCjvTZoKs?=
 =?us-ascii?Q?ls7M1iN+CPhwZB75DxqtHTi/8uK52T+hdksj0HFcCDLheHE8B5IqR+SZ1eei?=
 =?us-ascii?Q?GojaeRa3+e3889A6cEZhq+4Inws5z1llWmsx4eTyyRyuVtmycMBnQlRIJ6AQ?=
 =?us-ascii?Q?bXzB9ctBd/PTcvysocvGq5yR50MidHVYSPSVabT4cvVuXiU8O4jOOdskItwg?=
 =?us-ascii?Q?B/u7XP0XZiJYsNXroiHMqKbvBRu2S95aFjLgfKugYk6NLXvyt9m09t22KqsN?=
 =?us-ascii?Q?TLeid23OyZYdNnFVAJXylm0+0ygbfrChXvq3vC35XOp1naHu//erDMZXVhR7?=
 =?us-ascii?Q?dCiBz1DrvaTsR18fPRhQbF6Kcceg2VFWSEfplOgJ/hGdyM9zdWQDuweBfu38?=
 =?us-ascii?Q?kpmzFAkT02/CZTmY+636mDoJQDe+kElKMjpcPS/LX48UaTNSxPN5jTlrT9EU?=
 =?us-ascii?Q?TgJU+XSg75FBELfgjQ91NBM/UyokDvtWqmTnmMumhJmDuC3VQE0/XtcVNf6X?=
 =?us-ascii?Q?nhcX3J7/bCYEw12oT/8JbXxQakdlXQn8t0OMXbvr71qUNTX+UsqmdY/zsRey?=
 =?us-ascii?Q?h7i0ydhWKvVKoANEkWj9fA40wAouAlyMtXR8nBOvOqoWlXu8M8abE+WG0pGK?=
 =?us-ascii?Q?4FmUO1U5zin6cgNaNWlAQoM345bh6T84WF0nBrGeDQjgB7eExvKD7cWYiv1O?=
 =?us-ascii?Q?g+xAekwsFrlqye6C/iiM8OvaIWxdEM19P3resqvNB8BR2U+wl6ylubPSeIF+?=
 =?us-ascii?Q?Wy6SXUZkD6xjP+t+slBYO7HvX2SdB2Znj6k9/7U0l/yKggTKKGndLnxZhNAc?=
 =?us-ascii?Q?ocwyIEWzrAAk8CqSI9b43s80MPka1v/qJm6XmWPYbZJ6OABvBLojhEmU42/D?=
 =?us-ascii?Q?cYAJ1iAuC17ZBHH04SF+J8Yf0e2rhJ3RU+7XNJydofY4XJjQUa0wFyS4pEY8?=
 =?us-ascii?Q?BOIQlwjX/zql+ojyt1Dsl9dA9/qBFuayva0Q9XaIWHWluWN1yire6Q222ENm?=
 =?us-ascii?Q?r1dBgs9BUyfbrCHq69nHx5iYguSVPyLKwgHyGSdV+ve3eIgqZo7xGTyYk70V?=
 =?us-ascii?Q?vEzq0zS0LgDiFlYQTPmbivOwXf2ICHR0sdjQEEzjXNXRqrIFcN6MuwOdDBF8?=
 =?us-ascii?Q?QXI18XkiMc/WVDbi0fMfWNOIPpKXkutNmv7NG4kxFqF0Un4S9VQpqsaQNyvb?=
 =?us-ascii?Q?ZwPz1anZR6ctAakYPeXjVdVglrfpHUJq+91zqrzn?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 06300d03-0f57-4553-8bed-08da5a9116f4
X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB3176.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 30 Jun 2022 12:07:23.0104
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: W1xct5Q0oMUHf52n9WJcAjH6YOzW2FFPdJ7PSbj9WcpVygHkEZ1u+K5r32w5sEv4Q+k41c9vzjH1djtuv6fS+w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4838
X-Spam-Status: No, score=-1.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-ext4.vger.kernel.org>
X-Mailing-List: linux-ext4@vger.kernel.org


David Hildenbrand <david@redhat.com> writes:

> On 29.06.22 05:54, Alex Sierra wrote:
>> This case is used to migrate pages from device memory, back to system
>> memory. Device coherent type memory is cache coherent from device and CPU
>> point of view.
>>
>> Signed-off-by: Alex Sierra <alex.sierra@amd.com>
>> Acked-by: Felix Kuehling <Felix.Kuehling@amd.com>
>> Reviewed-by: Alistair Poppple <apopple@nvidia.com>
>> Signed-off-by: Christoph Hellwig <hch@lst.de>
>
>
> I'm not too familiar with this code, please excuse my naive questions:
>
>> @@ -148,15 +148,21 @@ static int migrate_vma_collect_pmd(pmd_t *pmdp,
>>  			if (is_writable_device_private_entry(entry))
>>  				mpfn |= MIGRATE_PFN_WRITE;
>>  		} else {
>> -			if (!(migrate->flags & MIGRATE_VMA_SELECT_SYSTEM))
>> -				goto next;
>
> Why not exclude MIGRATE_VMA_SELECT_DEVICE_PRIVATE here? IIRC that would
> have happened before this change.

I might be missing something as I don't quite follow - this path is for
normal system pages so we only want to skip selecting them if
MIGRATE_VMA_SELECT_SYSTEM or MIGRATE_VMA_SELECT_DEVICE_COHERENT aren't
set.

Note that MIGRATE_VMA_SELECT_DEVICE_PRIVATE doesn't apply here because
we already know it's not a device private page by virtue of
pte_present(pte) == True.

>>  			pfn = pte_pfn(pte);
>> -			if (is_zero_pfn(pfn)) {
>> +			if (is_zero_pfn(pfn) &&
>> +			    (migrate->flags & MIGRATE_VMA_SELECT_SYSTEM)) {
>>  				mpfn = MIGRATE_PFN_MIGRATE;
>>  				migrate->cpages++;
>>  				goto next;
>>  			}
>>  			page = vm_normal_page(migrate->vma, addr, pte);
>> +			if (page && !is_zone_device_page(page) &&
>
> I'm wondering if that check logically belongs into patch #2.

I don't think so as it would break functionality until the below
conditionals are added - we explicitly don't want to skip
is_zone_device_page(page) == False here because that is the pages we are
trying to select.

You could add in this:

>> +			    !(migrate->flags & MIGRATE_VMA_SELECT_SYSTEM))

But then in patch 2 we know this can never be true because we've already
checked for !MIGRATE_VMA_SELECT_SYSTEM there.

>> +				goto next;
>> +			else if (page && is_device_coherent_page(page) &&
>> +			    (!(migrate->flags & MIGRATE_VMA_SELECT_DEVICE_COHERENT) ||
>> +			     page->pgmap->owner != migrate->pgmap_owner))
>
>
> In general LGTM