Subject: Re: [PATCH v2 03/11] mm/gup: migrate PIN_LONGTERM dev coherent pages
 to system
To:     Alistair Popple <apopple@nvidia.com>, akpm@linux-foundation.org,
        linux-mm@kvack.org, rcampbell@nvidia.com,
        linux-ext4@vger.kernel.org, linux-xfs@vger.kernel.org,
        Alex Sierra <alex.sierra@amd.com>
Cc:     amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org,
        hch@lst.de, jgg@nvidia.com, jglisse@redhat.com, willy@infradead.org
References: <20211206185251.20646-1-alex.sierra@amd.com>
 <20211206185251.20646-4-alex.sierra@amd.com> <2858338.J0npWUQLIM@nvdebian>
From:   Felix Kuehling <felix.kuehling@amd.com>
Message-ID: <9fa224e6-d51a-d3cc-575b-3d56517a331a@amd.com>
Date:   Wed, 8 Dec 2021 11:58:27 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <2858338.J0npWUQLIM@nvdebian>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Content-Language: en-US
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?amlFejJtMjhYa2l0Z3owbm9rU0RIN1lUcnR3UnZ1NEtKNkE0VlVLeldWMnRs?=
 =?utf-8?B?TzVJVlo2d25xcXdFZXN0TDhMZm92azFKdVY4RXh3SlhlMGVqRjFOc2V4NFBG?=
 =?utf-8?B?NEszTUU2a0Q4a2tMRi9pbnJVVU1WZzh5SGhDakZFVEJkMUhVU0lDbTRCMEFx?=
 =?utf-8?B?VU1TaFZydkEwVWFLRkZ5WFZvTDJTUmhzVzZOZ2VKV2UxREpxNllCajBnYW5T?=
 =?utf-8?B?b01xQkFUbTlaaGFaZnhPRjRTQWFmeUh5Ry9MV0xLK1krVEtxdGVGM1NVUHFx?=
 =?utf-8?B?enl0cGU3QWVXakoxa3NhSFVBZzd1cUpwNXFlbzloUk0rZmY4MEhOOGRvQmNy?=
 =?utf-8?B?UGNZNStabm1jZ3J2Qm9BL3RJN3hTY3h2aHNTU3FQbjJuVHMxUC9MY0tEQjJl?=
 =?utf-8?B?bUgybWVoMWZpVDFuVUJkeGdyMzFDZHNENzNsL2g0cG00Mmx2a2dQa3NVdk0r?=
 =?utf-8?B?algwK3VpZ2NRVGNQU2ZIc0lJYUQ4aWlYdTZVdUFVK0xtbXdTUEJ0WVlXZnZs?=
 =?utf-8?B?akJ5TnJPSEZBeWtpck82RzhiVTF4clFBNHVTajZJUjd5T1U0VU1udUQzSlg4?=
 =?utf-8?B?Tm8zcy9nY0JXNUdGVUovQWJtU0p4SHVaM1k2L0g0WUdaV0YvM2JwcnBGa0J5?=
 =?utf-8?B?NFpTMURJaWZVTFlTa25IZW13ZGtodzVJSGw3Qi9PYnFOTnlXMFhCenREN1NF?=
 =?utf-8?B?VGJiR1RkWExiTmIvUjdSRXRkZE5lYldpV1pXUW5MbFRkb2pMeUJiNUI3T0pR?=
 =?utf-8?B?VW50ellKaU9IeUZUL3VxU1NEaUpaN29DNGNPVXordU9IZmtFaTM0UktIOElB?=
 =?utf-8?B?U0ZqODlNdU1GNWdYNk9zdzNQNXB2OVErZEdoaWlvM3BuTmlZSjcxZGlkdk1r?=
 =?utf-8?B?aXVlUjhFeVlQNkVzckFKWTZGM2pPQUZsa3NHM05EQTNoNEdtS1U0cTE0ZElq?=
 =?utf-8?B?cDRFR0sxMkNOWnM4QWx2Q2NUZHMvUHF3ZVlyQWpJRGRxcHFhVlIxaWVTZlZq?=
 =?utf-8?B?TDFCaFpKQXNxMWRpeFhMRldUdXkxRkRMT0lEWHpkdERUbTFJNGpRZjJGZW9C?=
 =?utf-8?B?ZTA0Y3VEeDVqaGFuNHlGeU1ZR2NIdFdvVnpXOUZBNHFvQXYyZnVXYVZsQldw?=
 =?utf-8?B?VFR2VmR0WmtqQytzMzZhK3l2UlpMZ1F0OTFnK2JxQkF1eFpzU0g5SFFuVisz?=
 =?utf-8?B?dnZlRHZlamJkMFdCRXQ0azFDbGVPRE9yNkx0NjhNSmVjUCtzWUtNRVNacFBE?=
 =?utf-8?B?RHA3bW5lblhOWlptWTdXRzRWeUlvR1NvMmk5Z29vaXl4VDJPR3l6WUxBNzRt?=
 =?utf-8?B?WFlvaHhwc2pwTk81ZXRwbkdPMjBKZFlkN0xYNE1oblk0d3NHdVBOYkJ5Q05m?=
 =?utf-8?B?ay9rK0d2OFdiVjdvZVBXbGw1MlZMU2llTlQxSlJPcUYrVE1vcEVIb2didXQ3?=
 =?utf-8?B?dlV0eWdBTUM3bWo2eStvZGZ2ek5Va3FndTNRWHVVY2taYUtHYjduL1VZdVgw?=
 =?utf-8?B?bnB2ZStDVWNhOVRoRTFVK2FuSlBYZ0ROSDJPZDVIb2xJN2o3WEtFZW9ZNmdx?=
 =?utf-8?B?alBVbFNvTmJQeUdsc1Y4dVIwdTVrQVV6M2hpUFpWeXNwa3NmQythc3d0eE9l?=
 =?utf-8?B?WGFyY24wanQ1b29rUGtwdmVKN1BWS2xBYjRKUVRvbi90ZXdrVlRMejZYSitq?=
 =?utf-8?B?ZjhkU3RFRUUvbU5GNlVOTDQwZ3V4M3ZOTGRUdC9XZHlkSVNrYjZTUkJkZEVG?=
 =?utf-8?B?VlNmMUFjbUFueDIwODJ3ZEh5VCtudmpiQjZvNHZvR1plKzhVTVdRSnlLbmxJ?=
 =?utf-8?B?TUpYM3J4MmVlY0J4ZEVXV05jNjR5c0RpejdNd0hQWCtIbU05Um9SWU1Pekhw?=
 =?utf-8?B?azNSS1dmeFVLWGxlSWVrSnNEeE5nUERxcXhqc21vK0FFZjcvdGlkQTBZZFV6?=
 =?utf-8?B?NTg5WWk5YVQyUzExMTlFcjR6U1I0NGRRLzN5eUpjNW4xcjA4a1hnRUdFTk5q?=
 =?utf-8?B?VkFQb3p3eFpnZWkxWU9kRkxSaE5SUjJkUkYzVjZuNy9KNkJNVU1Ub1BxcFFF?=
 =?utf-8?B?Zmd2K21oNjQ2Tk1PbnBXMlhYcnpYNjZUbG81ODJ1dWY0ZUFmS2NpcUo5V0gv?=
 =?utf-8?B?b0xBbXZTSHRmdlpBcm9adkY3REdUZ2xnR0l5a1VTemFUVmRFclpMOFpMT2VH?=
 =?utf-8?Q?NL34BxvKKp1YwDmIdJ0zQIU=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 13dc960f-a16f-4f07-522a-08d9ba6bf55f
X-MS-Exchange-CrossTenant-AuthSource: BN9PR12MB5115.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Dec 2021 16:58:29.2011
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: YjVk046th977F+xJFWNf8HmCXLMCGXA97oLE95R3umUK7vG80ySQ4ZJP7p/0yw3rZpjoIokgwZPN0WYjL9lAeA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5308
Precedence: bulk
List-ID: <linux-ext4.vger.kernel.org>
X-Mailing-List: linux-ext4@vger.kernel.org

Am 2021-12-08 um 6:31 a.m. schrieb Alistair Popple:
> On Tuesday, 7 December 2021 5:52:43 AM AEDT Alex Sierra wrote:
>> Avoid long term pinning for Coherent device type pages. This could
>> interfere with their own device memory manager.
>> If caller tries to get user device coherent pages with PIN_LONGTERM flag
>> set, those pages will be migrated back to system memory.
>>
>> Signed-off-by: Alex Sierra <alex.sierra@amd.com>
>> ---
>>  mm/gup.c | 32 ++++++++++++++++++++++++++++++--
>>  1 file changed, 30 insertions(+), 2 deletions(-)
>>
>> diff --git a/mm/gup.c b/mm/gup.c
>> index 886d6148d3d0..1572eacf07f4 100644
>> --- a/mm/gup.c
>> +++ b/mm/gup.c
>> @@ -1689,17 +1689,37 @@ struct page *get_dump_page(unsigned long addr)
>>  #endif /* CONFIG_ELF_CORE */
>>  
>>  #ifdef CONFIG_MIGRATION
>> +static int migrate_device_page(unsigned long address,
>> +				struct page *page)
>> +{
>> +	struct vm_area_struct *vma = find_vma(current->mm, address);
>> +	struct vm_fault vmf = {
>> +		.vma = vma,
>> +		.address = address & PAGE_MASK,
>> +		.flags = FAULT_FLAG_USER,
>> +		.pgoff = linear_page_index(vma, address),
>> +		.gfp_mask = GFP_KERNEL,
>> +		.page = page,
>> +	};
>> +	if (page->pgmap && page->pgmap->ops->migrate_to_ram)
>> +		return page->pgmap->ops->migrate_to_ram(&vmf);
> How does this synchronise against pgmap being released? As I understand things
> at this point we're not holding a reference on either the page or pgmap, so
> the page and therefore the pgmap may have been freed.
>
> I think a similar problem exists for device private fault handling as well and
> it has been on my list of things to fix for a while. I think the solution is to
> call try_get_page(), except it doesn't work with device pages due to the whole
> refcount thing. That issue is blocking a fair bit of work now so I've started
> looking into it.

At least the page should have been pinned by the __get_user_pages_locked
call in __gup_longterm_locked. That refcount is dropped in
check_and_migrate_movable_pages when it returns 0 or an error.


>
>> +
>> +	return -EBUSY;
>> +}
>> +
>>  /*
>>   * Check whether all pages are pinnable, if so return number of pages.  If some
>>   * pages are not pinnable, migrate them, and unpin all pages. Return zero if
>>   * pages were migrated, or if some pages were not successfully isolated.
>>   * Return negative error if migration fails.
>>   */
>> -static long check_and_migrate_movable_pages(unsigned long nr_pages,
>> +static long check_and_migrate_movable_pages(unsigned long start,
>> +					    unsigned long nr_pages,
>>  					    struct page **pages,
>>  					    unsigned int gup_flags)
>>  {
>>  	unsigned long i;
>> +	unsigned long page_index;
>>  	unsigned long isolation_error_count = 0;
>>  	bool drain_allow = true;
>>  	LIST_HEAD(movable_page_list);
>> @@ -1720,6 +1740,10 @@ static long check_and_migrate_movable_pages(unsigned long nr_pages,
>>  		 * If we get a movable page, since we are going to be pinning
>>  		 * these entries, try to move them out if possible.
>>  		 */
>> +		if (is_device_page(head)) {
>> +			page_index = i;
>> +			goto unpin_pages;
>> +		}
>>  		if (!is_pinnable_page(head)) {
>>  			if (PageHuge(head)) {
>>  				if (!isolate_huge_page(head, &movable_page_list))
>> @@ -1750,12 +1774,16 @@ static long check_and_migrate_movable_pages(unsigned long nr_pages,
>>  	if (list_empty(&movable_page_list) && !isolation_error_count)
>>  		return nr_pages;
>>  
>> +unpin_pages:
>>  	if (gup_flags & FOLL_PIN) {
>>  		unpin_user_pages(pages, nr_pages);
>>  	} else {
>>  		for (i = 0; i < nr_pages; i++)
>>  			put_page(pages[i]);
>>  	}
>> +	if (is_device_page(head))
>> +		return migrate_device_page(start + page_index * PAGE_SIZE, head);
> This isn't very optimal - if a range contains more than one device page (which
> seems likely) we will have to go around the whole gup/check_and_migrate loop
> once for each device page which seems unnecessary. You should be able to either
> build a list or migrate them as you go through the loop. I'm also currently
> looking into how to extend migrate_pages() to support device pages which might
> be useful here too.

We have to do it this way because page->pgmap->ops->migrate_to_ram can
migrate multiple pages per "CPU page fault" to amortize the cost of
migration. The AMD driver typically migrates 2MB at a time. Calling
page->pgmap->ops->migrate_to_ram for each page would probably be even
less optimal.

Regards,
  Felix


>
>> +
>>  	if (!list_empty(&movable_page_list)) {
>>  		ret = migrate_pages(&movable_page_list, alloc_migration_target,
>>  				    NULL, (unsigned long)&mtc, MIGRATE_SYNC,
>> @@ -1798,7 +1826,7 @@ static long __gup_longterm_locked(struct mm_struct *mm,
>>  					     NULL, gup_flags);
>>  		if (rc <= 0)
>>  			break;
>> -		rc = check_and_migrate_movable_pages(rc, pages, gup_flags);
>> +		rc = check_and_migrate_movable_pages(start, rc, pages, gup_flags);
>>  	} while (!rc);
>>  	memalloc_pin_restore(flags);
>>  
>>
>
>