Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <6ff9dcc5-c34b-963f-f5e7-7038eecae98b@nvidia.com>
Date:   Mon, 26 Sep 2022 15:14:09 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.3.0
Subject: Re: [PATCH 6/7] nouveau/dmem: Evict device private memory during
 release
Content-Language: en-US
To:     Lyude Paul <lyude@redhat.com>,
        Alistair Popple <apopple@nvidia.com>, linux-mm@kvack.org,
        Andrew Morton <akpm@linux-foundation.org>
Cc:     Michael Ellerman <mpe@ellerman.id.au>,
        Nicholas Piggin <npiggin@gmail.com>,
        Felix Kuehling <Felix.Kuehling@amd.com>,
        Alex Deucher <alexander.deucher@amd.com>,
        =?UTF-8?Q?Christian_K=c3=b6nig?= <christian.koenig@amd.com>,
        "Pan, Xinhui" <Xinhui.Pan@amd.com>,
        David Airlie <airlied@linux.ie>,
        Daniel Vetter <daniel@ffwll.ch>,
        Ben Skeggs <bskeggs@redhat.com>,
        Karol Herbst <kherbst@redhat.com>,
        Ralph Campbell <rcampbell@nvidia.com>,
        "Matthew Wilcox (Oracle)" <willy@infradead.org>,
        Alex Sierra <alex.sierra@amd.com>,
        linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org,
        amd-gfx@lists.freedesktop.org, nouveau@lists.freedesktop.org,
        dri-devel@lists.freedesktop.org, Jason Gunthorpe <jgg@nvidia.com>,
        Dan Williams <dan.j.williams@intel.com>
References: <cover.f15b25597fc3afd45b144df863eeca3b2c13f9f4.1664171943.git-series.apopple@nvidia.com>
 <072e1ce590fe101a4cdbd5e91b1702efebb6d0fd.1664171943.git-series.apopple@nvidia.com>
 <881735bda9b1ba0ecf3648af201840233508f206.camel@redhat.com>
From:   John Hubbard <jhubbard@nvidia.com>
In-Reply-To: <881735bda9b1ba0ecf3648af201840233508f206.camel@redhat.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cGNJNVh5c2tXOWxzUXEwNUNGVldVblZCbHRTRUlJSWl0dEJkajY2RXI4TVZh?=
 =?utf-8?B?ZnFyTWgrOVowVmhiekN0UE5sQkplMmx2U2dkYjNqNTVCRUJmMllVcDQ3NkNj?=
 =?utf-8?B?L1VtcC8zRWJHWDIyTFU0U2gvekN6dCthQkVjUjhtYVhBaEhCUGxiVFJhSENm?=
 =?utf-8?B?N0NPaUUzSXFJdWpXSEMzRS9scVA3eXBMM3Y4K1hzMDBTOFlNQ2dHNTlJWWJp?=
 =?utf-8?B?ekxnUnZrZnpFRmtVMlNoVUNyMXRKQ29xb01IYXlYRkNGL2tZTXZKaCtBR2kx?=
 =?utf-8?B?NmNST1RoZjd4Wk42cGlzbW1wOExPQjBCcEVvb0Y5MHpLcWRVOGZYWVV3ZTBQ?=
 =?utf-8?B?cDNRbGQ1MDNvRUMvdGozL2lxU1FRV1VYOElZSzE4ek5idmpzR2E0MXd6Y1Ux?=
 =?utf-8?B?UjNoamlwY0R6c1YzQW00VjVoR0kwS0w2aGJZS1RJOHZaOWdiZVpxZ0U3U3dx?=
 =?utf-8?B?RXV2VVJRMEtVODRnYTBhS2JLVXVVTnpsZjNZaHYxWHI5b3c5Zm1FUGxsVjJm?=
 =?utf-8?B?Ymlqa29FZkppbWdKdTB1UXdvR0g1dWxld0xrcTF4Uk9PRUUyKy91ZWRzVldI?=
 =?utf-8?B?MzBKcnVXZE5ENFV3WE45WHY0TzNHTXRvK2k5RDF2Wm9zRlFqK1BCZWlZUTF3?=
 =?utf-8?B?ajBLdE5BRjA3OW83ZjlWTnNHTXk5VWNNVG5LSTd6dmFrWFYvVEw3MWFkNjRG?=
 =?utf-8?B?K2krTFpHWW5hZ3diY1l3V2tjWVppc0h4MllYa0dJemNNVXd0TUlZVUtEbUhZ?=
 =?utf-8?B?S2J6UG1ZUU5QWTB4Y2Vrd2hxdVlUV1dXNGloK2RxV3llVkJ6cU00RHVKNkdY?=
 =?utf-8?B?OVdpUm02Rjh5YzJoQVJDeXVhTTRKNDVpL2hDUXVBSTNTUnl0UzVxUU5kRDFi?=
 =?utf-8?B?RG1TTVFpdkNUQjRHQ1QvQzF1TjBYb0N2UXZNV2ovZ0pOTzFlbWljazRqaHhK?=
 =?utf-8?B?MmRuMHhDZjVFVHE1S1JUT3g4b2hQVzRyVDNJRFhCVDhlSTJKTkRIcmJLVjhB?=
 =?utf-8?B?UldhNXp1UFFyaGxMZk0vMW9nekNnTU1VZ0RlQVJVUGVFTHJXbHpNQ3dMMEFX?=
 =?utf-8?B?TW5CeE9oRnFZNWc1K2FRbWhWazJuckxwSFdlVmtHNzFXbHdyTk0zSFUyN0Fh?=
 =?utf-8?B?UldMdkhuRFpNRUdxbkVmVjZTZHQxNWNMcjBDd2MyaVhUOVM0d1JDeDJLVkVQ?=
 =?utf-8?B?c1ZRR1MzZDBOM29FeW1oRXhrYzBTNmQ0TmhJczh5RC9TTkgxR05sVWhuYW9j?=
 =?utf-8?B?cnZ6c204ZyszaVhmNGlaSGlsVU5BS1pYa0dhUkY2Q3FDQXpDR1AvcG05RlZD?=
 =?utf-8?B?a3M0NDMrbUdkYVJ4TW14YTBHbUNma1MxbThGT3Q5eUJZWlNRK1NTeHpsclVX?=
 =?utf-8?B?OHRDVEdvdjlRblBrMmVTNVovOGIyS2pzY0F0Q2JFWXpzRDk5TUQ0enc0OUd6?=
 =?utf-8?B?bGt6RlZSU0IrVTMyb0lvU1ArV3c0bzZLOVRRVHNaaUNXQitta0d0OEwzT0Za?=
 =?utf-8?B?RnJFN2FhUHVhdHRSM2FzOXlXaTBEdTJCYVpPOHVtOFpLc2lPWHFBNnFYZ0pS?=
 =?utf-8?B?YytKNUo2dTJqY3FuVk9TM0g0NG9QREM3U3RGV0o4MUlaZG1TcHhqTWRmRmp5?=
 =?utf-8?B?RVQ5SzlyaUxPTlRqWnRIYmVhdVI1d0tPWmEwMWNMV1k1MXQrV0ZGaXE4dEtY?=
 =?utf-8?B?UG9QdmFIOXUxUUgraXZ1eHpCVVVjck5NUW5LdjNmOTllVnFnTkIrTGV5Lzgx?=
 =?utf-8?B?RU5qQjBKUDF6U2pBWFJSYzA1MWk2U3VJNjkrL1BsSGtPQWtibE5JYUtLKzZo?=
 =?utf-8?B?Q0oyR3NiVThxLzhYcTdic1R6ZGoxRWZWcTM2cmx2U2JqK3B1S01HZU9kRnUy?=
 =?utf-8?B?Wm51NWdCYjdaR2h0TGREK0lwVVdaQmVtamlsOGJCR2xiZjEyNkNtelU5ZkE5?=
 =?utf-8?B?dHIzazd6Q3l3bzhNbmptM0RXdktWaUxVSjY3L2tpNVYwQkdreGczOUJIcUFo?=
 =?utf-8?B?K0ZZUmVnSjFpU2k0eFdMVnd3VXVIemgyNTY0cTNQZ09SRTVJUWhUSzFzZ2tF?=
 =?utf-8?B?dU1BR25nUHo2ZWRYWlBXY1dPK3ptM1RjcjNvZWo5cHVsWkJWYUZiOXZvTGJp?=
 =?utf-8?Q?e5LXjD0x1ZheO89uOIrgPG98K?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ed5d9840-5741-4d29-e200-08daa00c716f
X-MS-Exchange-CrossTenant-AuthSource: BY5PR12MB4130.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Sep 2022 22:14:13.1430
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3HDQQNZg2glLVeW+9HZ7gEyMW2/HvB9UejVgflsOwiimpDsVxpXp94mlS1oTzcuvqAeHONVaUbrsEGSL8qt3Ug==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN9PR12MB5383
X-Spam-Status: No, score=-3.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,
        NICE_REPLY_A,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,
        SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 9/26/22 14:35, Lyude Paul wrote:
>> +	for (i = 0; i < npages; i++) {
>> +		if (src_pfns[i] & MIGRATE_PFN_MIGRATE) {
>> +			struct page *dpage;
>> +
>> +			/*
>> +			 * _GFP_NOFAIL because the GPU is going away and there
>> +			 * is nothing sensible we can do if we can't copy the
>> +			 * data back.
>> +			 */
> 
> You'll have to excuse me for a moment since this area of nouveau isn't one of
> my strongpoints, but are we sure about this? IIRC __GFP_NOFAIL means infinite
> retry, in the case of a GPU hotplug event I would assume we would rather just
> stop trying to migrate things to the GPU and just drop the data instead of
> hanging on infinite retries.
> 
Hi Lyude!

Actually, I really think it's better in this case to keep trying
(presumably not necessarily infinitely, but only until memory becomes
available), rather than failing out and corrupting data.

That's because I'm not sure it's completely clear that this memory is
discardable. And at some point, we're going to make this all work with
file-backed memory, which will *definitely* not be discardable--I
realize that we're not there yet, of course.

But here, it's reasonable to commit to just retrying indefinitely,
really. Memory should eventually show up. And if it doesn't, then
restarting the machine is better than corrupting data, generally.


thanks,

-- 
John Hubbard
NVIDIA