Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <240650f1-a5cd-8b0c-fe91-0f8b116d6fe9@intel.com>
Date:   Mon, 27 Mar 2023 09:23:04 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.8.0
Subject: Re: [PATCH v4 35/36] mm: Convert do_set_pte() to set_pte_range()
To:     Will Deacon <will@kernel.org>, Matthew Wilcox <willy@infradead.org>
CC:     Ryan Roberts <ryan.roberts@arm.com>, <linux-arch@vger.kernel.org>,
        <linux-mm@kvack.org>, <linux-kernel@vger.kernel.org>
References: <b39f4816-2064-e402-4e02-908f40c396d4@intel.com>
 <2fa5a911-8432-2fce-c6e1-de4e592219d8@arm.com>
 <ZBNXcmOrrOS4Rydg@casper.infradead.org>
 <b2c00aab-82ad-ea7a-df9d-c816b216b0f1@intel.com>
 <ZBPiOgYDLYBmVwOc@casper.infradead.org>
 <12d7564f-5b33-bdcc-1a06-504ad8487aca@intel.com>
 <25bf8e75-cc2e-7d08-dbba-41c53ab751b0@arm.com>
 <d2e90338-6200-f005-110d-4626fda067a2@intel.com>
 <20230324145828.GB27199@willie-the-truck>
 <ZB29hND1tt37dNUX@casper.infradead.org>
 <20230324172319.GA27842@willie-the-truck>
Content-Language: en-US
From:   Yin Fengwei <fengwei.yin@intel.com>
In-Reply-To: <20230324172319.GA27842@willie-the-truck>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?OFdkQnBqbG5Sc1NxalQ3Z1YzY0dVYWpieUMyczNOYW5BZFl5WDNkQ1h2WXN4?=
 =?utf-8?B?cHhpaXg2ZXpEM3F2YVljalB5TUNlL1VobWNtREdCUmVkdDNLbGw1UkpYYXBE?=
 =?utf-8?B?RHEyenlQWjBQd2lpazNDZ0E3cWcrQ3ZHay80TVkzeEhzU0hmbzFpZ0t1MFc4?=
 =?utf-8?B?M0lNT0F0blhPaVE2QkEzRndMbnVwS2lkMHRJZnZMVUJ6K0VoSCtTKzRwUjNq?=
 =?utf-8?B?OVFTWWdSZlAxUUxpQnI2b1lnUUFOYTFheHR0TzMwLy9GVHdFWCtwU0Z1NUx4?=
 =?utf-8?B?ckgrb0Jxc2hndjhoRnh0WTdabjZ6YmV5b3J1akVxaW9xTHNvVTRaWndsdDVV?=
 =?utf-8?B?U1FlVDM1M3lHNHR1b1RPSy9wbjJ4bzIwVnI5V2NNYWdjTVFnaVEzTjRDVm0w?=
 =?utf-8?B?ZGV2R1l3RDRiTVc0ei9IeUNTVnNzd1lhUzVvbktYSGVtUnE5M3RJaVNKTGdv?=
 =?utf-8?B?eWF0dERlNllWY0syS085citXSTNPVlN4OGNGbkN4VnkyVUNsVXBvWldNV3VO?=
 =?utf-8?B?c2RGOStWNEIxY0tVN1Jnbzh5SXBDY29NL1UxbWpCN0tGc0ticHZZeko4Lzk3?=
 =?utf-8?B?UTdIeXJLM2FES3MyTXdYL214ZGNLaGRsQ1pjdG5GLzM4TEhNT0p2ODVONk1r?=
 =?utf-8?B?MHVzbS80dHc3Wm11TzV2Vlcwb0FQNFhTM3hkMVh1YWlFNm96TGtXMUR3Z3dJ?=
 =?utf-8?B?V2ozbWRteithRDRudmptUkJoTTMwVStUL1NwSE1EYVJKeENWTm1CVmgwMEtR?=
 =?utf-8?B?bTlOWVg5N0VNV09DczlWK3NxOXN4cFY3VXd1QnRhVXN5TVVMWkdXMFA1Mkpq?=
 =?utf-8?B?RnAyVHpIQVNJMGpxRkx5U1g5OG5wL3RnbFFSdUVNdjFlU2I0TWRkT0Q4eSt3?=
 =?utf-8?B?SjEwTEtLK0RreVNCZ2psL2dGTlF5cmlNMTFPZlVnTG9vU29zWWpOd00zWXh2?=
 =?utf-8?B?eGlDVkpzUUhsR0dnOWpPTUpzTWVnVVBnWHhRS0E4L2k1QkFYdHdodzl4d1dK?=
 =?utf-8?B?Qm13aWdRa2xLQlBhcitnd2J4WEgrQlZHZHdIYlVvaktkUTBDYkFPdkNPTWVj?=
 =?utf-8?B?VHBybFl1cHJ5OFQxU2ZqdlJ0UkJFRFBnRlVFRDJWellKNWhidUkxYUtIdVpa?=
 =?utf-8?B?OUFhVHhVbHl3ekFMSHE5NHRLZ2lDVGtMTDQ0R0I3czZtNEsyaGpLUDNpQ2I2?=
 =?utf-8?B?MjZHd0svbDFKeFhLMS9FTmlJdVBzSkh4SXl2ZTlEUWZ6NmJXcThiV1YxZWtQ?=
 =?utf-8?B?NUd3c1hRL1lpakJiRW1aTFl0VWpncnBPNFNCYjhWeU04T2twenVOODFrZ1Rx?=
 =?utf-8?B?REQzTmh4SFZGemlwTU5icEJnWVAwR0RzK0VkNG92aVdiN1l5NFhTL2E5ODZP?=
 =?utf-8?B?WlNyTVY3eTVoWG92cXc1a05zOWZQZldFTVdNR0I2TTh2Y21kYnFZN3lXSjJD?=
 =?utf-8?B?WlpuRGl0Q21ubW1YYklYdDZTYUUyOHFyWUJuNnNBM1lQeG51RURCNkwvSE81?=
 =?utf-8?B?RmpwNk1QN0dwTVVUNEMvckxvTEY0dzRlcitsRElzS3JzbHplN295enA3bkhE?=
 =?utf-8?B?MkpyV05aV3Vib1lJSTB2MWdhRXFGNThTTldVWE55M0d1QVlLM3M2WE96N3ZL?=
 =?utf-8?B?dXZTZm92eGR0d21pRFR0SmdVQ2JCU0ZVTW5pMWlDU1hyZVZLT2R5d3dkSkpK?=
 =?utf-8?B?R0NteTZkSU1KNEpMMmFaUHJJS2MxNTV5VVZIUDlqZFhFekdTeDZJbVREaWsv?=
 =?utf-8?B?TTdDSmdmektRZ2RoUHRsTFFROTZEc3VzMmpOZ3ZMZWZFenlPeVBjY0prRXdi?=
 =?utf-8?B?SHlMcG9ENDRtZkgzOWt0d3NXa1hiNWViRFNucXJRNXN2NVlXSnpoSXZVRTV2?=
 =?utf-8?B?VlNSSFI3K1FVb01vYlc2bGxvbnhtakFDTDk2c0RwNFFkN3Q5TERMMGpUajkx?=
 =?utf-8?B?c2MxSWlQZS82RnQ4eDlBSm96YlFuclY0WU12NVFDdjh3TmRYUW40ZWQrY00r?=
 =?utf-8?B?aGVSbkVvSFUwK0hxSlNMQ21aazdYWXlJcFJNelRESjczbHRpaDhuN0ZuVlBH?=
 =?utf-8?B?V3VJUTB3ZjBTSDhuVml2ODlxYVFHL0k2Sll4RW4wWGlHV09haHUrME5SVEtm?=
 =?utf-8?Q?cJ0CmkjzYuAjgiFI146xtv6JM?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 4b19ce83-4f1a-479c-dcfe-08db2e62663a
X-MS-Exchange-CrossTenant-AuthSource: CO1PR11MB4820.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 27 Mar 2023 01:27:16.3542
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: VaRjZHL1/pIdftiWgb2daG/FykEMx3ziZnbGje7WUXZTcAFscEIneE89P8CVaidnVxe731NN0GehC0tFdmAvXQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR11MB7216
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_MED,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=unavailable
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 3/25/23 01:23, Will Deacon wrote:
> On Fri, Mar 24, 2023 at 03:11:00PM +0000, Matthew Wilcox wrote:
>> On Fri, Mar 24, 2023 at 02:58:29PM +0000, Will Deacon wrote:
>>> Yes, please don't fault everything in as young as it has caused horrible
>>> vmscan behaviour leading to app-startup slowdown in the past:
>>>
>>> https://lore.kernel.org/all/20210111140149.GB7642@willie-the-truck/
>>>
>>> If we have to use the same value for all the ptes, then just base them
>>> all on arch_wants_old_prefaulted_pte() as iirc hardware AF was pretty
>>> cheap in practice for us.
>>
>> I think that's wrong, because this is a different scenario.
>>
>> Before:
>>
>> We faulted in N single-page folios.  Each page/folio is tracked
>> independently.  That's N entries on whatever LRU list it ends up on.
>> The prefaulted ones _should_ be marked old -- they haven't been
>> accessed; we've just decided to put them in the page tables to
>> speed up faultaround.  The unaccessed pages need to fall off the LRU
>> list as quickly as possible; keeping them around only hurts if the
>> workload has no locality of reference.
>>
>> After:
>>
>> We fault in N folios, some possibly consisting of multiple pages.
>> Each folio is tracked separately, but individual pages in the folio
>> are not tracked; they belong to their folio.  In this scenario, if
>> the other PTEs for pages in the same folio are marked as young or old
>> doesn't matter; the entire folio will be tracked as young, because we
>> referenced one of the pages in this folio.  Marking the other PTEs as
>> young actually helps because we don't take pagefaults on them (whether
>> we have a HW or SW accessed bit).
>>
>> (can i just say that i dislike how we mix up our old/young accessed/not
>> terminology here?)
>>
>> We should still mark the PTEs referencing unaccessed folios as old.
>> No argument there, and this patch does that.  But it's fine for all the
>> PTEs referencing the accessed folio to have the young bit, at least as
>> far as I can tell.
> 
> Ok, thanks for the explanation. So as long as
> arch_wants_old_prefaulted_pte() is taken into account for the unaccessed
> folios, then I think we should be good? Unconditionally marking those
> PTEs as old probably hurts x86.
Yes. We do only mark PTEs old for arch_wants_old_prefaulted_pte()
system. Thanks.


Regards
Yin, Fengwei

> 
> Will