Received-SPF: pass (google.com: domain of linux-kernel+bounces-167055-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Message-ID: <3eb37a6d-ff52-41b0-8f74-d9d049e6ce05@intel.com>
Date: Fri, 3 May 2024 11:23:06 +1200
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v13 11/14] x86/sgx: Abstract check for global reclaimable
 pages
To: Haitao Huang <haitao.huang@linux.intel.com>, <jarkko@kernel.org>,
	<dave.hansen@linux.intel.com>, <tj@kernel.org>, <mkoutny@suse.com>,
	<linux-kernel@vger.kernel.org>, <linux-sgx@vger.kernel.org>,
	<x86@kernel.org>, <cgroups@vger.kernel.org>, <tglx@linutronix.de>,
	<mingo@redhat.com>, <bp@alien8.de>, <hpa@zytor.com>, <sohil.mehta@intel.com>,
	<tim.c.chen@linux.intel.com>
CC: <zhiquan1.li@intel.com>, <kristen@linux.intel.com>, <seanjc@google.com>,
	<zhanb@microsoft.com>, <anakrish@microsoft.com>,
	<mikko.ylinen@linux.intel.com>, <yangjie@microsoft.com>,
	<chrisyan@microsoft.com>
References: <20240430195108.5676-1-haitao.huang@linux.intel.com>
 <20240430195108.5676-12-haitao.huang@linux.intel.com>
Content-Language: en-US
From: "Huang, Kai" <kai.huang@intel.com>
In-Reply-To: <20240430195108.5676-12-haitao.huang@linux.intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?dHQ2aGRMc21nUFBGUzZXRVdsZmxkTDdCVXoyRi9BcURmeUZxTmQrem41Y0lV?=
 =?utf-8?B?Ly9oWkxlbzJvMGZ3YXRNVWlaZmZ3WTQrL0N0bW16M0tvakZkNEc0KzRxL3pn?=
 =?utf-8?B?TzlYL0hyYWhwWTl2ZEVlTXZpTXFRSkliSDBGczlhbjNpT2F5cmxiSnZKOVBH?=
 =?utf-8?B?b0tlNmprNXFyK1N4RkFmQ3VkMjZWRDlrbXpTc2hncWY1ZE45V2U3dSsvWTlD?=
 =?utf-8?B?S3c3WlBMTHJvZEd0SGRtZHEyM1RRRzRnQUpRQkRvVlN0TDNpVlAvQWs4NUlw?=
 =?utf-8?B?Wkpyakp1Vk85eGdNWWxnYTkwdE1ueXZVdEVDYkpPaG1yY0xpcUpxcGE1R0Nm?=
 =?utf-8?B?Y1kwZXRDNHhkbmJmbHpEMVpwMXFoQjNsSktsajR3SG5JNzF1RW1UNHZqV2VF?=
 =?utf-8?B?WUk1YUJ5KytGVHdOdGIxN3NiQVZMTDEveG9RL1VaZmhqS2VVcEZxKzhVSEVw?=
 =?utf-8?B?ZXBGS0htQTI0Y3JhNzJ0ZjNwUEpNakh3SEwzcndPVW8yZnFwamxJWlhXUjdj?=
 =?utf-8?B?Kyt1ejhvbFFKSE4veFJJS01SUVdMQlZ3ME9VTGc2RTN1U2xkRlk0YUV2aUQz?=
 =?utf-8?B?TjFXRXZNZlM4ZGw3T09MeHJZUWY2TSt3ajI2QVZoMUp3bHJhQW1rSUo1OU5u?=
 =?utf-8?B?d2o3N1RWQ2VjeVAvT2RZRmVoODRSVkNRelFmYSszM1l4eUVYNjEwVnpXekZM?=
 =?utf-8?B?VUhWalNEWGNuV0JDRzJoSWNyMDhHQnU2TWViM29HWWlMa3JTbGF4aFkvcUtD?=
 =?utf-8?B?QmVDZW1tQjhKZ3hwbG5BSEVsMnU0ZmFaWWJQTkFNTStjVUZxL1F2M1F1Zm1k?=
 =?utf-8?B?R2pGVjk0REl6VTgwSDFpWUZsb3RxUUZZUFFyWHNnK3VRWEpQcmR6MGJlc01s?=
 =?utf-8?B?L3ZIY0JEOUVNY1d3NjJKQWZCbDlYWk5oYUE4Sko5RnYwbm9VNUxVdXVLQmtt?=
 =?utf-8?B?Z1ZNbzJTOFdkTExuVkJ0WU9jV1dPSWhyS2hzbHpvdVdmUGxiTjRqbGVuVmNo?=
 =?utf-8?B?SlpWTk4zUCtJQktiNWdrSGdmN0h3V0YvUU54OGk0dytKUVFPZ0poVDVicURO?=
 =?utf-8?B?V2V1VDAvNnZCRlBvS2kzb2grZFFzNnl1QTdneDdNQmxEMG40SDhIaXllRC9t?=
 =?utf-8?B?K3hNTDUrb3dMKzBsU09UV2NFaXFvZlNzTEFCcFJ3em9Mb04vN2dJSExleWlE?=
 =?utf-8?B?SmhqdVBnRXZEVXRuemZ5dTEza3BnMXViUGc3VEZRbitFRkhWTUNFQTZueTY1?=
 =?utf-8?B?OUg2TS8zSXJEeXBJdklHSHhXdUxMbkNDeDlUTGpYeWJpTkpQd2pzRTBYQnBm?=
 =?utf-8?B?WU5adUNsOTlNaFFCWUtYbExUZzNyN21HZzh1MVZPZFZkbnNjL01SbDVsSjdn?=
 =?utf-8?B?ZkdpdytVRTF1UVE3M296UElnRk1lNmd1RlAwaDJydnM3dlBHWG9ucytkYm9S?=
 =?utf-8?B?WDN2R21sZ2RhTEVoZG1maW1pWVVmbUZNWWJhWjR4dlU2b242elcwa1dBZlNR?=
 =?utf-8?B?THJNZXVkWVArelVTMGN5NklEU21KZHo3aTgrYnRUUUJuaG5rajFLaHFuUzZs?=
 =?utf-8?B?WjBrUW5wU09qUGU4bzBBMUpaVThsTVE0SGJMS21RTWtPbU5LTzNjMkFWUllC?=
 =?utf-8?B?dWJsdW05Z0xQek8yYzJHYmcvMFdSS1Mxc2MzYTl2SnZUMktIRGtIS0JOS3Vq?=
 =?utf-8?B?YzYza2VxcE00YzRJNzhzaDFpbmJXanNPa3AzQ2dhNU1NYzhLNnI1anFBQXNF?=
 =?utf-8?B?dmUza2s2b1BjWWx5VTVwUy9zMzh1RnNUam85aGEyTDNsQzZSRU5jb0JQV1lK?=
 =?utf-8?B?V0h3RmZiOXhFVFR6dXdYYTYvVVVhSlNYQzFCT0VBYVNlcTZLU3ljNmt0aUg1?=
 =?utf-8?B?VHFqM3J1Q0NoZzNGb1EwTGJrMk91S1cvdElNRnlwUmpyMytZcllHWmQxUU1D?=
 =?utf-8?B?MytLV1VFeHlGckttNDE1MDVGeHMzYU12VzAwcnhGRmwvQXdzN0lTa3ByRG5i?=
 =?utf-8?B?Z0VIdW1RSDdKMkNYbGV3MjlJOGJiSys3ejlkNjVERS91dmRTUDF1VUdKdXJH?=
 =?utf-8?B?ZERQSkdyWnY4RTBIYXZsR3Qxd0loRnFlVHQ2eTA1VXBuOU9mTHR3bVkycDAr?=
 =?utf-8?Q?DhkSJUzhThcn64GVuiDhgIfBj?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 1dede960-bb03-449f-5829-08dc6afedad7
X-MS-Exchange-CrossTenant-AuthSource: BL1PR11MB5978.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2024 23:23:20.7146
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: eAkZibxeRARI9SKYBUtTkJkZ4Vz50vdGjDCGarSGv2QICj7GHK+WsAsxiCm6u1TrOgn9QqbVqPaQAU9afweQWA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR11MB4717
X-OriginatorOrg: intel.com


On 1/05/2024 7:51 am, Haitao Huang wrote:
> From: Kristen Carlson Accardi <kristen@linux.intel.com>
> 
> For the global reclaimer to determine if any page available for
> reclamation at the global level, it currently only checks for emptiness
> of the global LRU. That will be inadequate when pages are tracked in
> multiple LRUs, one per cgroup. For this purpose, create a new helper,
> sgx_can_reclaim_global(), to abstract this check. Currently it only
> checks the global LRU, later will check emptiness of LRUs of all cgroups
> when per-cgroup tracking is turned on.
> 
> Replace all the checks for emptiness of the global LRU,
> list_empty(&sgx_global_lru.reclaimable), with calls to
> sgx_can_reclaim_global().
> 
> Rename sgx_should_reclaim() to sgx_should_reclaim_global() as it is used
> to check if a global reclamation should be performed.
> 
> Co-developed-by: Sean Christopherson <sean.j.christopherson@intel.com>
> Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
> Signed-off-by: Kristen Carlson Accardi <kristen@linux.intel.com>
> Co-developed-by: Haitao Huang <haitao.huang@linux.intel.com>
> Signed-off-by: Haitao Huang <haitao.huang@linux.intel.com>
> Tested-by: Jarkko Sakkinen <jarkko@kernel.org>
> ---

Free free to add:

Reviewed-by: Kai Huang <kai.huang@intel.com>

One thing below:

[...]

> -static bool sgx_should_reclaim(unsigned long watermark)
> +static bool sgx_should_reclaim_global(unsigned long watermark)
>   {
>   	return atomic_long_read(&sgx_nr_free_pages) < watermark &&
> -	       !list_empty(&sgx_global_lru.reclaimable);
> +		sgx_can_reclaim_global();
>   }
>   
>   static void sgx_reclaim_pages_global(struct mm_struct *charge_mm)
> @@ -405,7 +413,7 @@ static void sgx_reclaim_pages_global(struct mm_struct *charge_mm)
>    */
>   void sgx_reclaim_direct(void)
>   {
> -	if (sgx_should_reclaim(SGX_NR_LOW_PAGES))
> +	if (sgx_should_reclaim_global(SGX_NR_LOW_PAGES))
>   		sgx_reclaim_pages_global(current->mm);
>   }
>   

Hmm.. Sorry for not pointing out in the previous version.

Perhaps it makes more sense to do the rename in the patch ...

   x86/sgx: Add basic EPC reclamation flow for cgroup

.. where we have actually introduced the concept of per-cgroup reclaim, 
and we literally have below change in that patch:

  void sgx_reclaim_direct(void)
  {
  	if (sgx_should_reclaim(SGX_NR_LOW_PAGES))
-		sgx_reclaim_pages();
+		sgx_reclaim_pages_global();
  }

So in that patch, the sgx_should_reclaim() literally just means we 
should do gloabl reclaim, but not the per-cgruop reclaim.  Thus, perhaps 
we just do the renaming here together with the new 
sgx_reclaim_pages_global().

If there's a new version needed, please move the renaming to that patch?