Received-SPF: pass (google.com: domain of linux-kernel+bounces-163097-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Message-ID: <5836260d-600d-4802-9f6a-b0bdb19ddbac@intel.com>
Date: Tue, 30 Apr 2024 10:18:05 +1200
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v12 12/14] x86/sgx: Turn on per-cgroup EPC reclamation
To: Haitao Huang <haitao.huang@linux.intel.com>, "hpa@zytor.com"
	<hpa@zytor.com>, "tim.c.chen@linux.intel.com" <tim.c.chen@linux.intel.com>,
	"linux-sgx@vger.kernel.org" <linux-sgx@vger.kernel.org>, "x86@kernel.org"
	<x86@kernel.org>, "dave.hansen@linux.intel.com"
	<dave.hansen@linux.intel.com>, "jarkko@kernel.org" <jarkko@kernel.org>,
	"cgroups@vger.kernel.org" <cgroups@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"mkoutny@suse.com" <mkoutny@suse.com>, "tglx@linutronix.de"
	<tglx@linutronix.de>, "Mehta, Sohil" <sohil.mehta@intel.com>, "tj@kernel.org"
	<tj@kernel.org>, "mingo@redhat.com" <mingo@redhat.com>, "bp@alien8.de"
	<bp@alien8.de>
CC: "mikko.ylinen@linux.intel.com" <mikko.ylinen@linux.intel.com>,
	"seanjc@google.com" <seanjc@google.com>, "anakrish@microsoft.com"
	<anakrish@microsoft.com>, "Zhang, Bo" <zhanb@microsoft.com>,
	"kristen@linux.intel.com" <kristen@linux.intel.com>, "yangjie@microsoft.com"
	<yangjie@microsoft.com>, "Li, Zhiquan1" <zhiquan1.li@intel.com>,
	"chrisyan@microsoft.com" <chrisyan@microsoft.com>
References: <20240416032011.58578-1-haitao.huang@linux.intel.com>
 <20240416032011.58578-13-haitao.huang@linux.intel.com>
 <524cf9b081d86ae61342fdfc370a3639d0010f94.camel@intel.com>
 <op.2mzyy7ktwjvjmi@hhuan26-mobl.amr.corp.intel.com>
Content-Language: en-US
From: "Huang, Kai" <kai.huang@intel.com>
In-Reply-To: <op.2mzyy7ktwjvjmi@hhuan26-mobl.amr.corp.intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MGVta3BhRW45S2ZGY3RnMjQ0a0hscEZPRktIWHhJMDJvaTEybUo4VE1OV25w?=
 =?utf-8?B?cFFsOElReW04Q2x3NFpDb0JqbnNwREdiRE15ZHFISnhOUzVBc0c4QUdjbUNa?=
 =?utf-8?B?V1VITWxQVFA5UzJsSkk2dXhCWVBjN281VmI3MkVNTzlhNFJHQkhOUVFTSm1U?=
 =?utf-8?B?b1RLRzdOTU9mSkF2bktjNjdXMDhET0M4dEU0NkZ4WFRBYTg2eVdWaFBVUmVU?=
 =?utf-8?B?aVBBQzVyOGppVUxuMlFDMnFhZFlsZStnbFVCYVpDTlZyMVBQV2UwcmV3T3lQ?=
 =?utf-8?B?aGlVanVYOGhSZys0SjBZRGZ5aFRRMDRJOXVKS1hKZXNJbmxDby9ZN21Vb0N2?=
 =?utf-8?B?M2FUSm42Y1VkR0tNbEsxcWJHWnZ1WG9OcHE2eURkdDhyelBwN29xOFRMTFl4?=
 =?utf-8?B?ZEJUUDFHNG45bUVmTS9EaGJ3dXA3ODZFQkpvTGVzTnBTOWVnbkZkQnhKNUhw?=
 =?utf-8?B?REFBVlQ3ZHNzdjRBUEZrakFzS1VzalNOV0FhRUZmUjYyU29ONktVeVVSd25I?=
 =?utf-8?B?TlduVVJkU09rK2Nna292Z2tkUVhYQzAvQXR2ZEZGVTBnN1lzM2RxRjFGR3Q3?=
 =?utf-8?B?MTJKWjduSG9uc3dibEdla3JGOG1KQ3RIbyt1Ry9wMEdDeW82d2NSUm15TEdH?=
 =?utf-8?B?UU45SjgwcStWaTRFakdPL3RXVkRDS01ZcGZwWFdNbDZPbnpCU2x6dkE4UmlY?=
 =?utf-8?B?alBjZXYySlVBNG0vc283SXMzOWhXRmpUYmR6UldrblgrOW1hd0FCTzFidzE2?=
 =?utf-8?B?aUtNeVBaNVNRbkt5QjAyTHUvbFluWVovRlpXSDdXeFZaSHNiWG5mcUQ3MlV1?=
 =?utf-8?B?UWhlNkZZWlB2ZWhYWGxDWE5MUEo4V0NiNlVIemF3R3BzaU9ZQVBQVjRmU2FZ?=
 =?utf-8?B?aXFiS2xrREFHN0xiZWoweVUwaEZ5REQ3c29naVdEWU1wc1I0QytLT0pRb0VD?=
 =?utf-8?B?MWIwaEp3b21SVDZRK2piOWIxYXhpSFVLZ2g0bjdlRDFPeE5LdVY3am5nSyt6?=
 =?utf-8?B?cE5pMzAvSEoxeU1vaEtSTTZJMTIwMHErZHZqZWcvZ0wrdy9MMHk1NVJKTEZJ?=
 =?utf-8?B?MkVlTGFUWTNGWWxuQURRZTBrZ3FRVjVFSzlIZkpwRTFTdFlUYk5CVHhYeVkx?=
 =?utf-8?B?RjVMOFlMK3lSTjY5T1M0Sjg3OU1xV1o4MEYxSnRYNFRvYnNJUDdEcXE2V0ho?=
 =?utf-8?B?Q2VWSEhoV25uQUxJdng1bDNaODQ3NHd5RlU0TFkvSU5ZdW02Y2pWQ3lBWkI3?=
 =?utf-8?B?bDdtRDZkM0VVZW5VQnMzaFpPSU9kMmpZSUkxK052UzFmSXF6RmV6a2FZM3JQ?=
 =?utf-8?B?dFFtOXp3dUN5S084UmR3Z09KYzJYVFJQRXYzS3N2Vm1qaTJXaFNDN1cvL2tU?=
 =?utf-8?B?ejNxOXN5bUJzN0ppN2RGYkFjN2N0MU5ZODJ3YkVkTGpvRTZUR1Mxb1dFclpl?=
 =?utf-8?B?QXlxRmJkYlhUd1BhNDdnT29FMy9oaEUwTTQ2ZCtYeHgxQ2lMMk00ZHJKWEJ2?=
 =?utf-8?B?K3ZOU1krL2ZYZkYxYmc5bGkrcS9EblZVOCt0Vkk5RWVlTTJwTHJZYnNKaTEz?=
 =?utf-8?B?MEU1dGh1OEpsOWdWRVFtc240QXl2eURlaDJaR0ZCc0ZQYTRVai9HSjJ1dVZM?=
 =?utf-8?B?RjBjeHhlL3VONW8vRFFpM3JKbjRtRWJ1V2w3YXRHbUowVjl1d2tFSEFjSXli?=
 =?utf-8?B?RUdNcWNYeUZXT2R2MytRV3RSaG53WHF6bkdic1ZBNW5HTzVIUGViTFZJMDQy?=
 =?utf-8?B?SFVrUjZIbFZkQTBiVG9tNGlDSllTZnNTNi80M3ZwaS9aWkNNVTVzWE11QlRW?=
 =?utf-8?B?bldrMEs5UHk4RUlsNFArdk1lcy94VG1VMkFnL0V2cmtXdjE5K1VZZVRjM3dr?=
 =?utf-8?B?ZWFLWCtUOVVIWGZKYmU4a0ErTThqa01ldUh1QjZZOTVwd3RuaC9LUEhvS2VD?=
 =?utf-8?B?S0ZMZmo5VFhrZGVrTXJJWDhjSUtWcC8raXpkendkVUVHMGtNUGVlSmVEelpS?=
 =?utf-8?B?eXgveHNJSFFiNnp1dVlZaThCWU55NFFFbzB3bkRpOWhPd0pnWVlOenp6Q1VH?=
 =?utf-8?B?NDZNVDllUGVwbGxuN3ZQV3VySzd5cXpDUlZXZWFkZkpjcS9GMGFJeE9tYkdW?=
 =?utf-8?Q?PW8dtJqfaTNVYcxTQY1Ya9UNj?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 0f208891-fd8f-42f7-c397-08dc689a45ec
X-MS-Exchange-CrossTenant-AuthSource: BL1PR11MB5978.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Apr 2024 22:18:18.9268
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3USyOwrzDYB2wfzuL9kS7OZSqd6SaQ99d4/0EPuqhH90IKrT7AFghjOgGRa/i1XJbILyJxxUvdu6vhohYcil4A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR11MB6503
X-OriginatorOrg: intel.com


>>>  /*
>>> @@ -42,7 +63,8 @@ static inline struct sgx_epc_lru_list 
>>> *sgx_lru_list(struct sgx_epc_page *epc_pag
>>>   */
>>>  static inline bool sgx_can_reclaim(void)
>>>  {
>>> -    return !list_empty(&sgx_global_lru.reclaimable);
>>> +    return !sgx_cgroup_lru_empty(misc_cg_root()) ||
>>> +           !list_empty(&sgx_global_lru.reclaimable);
>>>  }
>>
>> Shouldn't this be:
>>
>>     if (IS_ENABLED(CONFIG_CGROUP_MISC))
>>         return !sgx_cgroup_lru_empty(misc_cg_root());
>>     else
>>         return !list_empty(&sgx_global_lru.reclaimable);
>> ?
>>
>> In this way, it is consistent with the sgx_reclaim_pages_global() below.
>>
> 
> I changed to this way because sgx_cgroup_lru_empty() is now defined in 
> both KConfig cases.
> And it seems better to minimize use of the KConfig variables based on 
> earlier feedback (some are yours).
> Don't really have strong preference here. So let me know one way of the 
> other.
> 

But IMHO your code could be confusing, e.g., it can be interpreted as:

   The EPC pages can be managed by both the cgroup LRUs and the
   sgx_global_lru simultaneously at runtime when CONFIG_CGROUP_MISC
   is on.

Which is not true.

So we should make the code clearly reflect the true behaviour.