Received-SPF: pass (google.com: domain of linux-kernel+bounces-74158-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Content-Type: text/plain; charset=iso-8859-15; format=flowed; delsp=yes
To: "hpa@zytor.com" <hpa@zytor.com>, "tim.c.chen@linux.intel.com"
 <tim.c.chen@linux.intel.com>, "linux-sgx@vger.kernel.org"
 <linux-sgx@vger.kernel.org>, "x86@kernel.org" <x86@kernel.org>,
 "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
 "jarkko@kernel.org" <jarkko@kernel.org>, "cgroups@vger.kernel.org"
 <cgroups@vger.kernel.org>, "linux-kernel@vger.kernel.org"
 <linux-kernel@vger.kernel.org>, "mkoutny@suse.com" <mkoutny@suse.com>,
 "tglx@linutronix.de" <tglx@linutronix.de>, "Mehta, Sohil"
 <sohil.mehta@intel.com>, "tj@kernel.org" <tj@kernel.org>, "mingo@redhat.com"
 <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "Huang, Kai"
 <kai.huang@intel.com>
Cc: "mikko.ylinen@linux.intel.com" <mikko.ylinen@linux.intel.com>,
 "seanjc@google.com" <seanjc@google.com>, "anakrish@microsoft.com"
 <anakrish@microsoft.com>, "Zhang, Bo" <zhanb@microsoft.com>,
 "kristen@linux.intel.com" <kristen@linux.intel.com>, "yangjie@microsoft.com"
 <yangjie@microsoft.com>, "Li, Zhiquan1" <zhiquan1.li@intel.com>,
 "chrisyan@microsoft.com" <chrisyan@microsoft.com>
Subject: Re: [PATCH v9 08/15] x86/sgx: Implement EPC reclamation flows for
 cgroup
References: <20240205210638.157741-1-haitao.huang@linux.intel.com>
 <20240205210638.157741-9-haitao.huang@linux.intel.com>
 <fa091e657c2d3f3cc14aff15ad3484e0d7079b6f.camel@intel.com>
Date: Wed, 21 Feb 2024 00:44:08 -0600
Precedence: bulk
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
From: "Haitao Huang" <haitao.huang@linux.intel.com>
Organization: Intel
Message-ID: <op.2jhbnui0wjvjmi@hhuan26-mobl.amr.corp.intel.com>
In-Reply-To: <fa091e657c2d3f3cc14aff15ad3484e0d7079b6f.camel@intel.com>
User-Agent: Opera Mail/1.0 (Win32)

[...]
>
> Here the @nr_to_scan is reduced by the number of pages that are  
> isolated, but
> not actually reclaimed (which is reflected by @cnt).
>
> IIUC, looks you want to make this function do "each cycle" as what you  
> mentioned
> in the v8 [1]:
>
> 	I tested with that approach and found we can only target number of
> pages
> 	attempted to reclaim not pages actually reclaimed due to the
> uncertainty
> 	of how long it takes to reclaim pages. Besides targeting number of
> 	scanned pages for each cycle is also what the ksgxd does.
>
> 	If we target actual number of pages, sometimes it just takes too long.
> I
> 	saw more timeouts with the default time limit when running parallel
> 	selftests.
>
> I am not sure what does "sometimes it just takes too long" mean, but  
> what I am
> thinking is you are trying to do some perfect but yet complicated code  
> here.

I think what I observed was that the try_charge() would block too long  
before getting chance of schedule() to yield, causing more timeouts than  
necessary.
I'll do some re-test to be sure.

>
> For instance, I don't think selftest reflect the real workload, and I  
> believe
> adjusting the limit of a given EPC cgroup shouldn't be a frequent  
> operation,
> thus it is acceptable to use some easy-maintain code but less perfect  
> code.
>
> Here I still think having @nr_to_scan as a pointer is over-complicated.   
> For
> example, we can still let sgx_reclaim_pages() to always scan  
> SGX_NR_TO_SCAN
> pages, but give up when there's enough pages reclaimed or when the EPC  
> cgroup
> and its descendants have been looped:
>
> unsigned int sgx_epc_cgroup_reclaim_pages(struct misc_cg *root)
> {
> 	unsigned int cnt = 0;
> 	...
>
> 	css_for_each_descendant_pre(pos, css_root) {
> 		...
> 		epc_cg = sgx_epc_cgroup_from_misc_cg(css_misc(pos));
> 		cnt += sgx_reclaim_pages(&epc_cg->lru);
>
> 		if (cnt >= SGX_NR_TO_SCAN)
> 			break;
> 	}
>
> 	...
> 	return cnt;
> }
>
> Yeah it may reclaim more than SGX_NR_TO_SCAN when the loop actually  
> reaches any
> descendants, but that should be rare and we don't care that much, do we?
>
I assume you meant @cnt here to be number of pages actually reclaimed.  
This could cause  sgx_epc_cgroup_reclaim_pages() block too long as @cnt  
may always be zero (all pages are too young) and you have to loop all  
descendants.

Thanks

Haitao