Received-SPF: pass (google.com: domain of linux-kernel+bounces-147721-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Content-Type: text/plain; charset=iso-8859-15; format=flowed; delsp=yes
To: "hpa@zytor.com" <hpa@zytor.com>, "tim.c.chen@linux.intel.com"
 <tim.c.chen@linux.intel.com>, "linux-sgx@vger.kernel.org"
 <linux-sgx@vger.kernel.org>, "x86@kernel.org" <x86@kernel.org>,
 "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
 "jarkko@kernel.org" <jarkko@kernel.org>, "cgroups@vger.kernel.org"
 <cgroups@vger.kernel.org>, "linux-kernel@vger.kernel.org"
 <linux-kernel@vger.kernel.org>, "mkoutny@suse.com" <mkoutny@suse.com>,
 "tglx@linutronix.de" <tglx@linutronix.de>, "Mehta, Sohil"
 <sohil.mehta@intel.com>, "tj@kernel.org" <tj@kernel.org>, "mingo@redhat.com"
 <mingo@redhat.com>, "bp@alien8.de" <bp@alien8.de>, "Huang, Kai"
 <kai.huang@intel.com>
Cc: "mikko.ylinen@linux.intel.com" <mikko.ylinen@linux.intel.com>,
 "seanjc@google.com" <seanjc@google.com>, "anakrish@microsoft.com"
 <anakrish@microsoft.com>, "Zhang, Bo" <zhanb@microsoft.com>,
 "kristen@linux.intel.com" <kristen@linux.intel.com>, "yangjie@microsoft.com"
 <yangjie@microsoft.com>, "Li, Zhiquan1" <zhiquan1.li@intel.com>,
 "chrisyan@microsoft.com" <chrisyan@microsoft.com>
Subject: Re: [PATCH v12 07/14] x86/sgx: Abstract tracking reclaimable pages in
 LRU
References: <20240416032011.58578-1-haitao.huang@linux.intel.com>
 <20240416032011.58578-8-haitao.huang@linux.intel.com>
 <e218d2aab5d793a3b4c64f90f7bad997c745ba91.camel@intel.com>
Date: Tue, 16 Apr 2024 17:48:29 -0500
Precedence: bulk
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
From: "Haitao Huang" <haitao.huang@linux.intel.com>
Organization: Intel
Message-ID: <op.2mcey3spwjvjmi@hhuan26-mobl.amr.corp.intel.com>
In-Reply-To: <e218d2aab5d793a3b4c64f90f7bad997c745ba91.camel@intel.com>
User-Agent: Opera Mail/1.0 (Win32)

On Tue, 16 Apr 2024 09:07:33 -0500, Huang, Kai <kai.huang@intel.com> wrote:

> On Mon, 2024-04-15 at 20:20 -0700, Haitao Huang wrote:
>> From: Kristen Carlson Accardi <kristen@linux.intel.com>
>>
>> The functions, sgx_{mark,unmark}_page_reclaimable(), manage the tracking
>> of reclaimable EPC pages: sgx_mark_page_reclaimable() adds a newly
>> allocated page into the global LRU list while
>> sgx_unmark_page_reclaimable() does the opposite. Abstract the hard coded
>> global LRU references in these functions to make them reusable when
>> pages are tracked in per-cgroup LRUs.
>>
>> Create a helper, sgx_lru_list(), that returns the LRU that tracks a  
>> given
>> EPC page. It simply returns the global LRU now, and will later return
>> the LRU of the cgroup within which the EPC page was allocated. Replace
>> the hard coded global LRU with a call to this helper.
>>
>> Next patches will first get the cgroup reclamation flow ready while
>> keeping pages tracked in the global LRU and reclaimed by ksgxd before we
>> make the switch in the end for sgx_lru_list() to return per-cgroup
>> LRU.
>
> I found the first paragraph hard to read.  Provide my version below for
> your reference:
>
> "
> The SGX driver tracks reclaimable EPC pages via
> sgx_mark_page_reclaimable(), which adds the newly allocated page into the
> global LRU list.  sgx_unmark_page_reclaimable() does the opposite.
>
> To support SGX EPC cgroup, the SGX driver will need to maintain an LRU
> list for each cgroup, and the new allocated EPC page will need to be  
> added
> to the LRU of associated cgroup, but not always the global LRU list.
>
> When sgx_mark_page_reclaimable() is called, the cgroup that the new
> allocated EPC page belongs to is already known, i.e., it has been set to
> the 'struct sgx_epc_page'.
>
> Add a helper, sgx_lru_list(), to return the LRU that the EPC page should
> be/is added to for the given EPC page.  Currently it just returns the
> global LRU.  Change sgx_{mark|unmark}_page_reclaimable() to use the  
> helper
> function to get the LRU from the EPC page instead of referring to the
> global LRU directly.
>
> This allows EPC page being able to be tracked in "per-cgroup" LRU when
> that becomes ready.
> "
>
Thanks. Will use

> Nit:
>
> That being said, is sgx_epc_page_lru() better than sgx_lru_list()?
>

Sure

>>
>> Co-developed-by: Sean Christopherson <sean.j.christopherson@intel.com>
>> Signed-off-by: Sean Christopherson <sean.j.christopherson@intel.com>
>> Signed-off-by: Kristen Carlson Accardi <kristen@linux.intel.com>
>> Co-developed-by: Haitao Huang <haitao.huang@linux.intel.com>
>> Signed-off-by: Haitao Huang <haitao.huang@linux.intel.com>
>> Reviewed-by: Jarkko Sakkinen <jarkko@kernel.org>
>> Tested-by: Jarkko Sakkinen <jarkko@kernel.org>
>> ---
>>
>
> Feel free to add:
>
> Reviewed-by: Kai Huang <kai.huang@intel.com>

Thanks
Haitao