Received-SPF: pass (google.com: domain of linux-kernel+bounces-38111-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <fcbf3eec-e67b-4377-a888-a59b2583ab5f@shopee.com>
Date: Thu, 25 Jan 2024 15:54:55 +0800
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 3/3] x86/resctrl: Display cache occupancy of busy RMIDs
To: Reinette Chatre <reinette.chatre@intel.com>
Cc: fenghua.yu@intel.com, babu.moger@amd.com, peternewman@google.com,
 x86@kernel.org, linux-kernel@vger.kernel.org,
 James Morse <james.morse@arm.com>
References: <20240123092024.1271882-3-haifeng.xu@shopee.com>
 <8abb0237-90e3-484e-a549-d74f540d4045@intel.com>
From: Haifeng Xu <haifeng.xu@shopee.com>
In-Reply-To: <8abb0237-90e3-484e-a549-d74f540d4045@intel.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit


On 2024/1/25 06:25, Reinette Chatre wrote:
> (+James)
> 
> Hi Haifeng,
> 
> On 1/23/2024 1:20 AM, Haifeng Xu wrote:
>> If llc_occupany is enabled, the RMID may not be freed immediately unless
>> its llc_occupany is less than the resctrl_rmid_realloc_threshold.
>>
>> In our production environment, those unused RMIDs get stuck in the limbo
>> list forever because their llc_occupancy are larger than the threshold.
>> After turning it up , we can successfully free unused RMIDs and create
>> new monitor groups. In order to accquire the llc_occupancy of RMIDs in
>> each rdt domain, we use perf tool to track and filter the log manually.
>>
>> It's not efficient enough. Therefore, we can add a RFTYPE_TOP_INFO file
>> 'busy_rmids_info' that tells users the llc_occupancy of busy RMIDs. It
>> can also help to guide users how much the resctrl_rmid_realloc_threshold
>> should be.
> 
> I am addressing both patch 2/3 and patch 3/3 here.
> 
> First, please note that resctrl is obtaining support for Arm's Memory 
> System Resource Partitioning and Monitoring (MPAM) and MPAM's monitoring
> is done with a monitoring group that is dependent on the control group,
> not independent as Intel and AMD. Please see [1] for more details.
> 
> resctrl is the generic interface that will be used to interact with RDT
> on Intel, PQoS on AMD, and also MPAM on Arm. We thus need to ensure that
> the interface is appropriate for all. Specifically, for Arm there is
> no global "free RMID list", on Arm the free RMIDs (PMG in Arm language,
> but rmid is the term that made it into resctrl) are per control group.
> 
> Second, this addition seems to be purely a debugging aid. I thus don't see
> this as something that users may want/need all the time, yet when users do
> want/need it, accurate data is preferred. To that end, the limbo
> code already walks the busy list once per second. What if there is a
> new tracepoint within the limbo code that shares the exact data used during
> limbo list management?

OK, I'll try this way.

 From what I can tell, this data, combined with the
> per-monitor-group "mon_hw_id", should give user space sufficient data to
> debug the scenarios mentioned in these patches.
> 
> I did add James to this discussion to make him aware of your requirements.
> Please do include him in future submissions.
> 
> Reinette
> 
> [1] https://urldefense.proofpoint.com/v2/url?u=https-3A__lore.kernel.org_all_20231215174343.13872-2D1-2Djames.morse-40arm.com_&d=DwICaQ&c=R1GFtfTqKXCFH-lgEPXWwic6stQkW4U7uVq33mt-crw&r=3uoFsejk1jN2oga47MZfph01lLGODc93n4Zqe7b0NRk&m=-XE6uI2GOyk-qzRRAWvuDzQ9NgM2-QK-KLArnJEYmu02heN9gOh6VMbPeF1iZUZe&s=FySup-TxYl6c-jaA7Q8OFIVwbMdsMxZ3ChQ6Sj0HaLA&e= 

Thanks.