Received-SPF: pass (google.com: domain of linux-kernel+bounces-28018-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Message-ID: <60a1eed0-7956-4f4b-8113-d5f2d44f3595@intel.com>
Date: Tue, 16 Jan 2024 11:55:07 -0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/resctrl: Implement new MBA_mbps throttling heuristic
Content-Language: en-US
To: Tony Luck <tony.luck@intel.com>, Fenghua Yu <fenghua.yu@intel.com>, "Peter
 Newman" <peternewman@google.com>, <x86@kernel.org>
CC: James Morse <james.morse@arm.com>, Jamie Iles <quic_jiles@quicinc.com>,
	Babu Moger <babu.moger@amd.com>, Xiaochen Shen <xiaochen.shen@intel.com>,
	<linux-kernel@vger.kernel.org>, <patches@lists.linux.dev>
References: <20231201214737.104444-1-tony.luck@intel.com>
 <20240109220003.29225-1-tony.luck@intel.com>
From: Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <20240109220003.29225-1-tony.luck@intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?R3o4WUE1WG1tRzdZY3dLZmVwdGZvVEpBd3FEQUFyRkFEZjFWd2hmSk9DSWFm?=
 =?utf-8?B?TE9zRCt1anZWSlFzbndISzV6TXJRcmpUQnVIcld4cHhTVU50ZnNlcTF1czB2?=
 =?utf-8?B?a0x6djZoUDVEZEVqMmpPSUpPeEVkTjRWdFFkUnhqMWdCMUNVSDlxNWZRNzVz?=
 =?utf-8?B?ellxa0dKN2pIQmVjRFVSM1k5YXhGczdTR2JmOGp5SldaWG5XLzc0VDRpRjUx?=
 =?utf-8?B?Yk82THlsT1htSGRRNnlRajlJNGJUNnYrY0NzZEpBSFErWlZVN3A0ZjRWYnVJ?=
 =?utf-8?B?VS85SGIwVVprUXUyQnV5TTZCVTZVY2RpMEs0VlU5eFI4SG5CTEV6UHpoNVg0?=
 =?utf-8?B?QmtjM3dXSjRsazc1eHNKSVZjMzJPZFNWTHRNZENOVGVIQjBLSWsya3J3WDlD?=
 =?utf-8?B?UXIvU2x1bXVDaURHYmpteHhvWmp5cXF3TjhkN3NobG9uL0E3T3B6S2RFQSt0?=
 =?utf-8?B?KzhPWllidW0zbXdNa0krdjArR2VaS204eVBoa1Y1RDNFNWhBZFpWYzZHaW8r?=
 =?utf-8?B?SzV6dmtSQWxGNEZFd2tjVzVzeXFuR3M4SDFyU29oMklyY0VwaURsK3RmVTJH?=
 =?utf-8?B?ZzRoRzJaTEd1WjJVTnpyZ1hmYnNUQ3NUcjRCNitQTFIrV1Z5VmZ2dkU2RHQx?=
 =?utf-8?B?Y1l2TVIzOXIwZFdtYTIvQTRQeGtJYjc1TzRBRFdoZ0dxUzRmVUt3QTRFV0FP?=
 =?utf-8?B?RUtQUlVVUTJybzIrbW45aG9tWFgvbUpvQ0tWdm5GWGpKMWRFclNtZFZTQjF5?=
 =?utf-8?B?KyttVjg1bEJLTjk2S0NOOWJITXJrRGxYUlQxYTludzNVRkFWYVlvRXI1MmJB?=
 =?utf-8?B?NEt1d3FsZTlTMjh4YWcwc1NXRlJjWWJuYk04ajhvTk1jWUZvZkVXeFdQeWhw?=
 =?utf-8?B?VW9YM0E1TEsvZDRlQWVVdlNxclh5ajBjYzN1aGNWckJWczBsZlBKY1lWOFpW?=
 =?utf-8?B?MDgzenhBUnErM3h2VEV1aGpvWEwxR3R6TjNGNUhtYXkwZWpXK2RlNkxmbUZn?=
 =?utf-8?B?WFpBeWptTVB5VG93N3FwZUpra3NZUGk4TFAzY3Q3SEFiQW41WFg1RDg0dG5X?=
 =?utf-8?B?MnNLY0dFNmZzOXNYWW5TdTVHVkNSdi9Fd2JGejFYMllQZ3lyQlZNL0tET282?=
 =?utf-8?B?anZNYUtoc1NLZGRUM2ZPQTNUSU5Ic1R0SUExNVdkNTVCVk95emYyaG8rV0h1?=
 =?utf-8?B?MWhsNjRKYkxON0JJTW1xNUltR1l2OC85YlJkdjdKVkhyVHlIdFhkNUJaZldw?=
 =?utf-8?B?QW5wWHBENnpLRllSb2xBbmhWS045RnFJOHBmbE8zWThpa3B2TlJrYTd1SW16?=
 =?utf-8?B?M21TTi9CS3dyaGl5SDJpSGUrYmpDeVQwM1RSS2pSM2QzbWJIbHg4bUtNa3Rs?=
 =?utf-8?B?MmozUUlIK3FXakpXTXB0SlJlczRkRm5QUDlCcnNlZUxFOVhqaXlKRHQxMy9C?=
 =?utf-8?B?dFd2VWNyejNOaGozMS9DcjFtT0ZmVmZzVDAwdHN1ZEJRQmN6YkpkUzZNQVJV?=
 =?utf-8?B?bzErZXFUTmtJaUR0RVlhbDhWanpTdkFyVUd3RnErRWpHa29SYm9PVlJOZkZW?=
 =?utf-8?B?aVlHc2cxRlJRcVhUR0NyeWtnWkErZG1QMkQwclRCcm5mUXhrL2NqN1FVU2xy?=
 =?utf-8?B?eWZ6STZ6WWlrdUQ1TmlvWnBmZmtGRHR6aEl0VmhLTTlidnB3Q0xOamdDZ1NU?=
 =?utf-8?B?UzZoWlA2QlhaSE5VU0xLWGx2ejQ2TktvRVkwejdlZFNtT204dWVnTTFqV1c4?=
 =?utf-8?B?OCsrZ3VoeEE5NmM5Y3ZiamRJMmMwZ0szNHlGQUk4QWc4Z2dmaVI0K0wyT2do?=
 =?utf-8?B?YjMwckQxc0JzVFZzSkFmM3ViamFtZGFMOTM5amxzcnlsWnk0cGpGWjl0cDlz?=
 =?utf-8?B?TUEvVzRjR29WcjNDbVpHbE5jTGRJaE02dENqRkFpS0l2UDZyeWdaNnlONFBE?=
 =?utf-8?B?RkFDNldYUUVUdkxhWXI3NjdoMHhPaC84bW8yUjdZUHBRQ2FjUEE1ckFIekQz?=
 =?utf-8?B?MDZYVFlUQ25hT3d2VnVrTnppMEhpT2k2Z29JaTdXMkJKS28za3hkRW9ac2VO?=
 =?utf-8?B?MTlWWWFYVVpWRHY3ekV4R3ZIVUdtVGs3V3ZIOS9aV0x2SXpGUGZSak8xWXNP?=
 =?utf-8?B?MVNBcS9qNFpsMEtjV1p6VDl1NURsSmh1TENBelAzRFlnaWNqckU4QW9EbU1o?=
 =?utf-8?B?blE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: a734e7a3-d238-48c2-0620-08dc16cd0b98
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 16 Jan 2024 19:55:09.9951
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: CzZv3RnjnuJnoforHvNpJtPQW6biP9zOIqcATpVFkMtJn8t7QVEE621bfJmUsfnuu41EllVDjHw7WDyTclnhhYIMlGGutgjehMHbkhC0BZ0=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR11MB6127
X-OriginatorOrg: intel.com

Hi Xiaochen,

On 1/9/2024 2:00 PM, Tony Luck wrote:
> The MBA_mbps feedback loop increases throttling when a group is using
> more bandwidth than the target set by the user in the schemata file, and
> decreases throttling when below target.
> 
> To avoid possibly stepping throttling up and down on every poll a
> flag "delta_comp" is set whenever throttling is changed to indicate
> that the actual change in bandwidth should be recorded on the next
> poll in "delta_bw". Throttling is only reduced if the current bandwidth
> plus delta_bw is below the user target.
> 
> This algorithm works well if the workload has steady bandwidth needs.
> But it can go badly wrong if the workload moves to a different phase
> just as the throttling level changed. E.g. if the workload becomes
> essentially idle right as throttling level is increased, the value
> calculated for delta_bw will be more or less the old bandwidth level.
> If the workload then resumes, Linux may never reduce throttling because
> current bandwidth plu delta_bw is above the target set by the user.
> 
> Implement a simpler heuristic by assuming that in the worst case the
> currently measured bandwidth is being controlled by the current level of
> throttling. Compute how much it may increase if throttling is relaxed to
> the next higher level. If that is still below the user target, then it
> is ok to reduce the amount of throttling.
> 
> Fixes: ba0f26d8529c ("x86/intel_rdt/mba_sc: Prepare for feedback loop")
> Reported-by: Xiaochen Shen <xiaochen.shen@intel.com>
> Signed-off-by: Tony Luck <tony.luck@intel.com>
> ---
> 
> This patch was previously posted in:
> 
>   https://lore.kernel.org/lkml/ZVU+L92LRBbJXgXn@agluck-desk3/#t
> 
> as part of a proposal to allow the mba_MBps mount option to base its
> feedback loop input on total bandwidth instead of local bandwidth.
> Sending it now as a standalone patch because Xiaochen reported that
> real systems have experienced problems when delta_bw is incorrectly
> calculated.
> 

Does this new heuristic fix the problems you observe? If so, would you be
willing to provide a "Tested-by" tag?

Thank you.

Reinette