Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.32 as permitted sender) client-ip=23.128.96.32;
Message-ID: <95a21b05-1838-4098-854f-f60f82f26f5f@intel.com>
Date:   Wed, 29 Nov 2023 15:48:37 -0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v4] x86/resctrl: Add mount option to pick total MBM event
Content-Language: en-US
To:     Tony Luck <tony.luck@intel.com>, Fenghua Yu <fenghua.yu@intel.com>,
        "Peter Newman" <peternewman@google.com>,
        Jonathan Corbet <corbet@lwn.net>,
        "Shuah Khan" <skhan@linuxfoundation.org>, <x86@kernel.org>
CC:     Shaopeng Tan <tan.shaopeng@fujitsu.com>,
        James Morse <james.morse@arm.com>,
        Jamie Iles <quic_jiles@quicinc.com>,
        Babu Moger <babu.moger@amd.com>,
        Randy Dunlap <rdunlap@infradead.org>,
        <linux-kernel@vger.kernel.org>, <linux-doc@vger.kernel.org>,
        <patches@lists.linux.dev>
References: <20231026200214.16017-1-tony.luck@intel.com>
 <20231128231439.81691-1-tony.luck@intel.com>
From:   Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <20231128231439.81691-1-tony.luck@intel.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WDVPSkYrMC9URDlTS3VzNi94L29rbEIwemdhYzRCR3BYRklUd0FwMmo5dmRw?=
 =?utf-8?B?VnVaSFdiY2F5UGQ2NCtBUk5VTDJHd056dEFhTnpmVlowTUV0WitSY0VBaEQ0?=
 =?utf-8?B?QkljTVY5V2hLS2tOZk94a2crRlc4bmUzWFBYL0RjRG1TNlFVRkNnNUpUTkJJ?=
 =?utf-8?B?RGVYODAzK0FQSXNDRWZNL0pQWmFnWXNDZFY3bVdtOXNNUkYydEl6cEdXcUlS?=
 =?utf-8?B?aDA4eVFVMzBFL01MekxBd2d1NTlyVm9ucUs3dkZjYmNVN29zZStVeXp2QUVl?=
 =?utf-8?B?cGJHK2wweHQ2RGlrZWhmNmJ2LzZxSit1VFdJdGpFa0lKbjV3djVGK1JlOUMr?=
 =?utf-8?B?ZXU4d1BRSTF2WXB6VzdjRU1RU0NCSFpLSHh0bmJ3M1FxR0VyQzVUeFZxZHBJ?=
 =?utf-8?B?SHhCWXBHbHRINXY5N2VyWEZybW9LOFowczd5QmkxZ3VkbGxITlNVdW9sN2FB?=
 =?utf-8?B?S054TStLd0QzcVBJL240M3lPcGhZc3IxZzZDelUxZXN3dnNNcEZNME1GRU5z?=
 =?utf-8?B?THpiSWpnN0ZwV05PR0R4S3pkaVNuSGhZbU1NY0VsOGdDamNVd2RUN3AwaVl1?=
 =?utf-8?B?eW1vcUtxVlVXRGsrM3NqMGVxNXRhMEJrWmczY1NuQzUwVDJxU01IRmxwTzRy?=
 =?utf-8?B?MXR5TVYwQ0VkTUlGdmdUSUx6cTMwSkZSNmdxejR6RkpVTktmWnp5SlIwV1BZ?=
 =?utf-8?B?b0hJRGNxaXBUUE5yV245aTVwUmtHZ2xyNTc1czVrQ29VSmdZUWV6TVl4aHhx?=
 =?utf-8?B?dmp3RDBOT1RZYkxZZmFISm1BSjJDT3phT29JazBIOXZ0TXYvZThDMGwrd0Fh?=
 =?utf-8?B?ZkRpQWQ0VU8yN0xnSEtBUEtCdzFzVXdNQXJ3SDRod283UmdwcU5iWFI4WFlW?=
 =?utf-8?B?QUgwNCtiTUdLdXg0M0RUSzlIU3E5NzFtMlNkdHo5ajh4T25jNmNLZXc3NkUy?=
 =?utf-8?B?T29WV1Y5T1VhN2k0bHoyZkI2TlAvRlFQbkNXK1gzSkJ3elhCRDVoL25RcXpq?=
 =?utf-8?B?anBudEIxSUJzR1FxRUg5Zlh2U2NtUXEzTEk5TkNLSjlvdlhET0RzNTRDTW5T?=
 =?utf-8?B?YXY1QUFmTytxZ2lzUmdUWnl3RFF6WXMxbng0TmpsRGxvZHFTV0N4VldFL09I?=
 =?utf-8?B?YWJiUEJnOTNoZGxwdjYzWTBXQ1V0RzArTHY4d2duZ0YrT2VPZXFMc0l6UDBN?=
 =?utf-8?B?ZU5KejdrV2JzWXFoeFdydE9lakN5cFVpU2EvdFl1M294clFwOCtMQXhJajlu?=
 =?utf-8?B?alBPT0tYWCtiTTh1WmxmSWdqL09ycmVUd3pwNWMzVHVQYUptUCtBczRJZ1g5?=
 =?utf-8?B?VGszOVNScE04V0lDdzM1cDVKOFRucUdQMllVRlh5ck1PWDdqZFJnSE5qMnlr?=
 =?utf-8?B?M21yZC95L1Bsc3VVZ2kySTEvNkw3d1lDTmd1WkdXWFRFUXJDWGdsZmdjYmRw?=
 =?utf-8?B?U3NwTk1JOFFLRkhwVTJPeGQ2KzNJclp3c2oxNmhuU2RCQVU3WVNFa1JPQlI3?=
 =?utf-8?B?NnlIcDcrY08zeDBDYmk0SlZ3OUsrTUpsc3ZLVjNQNUwrTmFtZTFxTkd3S29U?=
 =?utf-8?B?VW40M1VkcnJ1aTB5aFNUeWl3d2N4aUo4cUluVE1MeWxUK0NqMW16L3B1dDVI?=
 =?utf-8?B?aUV0dndJenVneGlBd0Zydkt6U2RaN29lbEt1cmNVZEdrL0dkUU9wcDdHOGo5?=
 =?utf-8?B?VXUwKzhFc3dCeUxWOHpLL2pLdHlhTm00eHE1b3F0Qy9zNnpzaXNHbE5HWnhJ?=
 =?utf-8?B?a001dFVlcFZsVlZBWmQ2WVIvbHR3VkYwZTBLNmREek0wb1QzeExrbWZTSUkv?=
 =?utf-8?B?dndBeFVsdmNtaFVFcXV0TjRxWUxINDJBMG0rK3N6UDlyWUZnVkpEbEQwVmFs?=
 =?utf-8?B?clVBTElSR2hqdVJwS2FnNk4zelBoTEJnUDVtdzlFWDdtMnloR0pvMFAvMGht?=
 =?utf-8?B?MUhGY3FQMWpMaFpSbTVwMEhmQnFpMjc2aE5RMmI2WVJkWkZqMnZyaTBHQ3dE?=
 =?utf-8?B?TkowU0FXU3RjYndJNmNqQksrdlBrYzhadWhBd3hWd3N3NWR0UkxUMWVNQ1dW?=
 =?utf-8?B?UVJoRmkzdHEyZHhubExmSFpSNVduZ042MC8vZlBzVko5bWhGMFBQWDN3VFE0?=
 =?utf-8?B?R2kwY2ZYMjI2YlBZWC9PbHJzdVFIMWdsTGk1VTlHRjlyanVQWjVtcDdLb2xT?=
 =?utf-8?B?anc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: b5752ccb-499a-49d3-7e47-08dbf135b66f
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Nov 2023 23:48:40.0939
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: D7jBfkveaYLDgiiMuFH6GsqUX4R5suGBBhe1pqWT4inFrHsijPpRtFIF/sumCxJd9ducv2hn4LAjOG+0dx+I/587v/GLwWQa2dmaasOJl7Y=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR11MB7334
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on agentk.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (agentk.vger.email [0.0.0.0]); Wed, 29 Nov 2023 15:48:58 -0800 (PST)

Hi Tony,

On 11/28/2023 3:14 PM, Tony Luck wrote:
> Add a "total" mount option to be used in conjunction with "mba_MBps"
> to request use of the total memory bandwidth event as the feedback
> input to the control loop.

"total" is very generic. It is also not clear to me why users
would need to use two mount options. What if the new mount option
is "mba_MBps_total" instead, without user needing to also provide
"mba_MBps"? 

> 
> Also fall back to using the total event if the local event is not
> supported by the CPU.
> 
> Update the once-per-second polling code to use the event (local
> or total memory bandwidth).

Please take care to describe why this change is needed, not just
what it does. This is required by x86. For confirmation:
https://lore.kernel.org/lkml/20231009172517.GRZSQ3fT05LGgpcW35@fat_crate.local/

> 
> Signed-off-by: Tony Luck <tony.luck@intel.com>
> ---
> 
> Changes since v3:
> 
> Reinette suggested that users might like the option to use the total
> memory bandwidth event. I tried out some code to make the event runtime
> selectable via a r/w file in the resctrl/info directories. But that
> got complicated because of the amount of state that needs to be updated
> when switching events. Since there isn't a firm use case for user
> selectable event, this latest version falls back to the far simpler
> case of using a mount option.

(I did not realize that that discussion was over.)

> 
>  Documentation/arch/x86/resctrl.rst     |  3 +++
>  arch/x86/kernel/cpu/resctrl/internal.h |  3 +++
>  arch/x86/kernel/cpu/resctrl/monitor.c  | 20 +++++++++-----------
>  arch/x86/kernel/cpu/resctrl/rdtgroup.c | 15 ++++++++++++++-
>  4 files changed, 29 insertions(+), 12 deletions(-)
> 
> diff --git a/Documentation/arch/x86/resctrl.rst b/Documentation/arch/x86/resctrl.rst
> index a6279df64a9d..29c3e7137eb8 100644
> --- a/Documentation/arch/x86/resctrl.rst
> +++ b/Documentation/arch/x86/resctrl.rst
> @@ -46,6 +46,9 @@ mount options are:
>  "mba_MBps":
>  	Enable the MBA Software Controller(mba_sc) to specify MBA
>  	bandwidth in MBps
> +"total":
> +	Use total instead of local memory bandwidth to drive the
> +	MBA Software Controller
>  "debug":
>  	Make debug files accessible. Available debug files are annotated with
>  	"Available only with debug option".
> diff --git a/arch/x86/kernel/cpu/resctrl/internal.h b/arch/x86/kernel/cpu/resctrl/internal.h
> index a4f1aa15f0a2..f98fc9adc2da 100644
> --- a/arch/x86/kernel/cpu/resctrl/internal.h
> +++ b/arch/x86/kernel/cpu/resctrl/internal.h
> @@ -59,6 +59,7 @@ struct rdt_fs_context {
>  	bool				enable_cdpl2;
>  	bool				enable_cdpl3;
>  	bool				enable_mba_mbps;
> +	bool				use_mbm_total;
>  	bool				enable_debug;
>  };

Why did you choose new member to not follow existing custom of having
an enable_ prefix?

>  
> @@ -428,6 +429,8 @@ extern struct rdt_hw_resource rdt_resources_all[];
>  extern struct rdtgroup rdtgroup_default;
>  DECLARE_STATIC_KEY_FALSE(rdt_alloc_enable_key);
>  
> +extern enum resctrl_event_id mba_mbps_evt_id;
> +

This global seems unnecessary. struct resctrl_membw.mba_sc indicates if
the software controller is enabled. Creating this global fragments
related information.

One option could be to change the type of struct resctrl_membw.mba_sc to
enum resctrl_event_id. I assume that 0 would never be a valid event ID and
can thus be used to know if the software controller is disabled. If this
is done then enum resctrl_event_id's documentation should be updated
with this assumption/usage.

Reinette