Received-SPF: pass (google.com: domain of linux-kernel+bounces-96212-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Message-ID: <1d3e8b68-28fa-419b-b129-0c39df34f718@intel.com>
Date: Thu, 7 Mar 2024 12:41:08 -0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 00/17] x86/resctrl : Support AMD Assignable Bandwidth
 Monitoring Counters (ABMC)
Content-Language: en-US
To: Peter Newman <peternewman@google.com>, <babu.moger@amd.com>
CC: James Morse <james.morse@arm.com>, <corbet@lwn.net>,
	<fenghua.yu@intel.com>, <tglx@linutronix.de>, <mingo@redhat.com>,
	<bp@alien8.de>, <dave.hansen@linux.intel.com>, <x86@kernel.org>,
	<hpa@zytor.com>, <paulmck@kernel.org>, <rdunlap@infradead.org>,
	<tj@kernel.org>, <peterz@infradead.org>, <yanjiewtw@gmail.com>,
	<kim.phillips@amd.com>, <lukas.bulwahn@gmail.com>, <seanjc@google.com>,
	<jmattson@google.com>, <leitao@debian.org>, <jpoimboe@kernel.org>,
	<rick.p.edgecombe@intel.com>, <kirill.shutemov@linux.intel.com>,
	<jithu.joseph@intel.com>, <kai.huang@intel.com>, <kan.liang@linux.intel.com>,
	<daniel.sneddon@linux.intel.com>, <pbonzini@redhat.com>,
	<sandipan.das@amd.com>, <ilpo.jarvinen@linux.intel.com>,
	<maciej.wieczor-retman@intel.com>, <linux-doc@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>, <eranian@google.com>
References: <20231201005720.235639-1-babu.moger@amd.com>
 <2f373abf-f0c0-4f5d-9e22-1039a40a57f0@arm.com>
 <474ebe02-2d24-4ce3-b26a-46c520efd453@amd.com>
 <b6bb6a59-67c2-47bc-b8d3-04cf8fd21219@intel.com>
 <3fe3f235-d8a6-453b-b69d-6b7f81c07ae1@amd.com>
 <9b94b97e-4a8c-415e-af7a-d3f832592cf9@intel.com>
 <1ae73c9a-cec4-4496-86c6-3ffcef7940d6@amd.com>
 <32a588e2-7b09-4257-b838-4268583a724d@intel.com>
 <088878bd-7533-492d-838c-6b39a93aad4d@amd.com>
 <9b20589b-6220-4ae7-bfc4-4a826b7114b1@intel.com>
 <5ddb7031-a828-4001-bfdf-5477cb85e9ed@amd.com>
 <1738493b-3248-4c9e-82a8-1599a033440d@intel.com>
 <369ab28a-f3fa-4359-8e73-4dcf214c9b6e@amd.com>
 <54687d59-d0e4-4fe7-b25f-dc1fead01ea1@intel.com>
 <11487a31-908e-d474-50c6-65617d417deb@amd.com>
 <c73f444b-83a1-4e9a-95d3-54c5165ee782@intel.com>
 <55b545fd-2851-0d0f-ac37-ec59838fb4b4@amd.com>
 <1f366890-d9ff-4ac4-9af9-1ea3128a87fc@intel.com>
 <41ca7504-c3fb-ddb6-e149-4ff82019d678@amd.com>
 <CALPaoCi=PCWr6U5zYtFPmyaFHU_iqZtZL-LaHC2mYxbETXk3ig@mail.gmail.com>
From: Reinette Chatre <reinette.chatre@intel.com>
In-Reply-To: <CALPaoCi=PCWr6U5zYtFPmyaFHU_iqZtZL-LaHC2mYxbETXk3ig@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?c0hsaUo1VFhFb1NYUTNtMlZEeUgwYkdMbnVoTWRPTHlrTVdPQlhIUDVrc2RC?=
 =?utf-8?B?bit0bkRxLzd4MkZXZzRCUEhLTm9kb0xoVDdiNUdzZkN5L1Boc0dyWGdyQ3BH?=
 =?utf-8?B?Z2xZVlV6ZlFNNGhSM1JSUm94M2RxMk1LUEh2MGszT05iVEZQZnU1dEVYcm52?=
 =?utf-8?B?cUZwU1h1dDhuZXVkbTRDZEd0aXFwbHZ3Ymd1NTk4WFNSYzBwcFZuOXg2RDRl?=
 =?utf-8?B?UmcrVVRFSEJwYWdxZWF3OVBveTRPdk16U3hmRUcyQk1tc2dIaWxXR3V6clFE?=
 =?utf-8?B?ZTAwUWp2TjRIcER6VDYzR0JkWDBhak5SbkNoK0dPNVRneTdPRVRJZDVDOVpM?=
 =?utf-8?B?RXhBN0YzWFREYzl1U1NlckkrWm5ucFhXTlpQNTkxbGM3VkJxVmVYcTBRTTFn?=
 =?utf-8?B?eGd5eDZCajNoNWlnQ3pWK2pYMGJ4Q1RrVzVXbnRQbzNZWVEyUzJSbHVNTmxY?=
 =?utf-8?B?TzF5UnF1QlFkSTUyLzdwRnY0Mk0rck9JZm9zdEVWcmtCQ1RKNTZxQWZobUtQ?=
 =?utf-8?B?V2FmYzhPeS9Hcjd6Sm00ZVVZQ0JvdUp4TDdPajVMWUtldG5xb1Iyc2gxWUtK?=
 =?utf-8?B?NkZjbmtiNFZRTmtURFo0Tm9menVENlpMdEZ2UTlobDFLa3FjaUJYSEZQQy9t?=
 =?utf-8?B?dWNJa3dzRzJ6ZEJmdUkwWDFwcWMxYVpwUkF3bi9wdUFxVzA3cysvZ05IYlAv?=
 =?utf-8?B?YVpRMktqRmpTei9HaXpadWV5REZLUGxEU1IwckVQZ0lOZVY4WUZEQWpxaFZx?=
 =?utf-8?B?TDN1S2JicStvVlJIWTlaTTNlMEpFMWVBWHBzTTlmMmpoeC9iWEppQy9Zc3cx?=
 =?utf-8?B?ajV6UGp2TVJLNHJ3Z09vQXdTUnVWRmVFL0hQbzFTOWhFZStZQzFZMksydHBE?=
 =?utf-8?B?c1JMc1FibTVTcm81Qzc1NWFlRFhaaGNlREp3VjNvTER0RHNNcFgwRHRGeHFr?=
 =?utf-8?B?RXVSWTJDVmlCcHNXaXNaNU1xa2pBd3A4QVM1NDRvMVJSMXVBTTNMd0N4empk?=
 =?utf-8?B?elE4RlhYNE5IUVNSeG0wOEdLcVVSTFdzUE9YY25BSEhIajMvNTVKckkzN3Ez?=
 =?utf-8?B?UXE2cWJsZVZvUjhFUW9heHpsUEdNTC9LdmQrM29UZDNJb29KWnJEUjBXN1Na?=
 =?utf-8?B?bUVZTmpBLy9rNnE5QWpkTmlaTkZxMGdCOHd4eE4wT01lVzhOU0ZWK0FNaGM3?=
 =?utf-8?B?RzN2d3B2K0VuZ0kxaytzSVBYcmN2L2lpM3AzWHFnYVJnYmxYNGE2QTY3TDYz?=
 =?utf-8?B?Z3BlZHo4ZVY4UWNsS1p3TFRwNXZ6SFBkUklMb0JTN05XeEJvcUJJcHU2RmEy?=
 =?utf-8?B?ZThacnhQNEZEMThwREdSeElwR0haaURvYkM2ejltOE8zRmFsV3NVNkFPT2RR?=
 =?utf-8?B?ME9hVWJvY0dLenlGQUVBUkx2a3ZkOXZ0V2xWdHJTMDFyd2l5NHk3MWRwZVBS?=
 =?utf-8?B?ZlRzMFJ3VjczL1pjaFJpM0JBOG1HZTNrWERqaGVseWVjY1FkeVJlcUxndnFJ?=
 =?utf-8?B?STFLTXh1K2IyNU9LdHg0U2FSaTFsRi9ndjRlK3Z4ZkdTQlJEOTBROEFjcG5y?=
 =?utf-8?B?ckptZ0tCNWZibWVleFAxS3hiNXVSMm9iNkhWbTlJMWlCUHJxSjc3NGR0Qk1X?=
 =?utf-8?B?UDY5KzkzeU92ZEs5bUttMG5RTFh0aUJaYlYyREhIeTR6UkxwQW40eURoMkNY?=
 =?utf-8?B?b1dYS1lFNTFBRlZoUk9maENHblNYN2VzWXEzYngzZW8zZGF5UFFsSWRNTXhj?=
 =?utf-8?B?dkNJQ0RIUmNoWFlxTU5aMmdaYjNZQVVPMEF4NFUvNTlxSXY1K0I2S3Q2enNZ?=
 =?utf-8?B?TFlBTWtxbndZam9OUnlWK2ZZYVFOUWVEYzdMYW5GTWdjcG4wTnNvN2hpQklp?=
 =?utf-8?B?VlYyZTg4cWhNNWU2QlVycGxUQjQ0R0tyS28rOWdUL2VmeGxvK0k3M0JEMkJP?=
 =?utf-8?B?TWU2ejlKT2lqdy9OWExFQ0QydWxQSisyZWVmNXIyZm1LZzNzTUxaWTkrSTBS?=
 =?utf-8?B?bmQzbkMyWk9GNVFmTE9pUzA1Z3VZSGJPSjJsMktpRXVBV0FIcUpDT29CN0JY?=
 =?utf-8?B?a1NlZmk0ZldJZHMxbi9lRkJqR2RpTy9SY2dLT3hCbTZVc3dVZjV3amFrT3lu?=
 =?utf-8?B?VG9xVzVJQ0RDbjlzdjl2bjN3TTQxdXdpYVdkZDJWb2d0VWJFM200TlpRRHZt?=
 =?utf-8?B?cVE9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: aaa3ec98-33b5-4eeb-32f5-08dc3ee6ed1e
X-MS-Exchange-CrossTenant-AuthSource: SJ2PR11MB7573.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Mar 2024 20:41:12.2932
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: TDB3Dl1uEvDaXSgL0pT3pAdqQsWn7YDu8LcoIXTTQD3H9PmKV9bGoIlygwllRfNQIX1x8fvWB5tmSI3S8JXpaB8oa76a83IOZ+sdQV4VNX0=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA0PR11MB7377
X-OriginatorOrg: intel.com

Hi Peter,

On 3/7/2024 10:57 AM, Peter Newman wrote:
> Hi Babu,
> 
> On Mon, Mar 4, 2024 at 2:24 PM Moger, Babu <bmoger@amd.com> wrote:
>> Based on our discussion, I am listing few examples here. Let me know if
>> I missed something.
>>
>>    mount  -t resctrl resctrl /sys/fs/resctrl/
>>
>> 1. Assign both local and total counters to default group on domain 0 and 1.
>>     $echo "//00=lt;01=lt" > /sys/fs/resctrl/info/L3_MON/mbm_assign_control
>>
>>     $cat /sys/fs/resctrl/info/L3_MON/mbm_assign_control
>>     //00=lt;01=lt
>>
>> 2. Assign a total event to mon group inside the default group for both
>> domain 0 and 1.
>>
>>     $mkdir /sys/fs/resctrl/mon_groups/mon_a
>>     $echo "/mon_a/00+t;01+t" >
>> /sys/fs/resctrl/info/L3_MON/mbm_assign_control
>>
>>     $cat /sys/fs/resctrl/info/L3_MON/mbm_assign_control
>>     //00=lt;01=lt
>>     /mon_a/00=t;01=t
>>
>> 3. Assign a local event to non-default control mon group both domain 0
>> and 1.
>>     $mkdir /sys/fs/resctrl/ctrl_a
>>     $echo "/ctrl_a/00=l;01=l"  >
>> /sys/fs/resctrl/info/L3_MON/mbm_assign_control
>>
>>     $cat /sys/fs/resctrl/info/L3_MON/mbm_assign_control
>>     //00=lt;01=lt
>>     /mon_a/00=t;01=t
>>     /ctrl_a/00=l;01=l
>>
>> 4. Assign a both counters to mon group inside another control
>> group(non-default).
>>     $mkdir /sys/fs/resctrl/ctrl_a/mon_ab/
>>     $echo "ctrl_a/mon_ab/00=lt;01=lt" >
>> /sys/fs/resctrl/nfo/L3_MON/mbm_assign_contro
>>
>>     $cat /sys/fs/resctrl/info/L3_MON/mbm_assign_control
>>     //00=lt;01=lt
>>     /mon_a/00=t;01=t
>>     /ctrl_a/00=l;01=l
>>     ctrl_a/mon_ab/00=lt;01=lt
>>
>> 5. Unassign a counter to mon group inside another control
>> group(non-default).
>>     $echo "ctrl_a/mon_ab/00-l;01-l" >
>> /sys/fs/resctrl/nfo/L3_MON/mbm_assign_control
>>
>>    $cat /sys/fs/resctrl/info/L3_MON/mbm_assign_control
>>    //00=lt;01=lt
>>    /mon_a/00=t;01=t
>>    /ctrl_a/00=l;01=l
>>    ctrl_a/mon_ab/00=t;01=t
>>
>> 6. Unassign all the counters on a specific group.
>>     $echo "ctrl_a/mon_ab/00=_" >
>> /sys/fs/resctrl/nfo/L3_MON/mbm_assign_control
>>
>>     $cat /sys/fs/resctrl/info/L3_MON/mbm_assign_control
>>     //00=lt;01=lt
>>     /mon_a/00=t;01=t
>>     /ctrl_a/00=l;01=l
>>     ctrl_a/mon_ab/00=_;01=_
> 
> The use case I'm interested in is iterating 32 counters over 256
> groups[1]. If it's not possible to reassign 32 counters in a single
> write system call, with just one IPI per domain per batch reassignment
> operation, then I don't see any advantage over the original proposal
> with the assignment control file in every group directory. We already
> had fine-grained control placing assign/unassign nodes throughout the
> directory hierarchy, with the scope implicit in the directory
> location.

The intent of this interface is to support modification of several
groups with a single write. These examples only show impact to a single
group at a time, but multiple groups can be modified by separating
configurations with a "\n". I believe Babu was planning to add some
of these examples in his next iteration since it is not obvious yet.

> 
> The interface I proposed in [1] aims to reduce the per-domain IPIs by
> a factor of the number of counters, rather than sending off 2 rounds
> of IPIs to each domain for each monitoring group.

I understood the proposed interface appeared to focus on one use case
while the goal is to find an interface to support all requirements. 
With this proposed interface it it possible to make large scale changes
with a single sysfs write.

Reinette