Received-SPF: pass (google.com: domain of linux-kernel+bounces-79227-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <21d05834-ca2f-8094-501d-f243e74cdd26@amd.com>
Date: Fri, 23 Feb 2024 15:47:17 -0600
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Reply-To: babu.moger@amd.com
Subject: Re: [PATCH v2 00/17] x86/resctrl : Support AMD Assignable Bandwidth
 Monitoring Counters (ABMC)
Content-Language: en-US
To: Peter Newman <peternewman@google.com>, James Morse <james.morse@arm.com>
Cc: babu.moger@amd.com, Reinette Chatre <reinette.chatre@intel.com>,
 corbet@lwn.net, fenghua.yu@intel.com, tglx@linutronix.de, mingo@redhat.com,
 bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com,
 paulmck@kernel.org, rdunlap@infradead.org, tj@kernel.org,
 peterz@infradead.org, yanjiewtw@gmail.com, kim.phillips@amd.com,
 lukas.bulwahn@gmail.com, seanjc@google.com, jmattson@google.com,
 leitao@debian.org, jpoimboe@kernel.org, rick.p.edgecombe@intel.com,
 kirill.shutemov@linux.intel.com, jithu.joseph@intel.com,
 kai.huang@intel.com, kan.liang@linux.intel.com,
 daniel.sneddon@linux.intel.com, pbonzini@redhat.com, sandipan.das@amd.com,
 ilpo.jarvinen@linux.intel.com, maciej.wieczor-retman@intel.com,
 linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, eranian@google.com
References: <20231201005720.235639-1-babu.moger@amd.com>
 <cover.1705688538.git.babu.moger@amd.com>
 <7c26af23-fa1e-4e01-8088-8fbd9be3d6f3@intel.com>
 <431d6ac4-53cb-2f73-3cda-22616df2f96a@amd.com>
 <4bb63a78-0d0d-47bc-ad65-558af8bc5519@intel.com>
 <51c60991-eb10-40e8-b3ab-676b92b0c662@amd.com>
 <CALPaoChhKJiMAueFtgCTc7ffO++S5DJCySmxqf9ZDmhR9RQapw@mail.gmail.com>
 <1a8c1cd6-a1ce-47a2-bc87-d4cccc84519b@arm.com>
 <CALPaoCgNLtA7E2tgQZ6gmbZ=OF0nE0Lbi=1C7oR3F0wM4YRbjw@mail.gmail.com>
From: "Moger, Babu" <bmoger@amd.com>
In-Reply-To: <CALPaoCgNLtA7E2tgQZ6gmbZ=OF0nE0Lbi=1C7oR3F0wM4YRbjw@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Ty9jQU5qaVJ0UlJpOGI3UFhrVnVuZTk5Q0VwVzBrZW1OdUlDUEQzeXdMMmVX?=
 =?utf-8?B?emxFREpuWWJYVzM1VCtPSFpSQisyZXVpY3RuR2tMeUMzZWFLZk5wL3lzdjk0?=
 =?utf-8?B?dlp3OWpwVzZJOEsyUmtKYld4Vm43QjRhMjNmZFViWXlCOGJnSHg2K0t1QmhP?=
 =?utf-8?B?cm9hVUxyald3cDVPSEVMTWY2dHpvanZoTEVra3RkNlhaTHgycnV3UHFxMTBk?=
 =?utf-8?B?VUlnNG5sb0RvdVVBSlQ1Z1YxZFdNMVVzSmN6aXZlWnlZaVRXR1VOWE1ZYzN6?=
 =?utf-8?B?a3RGblFpNGxRbHJ5MkhlWkVUVFVvQ1lMcHd4b2FiZFJBTE5BdlU0M2Z5Wkcw?=
 =?utf-8?B?b3pwQm04WFUvUm9VUU40N0JyRnRRZTVrcWlodW1vTlpxQyt0eUhhamFpWDlL?=
 =?utf-8?B?YUtJOTJpd29wdVc5YzJ1aUl3Nno2dFlEWDNra29EckNPcVFSanBrcGNoRURH?=
 =?utf-8?B?NERzOW10bitPWFF5OGJHSGlINWM4OWhPaFhpdTk4SjA2a015ajlTeHB6RFhw?=
 =?utf-8?B?NmdCTHpRcGVxdXA2VERvYVBtYkg5ZjNDVnRuZExlWnB5RUhsaTd1MmtSRFdF?=
 =?utf-8?B?RHBldFZXaU1sUTVhbE1HaEdJOFNIN3JwUlRnVS9qNzI5Wjc3R3FDd3VWbmRF?=
 =?utf-8?B?bjFxY2JyWG9CbFFyY0dubmovSFhra2x3ZGVlOE8vamo5cUg3QlVoSHVlMml6?=
 =?utf-8?B?bml0T2ttU290SlVYOVpaZWZFUU10Z0xBM2NyeDhzZGM1MnJRalBrbk9wMUNs?=
 =?utf-8?B?cEdsRjRvMC9YMXlQMG8xbHkrOG9XVG4yTGxLMitHSTN6WkNXa1lFd2ZXSWtj?=
 =?utf-8?B?U05KQ2pRdStOMFJ3VEdqaFVBNTh5b2psdkRmRWszV1BoRVM1ZTdYNGlsQjhP?=
 =?utf-8?B?Mm80Mk9XZ3RpYWMzV2FCNTRWaGtWcWpqZzd4cHc1ZllSbUV0VStXTU8ybXRW?=
 =?utf-8?B?OXFPaXNkVldUY3E3ODNpSWlTSXNFZFNBcVZUUEFGMDNTay84L3JQUEJxbmRu?=
 =?utf-8?B?NzZhOW10MDdkRjEva1p2NmR5Qndyb2dFeXcvUnJBMks2endLVzJhbERxSktp?=
 =?utf-8?B?NzIxV0poZExqcGpObmZiaUJpaGJ6YzNOV1orTmxHTldoL0hPand0QS9Sd1BE?=
 =?utf-8?B?WTlLNFVaUi9OdHhiTVovNy92c2RWZ2p4RWpZNDdTQi9nMnlFV1JvOVROZllu?=
 =?utf-8?B?ZUE0K1dEeXhKRFkrUVVrMWRwSnZSbXowdCtBWXJRaTAxUHJiZXVPV09rdXJa?=
 =?utf-8?B?WjNzQUF2Nmk2MUtHQ3JMbjdRUEhtNHVBK09hNGhnNGhJWE9QUitFWk1TRE53?=
 =?utf-8?B?REFuNkVwcXB6c2pHOUFFSGhsVG9jRjM4Y3dqZS9nYlpvNjlrVitkQkt6ejFp?=
 =?utf-8?B?am1CUEZ4Skw2cHNwMmE0clJFVi90UmwrQ3BUeEhrOTR6NFVtNzB2V2sxSFhi?=
 =?utf-8?B?b3VTN09WTGs0V01mamVQWUljWW5zL3RHQ1BUR2IyQkxHc1hSMGc4eStpMVMy?=
 =?utf-8?B?VVRHektDTkc4VDk1cU1sdnQ3QlZEaVVzTHVxQWxSKzhMZ01xcWlTUWlnRXFN?=
 =?utf-8?B?ZzhkSG4zdlhYalpIUVBnYWgxZmpIZlE5V2trSHJ5cWxIV3Bxczh0THpuRDNy?=
 =?utf-8?B?S2ZNSHJ5VjMrU0hNQzlFZ3RsTDQxZzZYUDlxUnF6WFR4VDdGYk5OT3o3L2dv?=
 =?utf-8?B?bkhsV1d4OVVFTVVpWUl3YTViMEVnRy9sYitZbnR2Y2E4eTh2TzFvWGlJZnJ4?=
 =?utf-8?B?U25kY0dzd2lmMnBBaERDcHFuN3BwODlTY3g0Wko1UFFVTFlUT2lUM2YzZmd5?=
 =?utf-8?B?R0EyM3VXNjhZcldweWJGd3pLU2k2L0xOeDZ1eFNTcC9MZnU2SnY5WjM3ZnZy?=
 =?utf-8?B?ME81QUkvdWM0NlRHYzBBNHVpc0ZPcWxBNnI2RE44RlUrMDlhQ0xhQmt2Mm5i?=
 =?utf-8?B?MVRVUC9qREVGYnF4OGs0K3FtdmhLMHZhc0VaQU5sL0FPK1FDRDN0cXV6cDB4?=
 =?utf-8?B?UjREZmxDWHU5dnBQenlpR3B3VWJyWTE5QlpCR3cwRzRJMlZYUHBHQ1BhVTBN?=
 =?utf-8?B?eUJNQWRuZmxtNTQ5SEVpdHB3WitWUWpzc205bkpQaGtheXlCSUQydmdkc0k1?=
 =?utf-8?B?N1RvYkNFOFV0VWVtc1YvZ2gzV1Q1RkhrNFZzbGE0eDlVWmN6TUlPOTVjMkFh?=
 =?utf-8?Q?KpSFPGdpOziTH+VdejYi5gNA+zN5UmHngmtiGSJL4Cz7?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 77dd9e50-c230-4255-d97a-08dc34b90406
X-MS-Exchange-CrossTenant-AuthSource: MW3PR12MB4553.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Feb 2024 21:47:22.2500
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: YaaKEpqmLuIT6b/Jxv+hEmX6h10aDNr7NGdcgdQyDDcYrQAQ8r7ejVFaYqnojKOQ
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB5839


On 2/20/2024 12:11 PM, Peter Newman wrote:
> Hi James,
>
> On Tue, Feb 20, 2024 at 7:21 AM James Morse <james.morse@arm.com> wrote:
>> On 16/02/2024 20:18, Peter Newman wrote:
>>> On Thu, Feb 8, 2024 at 9:29 AM Moger, Babu <babu.moger@amd.com> wrote:
>>>> On 2/5/24 16:38, Reinette Chatre wrote:
>>>>> You have made it clear on several occasions that you do not intend to support
>>>>> domain level assignment. That may be ok but the interface you create should
>>>>> not prevent future support of domain level assignment.
>>>>>
>>>>> If my point is not clear, could you please share how this interface is able to
>>>>> support domain level assignment in the future?
>>>>>
>>>>> I am starting to think that we need a file similar to the schemata file
>>>>> for group and domain level monitor configurations.
>>>> Something like this?
>>>>
>>>> By default
>>>> #cat /sys/fs/resctrl/monitor_state
>>>> default:0=total=assign,local=assign;1=total=assign,local=assign
>>>>
>>>> With ABMC,
>>>> #cat /sys/fs/resctrl/monitor_state
>>>> ABMC:0=total=unassign,local=unassign;1=total=unassign,local=unassign
>>> The benefit from all the string parsing in this interface is only
>>> halving the number of monitor_state sysfs writes we'd need compared to
>>> creating a separate file for mbm_local and mbm_total. Given that our
>>> use case is to assign the 32 assignable counters to read the bandwidth
>>> of ~256 monitoring groups, this isn't a substantial gain to help us. I
>>> think you should just focus on providing the necessary control
>>> granularity without trying to consolidate writes in this interface. I
>>> will propose an additional interface below to optimize our use case.
>>>
>>> Whether mbm_total and mbm_local are combined in the group directories
>>> or not, I don't see why you wouldn't just repeat the same file
>>> interface in the domain directories for a user needing finer-grained
>>> controls.
>> I don't follow why this has to be done globally. resctrl allows CLOSID to have different
>> configurations for different purposes between different domains (as long as tasks are
>> pinned to CPUs). It feels a bit odd that these counters can't be considered as per-domain too.
> Assigning to all domains at once would allow us to better parallelize
> the resulting IPIs when we do need to iterate a small set of monitors
> over a large list of groups.

Planning to work on v3 of this series. For now, I will exclude the 
global assignment option from this series.

We can add the global assignment support when we get time to optimize 
assignments at later point.

Thanks

Babu Moger