Received-SPF: pass (google.com: domain of linux-kernel+bounces-184045-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <dda69695-0de4-4d0c-936a-128c6c1948f0@amd.com>
Date: Mon, 20 May 2024 13:03:23 -0500
User-Agent: Mozilla Thunderbird
Reply-To: babu.moger@amd.com
Subject: Re: [RFC PATCH v3 00/17] x86/resctrl : Support AMD Assignable
 Bandwidth Monitoring Counters (ABMC)
To: Peter Newman <peternewman@google.com>
Cc: Reinette Chatre <reinette.chatre@intel.com>, corbet@lwn.net,
 fenghua.yu@intel.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
 dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com,
 paulmck@kernel.org, rdunlap@infradead.org, tj@kernel.org,
 peterz@infradead.org, yanjiewtw@gmail.com, kim.phillips@amd.com,
 lukas.bulwahn@gmail.com, seanjc@google.com, jmattson@google.com,
 leitao@debian.org, jpoimboe@kernel.org, rick.p.edgecombe@intel.com,
 kirill.shutemov@linux.intel.com, jithu.joseph@intel.com,
 kai.huang@intel.com, kan.liang@linux.intel.com,
 daniel.sneddon@linux.intel.com, pbonzini@redhat.com, sandipan.das@amd.com,
 ilpo.jarvinen@linux.intel.com, maciej.wieczor-retman@intel.com,
 linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, eranian@google.com,
 james.morse@arm.com
References: <cover.1711674410.git.babu.moger@amd.com>
 <CALPaoCjZ3oLdKymJjASt0aqtd0GGOme7LavvYOtPYTb_rA-mYQ@mail.gmail.com>
 <b35dc4e9-7e8b-42ed-9a51-ae50d521cf4b@amd.com>
 <CALPaoChxYoJx8eR48EkSKf-hu2p2myQJLZEhj_Pq6O4R15-=5A@mail.gmail.com>
 <6edffe1b-e9a9-4995-8172-353efc189666@amd.com>
 <ab2a6a4b-3740-47c6-9443-e6bb7a0c1adb@intel.com>
 <CALPaoCiYFKeASPMDwzzaHLw4JiMtBB6DTyVPgt0Voe3c3Tav_A@mail.gmail.com>
 <b725e4ca-8602-eb26-9d47-914526621f52@amd.com>
 <CALPaoCiu2_UHyGjsyitz28BL1N93TSn28E1r-6nhXg--bzmU+Q@mail.gmail.com>
 <d7f3c5b1-c39d-4c66-92c3-5b096b9e0579@intel.com>
 <CALPaoCiJ9ELXkij-zsAhxC1hx8UUR+KMPJH6i8c8AT6_mtXs+Q@mail.gmail.com>
 <fae7fd93-27b7-4f94-964b-9c909f85f2fe@amd.com>
 <CALPaoCihfQ9VtLYzyHB9-PsQzXLc06BW8bzhBXwj9-i+Q8RVFQ@mail.gmail.com>
Content-Language: en-US
From: "Moger, Babu" <babu.moger@amd.com>
In-Reply-To: <CALPaoCihfQ9VtLYzyHB9-PsQzXLc06BW8bzhBXwj9-i+Q8RVFQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?aFNSUENFRTduZFhBSHFUQTllU2thUzNBNHY3ME02cmdvaTNNQ1NxOFBiaWRY?=
 =?utf-8?B?aVFKSFcwZWI3Vm1BZmkzZjVWNzlGaU04T0J2VUpuU05wNmpiMkFoaXZMSWNo?=
 =?utf-8?B?Ymd0b1NndlFnT1Bkb2QyUGx5dGE5bU5iTXRuQVZ4bUttUS93V2V0UWN3THYv?=
 =?utf-8?B?SlQ2R1VpeGNZTFBVbGxETW81eHFNOFR2Rk85aHkxbjJkUEpEb0ZJWVVJcytD?=
 =?utf-8?B?enE4VEtZK2ZxQmN6MVdSU0YzWGZ5Y0g0ZTdZZ3dLY2Y2UTFvekg5YlhEYnVU?=
 =?utf-8?B?SGRyOENoRVpJc25PQk1xZ3l0di9VVGd6ZVZ3TEJmbGxwbm8zZk45bG5TSTlU?=
 =?utf-8?B?KzJGaDNWd1pEWloweWE0Y1EvbTZGNkNtVWZ4MC94NC90a1NTNS93TkN4QWZn?=
 =?utf-8?B?czkrTkdtTXZzZTVtcnU2em1NZERzbjBhWmxmbDFUMWZFOW4renhoSGY0YkMy?=
 =?utf-8?B?NzBaTnNYMkJYMldNT1VTajFkUGNoNnJqNHlkQk9lSVI0OWJicE8zL1dTMjU4?=
 =?utf-8?B?VXZNQ2dmaHkyaTFURlFBelFOL0MwN3psenFYY2xWUk1uazZPK21EWEhZQ2xl?=
 =?utf-8?B?OWF1RXEwS3lZS3RkRG1nb0M0Skl2ZU5QWEl4NjRiVDAvczZKVXdWT3M1ZUxC?=
 =?utf-8?B?aGpqamVQelVONlJTei93M2lGaXlxdFdyVEJLMXVJMTBZb25XRW1OWmF0c1oz?=
 =?utf-8?B?czVyZGplV3hycHNuaG5ybWNES1BYQnlvSVc1d3FrSDZmTTM2dVJ6UU9Mbm1E?=
 =?utf-8?B?S3pON2x6VW9kaVh5YXZjR0s2Tm9NQmFHUjVRWW9QTHJ3ZWQ4UFE3bTNrVHFT?=
 =?utf-8?B?YStORGZuc2RHWEtUb2ZNbGlUVW1OdmFoZjBuelRxREhzWmtWeGp0b2pBZFZz?=
 =?utf-8?B?cWZhTWtSZmRVSEFKTGl4alFRMzhaS0Mrcy9NTWtWU2NEQ2NjSGRiRU1hVjlt?=
 =?utf-8?B?Y1NYZmI2YXh5TmM4dThuSzVGZHRKNlpZRTZzVnh0TE4ycmg2M1pEY3dCUHdM?=
 =?utf-8?B?amZKVFZPYytqUHNwRWtEQ0lwVURMM0liSXA2Vkk1UVk1SkxhdFdmQlNoTi94?=
 =?utf-8?B?MjRVMDk5eFN6MmV5eCtiZTFCMldVZmI4QS9VU2F3M1Jla2JLWStPUlJmQWhE?=
 =?utf-8?B?c1F5T2VOQ1Y2ajFjYVVqRjJ2ejVIWFRneWRQKzNNR2E2ZGh6L3VvOVR2MVo2?=
 =?utf-8?B?MG1zM2ZMUVNTNTc3VEdrTmJHTHl1RzVha1FYRGdiZE5PSGsxV0YxT3FXTVJP?=
 =?utf-8?B?ZlVKSHdOeHV1d2pCa0Fqc0ltSE02WUlpWTVIOFA2R3pXNjRLMEhDR1M4Ylpi?=
 =?utf-8?B?a0pkSFVqYklRRzVrYmxSVTEyUjNQeE5ocDErNXRFL2c4M3BjSkg0RG8ySHRZ?=
 =?utf-8?B?ZEQyYTdsRzBVS0k4VHd6c3FuNkF2bjdqc2dXV2xQRHFlSGgrK2JTdytJMFd4?=
 =?utf-8?B?MXVvWi9QUURJM3hNSVdsL0RnT3ZkakhjQUw2YXFJS2VWcVFYQlFPOEFYai9P?=
 =?utf-8?B?UERES3QwU0g0Y0NNaTcwQXU1N29Od0RHOFJYc2VPODVnZ3A4V25LRzM1TVMr?=
 =?utf-8?B?NXFGQ2E1VGFiMnZsOFE1NllIRkJqcXJTUGFaWmZ4OWd6bFNxTi9yL2N5dkNG?=
 =?utf-8?B?TExxSjZRb21pUEpKcEx3U1BqeVpMOEpMN2ZmcGIwZVhvMnFaVGF0dWlHNm1I?=
 =?utf-8?B?K2FNdUZzeUhmMStNUE1lWE84dm9oMHZqSjE5TWxNbE10WVNpMzh2eWN2RzUr?=
 =?utf-8?B?d0lpQ3pRUDIyMEx5R3pRZHRKVUNrTEtpMVcwSDRxVjl1bG1uS3lhbExDSVFZ?=
 =?utf-8?B?ZmNDNmxhdU9CZGcxNW1ENkR4ejFNNGpRdkFHTHI2ZGYyeDl6ZkY5Q1l1cjB2?=
 =?utf-8?B?c1hCK1Y3eGIrT3ZPd3AvSFJpb2drZmxSYkNpa0w0c3U2ZTQ4eXB2a0ZkWDF1?=
 =?utf-8?B?RnhtUXptR0ZaRGQ5YSs4V0hhSWNmSTBLbEtwU2JXd2Y1cjk0RllWYVF1ZzIw?=
 =?utf-8?B?Zkdlb1FWZVJXUFV1TDJmNC9wY2tjendRYTRwdlB4Vi9QcWV3aDY2T2oxMkk0?=
 =?utf-8?B?WEFWU1Vlay9TdytRWk03bVpYdzdiZFlzcGFUdXlVN21oSTB5bVcybVIzNEo3?=
 =?utf-8?Q?OIEY=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 736a1845-c4a1-45ae-7aa1-08dc78f72677
X-MS-Exchange-CrossTenant-AuthSource: MW3PR12MB4553.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 May 2024 18:03:27.9077
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: XdTKoCzjx4/yH3paYEJsetkQzwiZz8EA2gY9uQq9EhAfYc+Hr7CS6Nxw/4chatZY
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4169

Hi Peter,

On 5/20/24 11:00, Peter Newman wrote:
> Hi Babu,
> 
> On Mon, May 20, 2024 at 7:25 AM Moger, Babu <babu.moger@amd.com> wrote:
>>
>> Hi Peter,
>>
>> On 5/17/24 16:51, Peter Newman wrote:
>>> Hi Reinette, Babu,
>>>
>>> On Fri, May 3, 2024 at 2:15 PM Reinette Chatre
>>> <reinette.chatre@intel.com> wrote:
>>>>
>>>> Hi Peter,
>>>>
>>>> On 5/3/2024 2:00 PM, Peter Newman wrote:
>>>>> Hi Babu,
>>>>>
>>>>> On Fri, May 3, 2024 at 1:44 PM Moger, Babu <bmoger@amd.com> wrote:
>>>>>>
>>>>>> Hi Peter,
>>>>>>
>>>>>> On 5/2/2024 7:57 PM, Peter Newman wrote:
>>>>>>> Hi Reinette,
>>>>>>>
>>>>>>> On Thu, May 2, 2024 at 4:21 PM Reinette Chatre
>>>>>>>> I do think ABMC should be enabled by default when available and it looks
>>>>>>>> to be what this series aims to do [1]. The way I reason about this is
>>>>>>>> that legacy user space gets more reliable monitoring behavior without
>>>>>>>> needing to change behavior.
>>>>>>>
>>>>>>> I don't like that for a monitor assignment-aware user, following the
>>>>>>> creation of new monitoring groups, there will be less monitors
>>>>>>> available for assignment. If the user wants precise control over where
>>>>>>> monitors are allocated, they would need to manually unassign the
>>>>>>> automatically-assigned monitor after creating new groups.
>>>>>>>
>>>>>>> It's an annoyance, but I'm not sure if it would break any realistic
>>>>>>> usage model. Maybe if the monitoring agent operates independently of
>>>>>>
>>>>>> Yes. Its annoyance.
>>>>>>
>>>>>> But if you think about it, normal users don't create too many groups.
>>>>>> They wont have to worry about assign/unassign headache if we enable
>>>>>> monitor assignment automatically. Also there is pqos tool which uses
>>>>>> this interface. It does not have to know about assign/unassign stuff.
>>>>>
>>>>> Thinking about this again, I don't think it's much of a concern
>>>>> because the automatic assignment on mongroup creation behavior can be
>>>>> trivially disabled using a boolean flag.
>>>>
>>>> This could be a config option.
>>>
>>> I'd like to work out the details of this option.
>>>
>>> info/L3_MON/mbm_assign_on_mkdir?
>>>
>>> boolean (parsed with kstrtobool()), defaulting to true?
>>
>> I am thinking is not a big concern. We only have limited (32) counters.
>> Automatic monitor assignment works only for first 16 groups(2 counters for
>> each group). When the counters are exhausted auto assignment does not
>> work. In your case(with more than 16 groups) the auto assignment does not
>> work. I feel having a config option is really not necessary.
> 
> I'm not sure I follow the argument you're trying to make because it
> doesn't sound like an argument against adding a config option. What
> exactly do you mean by "work" vs "not work"?
> 
> Also it doesn't address my original concern about needing to manually
> (and non-atomically) undo the auto assignment in order to account for
> where the monitors are assigned or ensure that creating a new
> monitoring group will succeed.
> 

Sorry for the confusion.

Auto monitor assignment works only for small number of groups(15 or less).

After that point user can create more groups. But auto assignment will not
work because the hw counters are all exhausted. You need to manually
unassign a counter from another group and use that counter for new assignment.

I assume that you are dealing with more than 16 groups. In that case, you
have to manually assign/unassign anyways.

Having a config option "info/L3_MON/mbm_assign_on_mkdir" will not be much
helpful for you.

-- 
Thanks
Babu Moger