Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <1cc4ffae-3d36-b369-7cc9-b7130701b944@amd.com>
Date:   Mon, 7 Nov 2022 12:50:35 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.2.2
Reply-To: babu.moger@amd.com
Subject: Re: [PATCH v8 10/13] x86/resctrl: Add sysfs interface to write
 mbm_total_bytes_config
Content-Language: en-US
To:     Peter Newman <peternewman@google.com>
Cc:     akpm@linux-foundation.org, bagasdotme@gmail.com, bp@alien8.de,
        chang.seok.bae@intel.com, corbet@lwn.net,
        damien.lemoal@opensource.wdc.com, daniel.sneddon@linux.intel.com,
        dave.hansen@linux.intel.com, eranian@google.com,
        fenghua.yu@intel.com, hpa@zytor.com, james.morse@arm.com,
        jmattson@google.com, jpoimboe@kernel.org,
        linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
        mingo@redhat.com, paulmck@kernel.org,
        pawan.kumar.gupta@linux.intel.com, pbonzini@redhat.com,
        peterz@infradead.org, quic_neeraju@quicinc.com,
        rdunlap@infradead.org, reinette.chatre@intel.com,
        sandipan.das@amd.com, songmuchun@bytedance.com, tglx@linutronix.de,
        tony.luck@intel.com, x86@kernel.org
References: <166759206900.3281208.11975514088019160962.stgit@bmoger-ubuntu>
 <20221107102134.255757-1-peternewman@google.com>
From:   "Moger, Babu" <babu.moger@amd.com>
In-Reply-To: <20221107102134.255757-1-peternewman@google.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WlBqQzJhUjR1RXN5SWd3VkFwZVA5QWE4WURkR0Z4Rko3bXAvZVM1U2VFbnZE?=
 =?utf-8?B?Y3hwNFZLTzY5bjhoSXJTbXJPckd4c3FweU50MjlUTnEzbzhxRWV0WG5JbVRh?=
 =?utf-8?B?ZWdxZG9EL0N4YXp4YmpqazhpZlNId2pxUzE3VXRZV2FRVkU1OEQ3WGRIaUt1?=
 =?utf-8?B?TWxhK1hPRzlZNDdXSTh5cUVDRytMcFRxYm1kT0FzdG5wRmo5am96bmwraXpm?=
 =?utf-8?B?cXFaY3UxeUxRV24rY2ZJOXVrMkdHSjgyUkFZbDlPM0plaGx1UkVBRHp2YjBH?=
 =?utf-8?B?eVowRWxkWmJ6U0JIaUtiL3J5NUNGT1ZZdlgyN2I5RmFBMGlFNVpPOENLbEd3?=
 =?utf-8?B?SUZZdzdJcXEvcXk3Zk85bUJ5aG40UDg2aURDMjA2bXdHRTZndnNHMTRiWGM4?=
 =?utf-8?B?RHZUWVA3Y2wzeTNCU0kvRVQ1VVNxdDgzc0JkMDdNajUzSjJyRjgxVURSS3Va?=
 =?utf-8?B?MUJMZTZzdHl6QXNOdmhkeURQdnptRm80ZW5QaGRycGtTc0cxYlRsei93WWIv?=
 =?utf-8?B?anpiL0FUME5aa0xTd3R4SERXbnAzWnI2RnZwQkxQZ3I0NkFzbzBzUlBRYnFh?=
 =?utf-8?B?MllTVEIySEp6V3duMHZpRDF5Wm1qS1dmS0Jsc1dPVTlOREFWeWJXeXpCTysz?=
 =?utf-8?B?d3B2YnR2ZC96V0lvWENCdStxdEh3SERqT3J5RGEyWUN0NGdrOTJMU1gvSDhO?=
 =?utf-8?B?aHdtWWNmVk5sKzhSNE5LRFphOWpiQ1hEb3UyeE56REpjZlorZW8wa3hMQU1y?=
 =?utf-8?B?aTdxczRwRlNXMFZFVk01WFM3MjhUUnpEUlk2MHhjcTIrVTc3dlVzUUNWdzEr?=
 =?utf-8?B?eFg5N2xIOVhONk1aK1JUTWRwcWZPaHVSb1ByUW1uZzZGeWJsWTVDQ1ZRM1hG?=
 =?utf-8?B?K2h6RFBhNEFBM1IrL0wybkVuQU5Rd2dHTTF0dGErUUg2eGNrdGhvbHE5M1Q0?=
 =?utf-8?B?TzJ4bTU5aUxXYWZJSkpJVzRnTTFGSmpVMFUzYU11T2EweWtVSmpFMnNZTDk5?=
 =?utf-8?B?cGwvRmFqVVgwZGsveFZ0R1NRQmgxWjcwZlZkbitCM2ljTmwzeEMrMnZwVGFH?=
 =?utf-8?B?UnE5bFpCVTdhbE12MFBhUGdocWQzTWEvSmxVenBwRjFaWnFFZDhaQ0FLNVht?=
 =?utf-8?B?cVVVYmxRWS9JVFRXcHU5am42TDV1TE9EUytIZ1F6ajNMNWVEWWhYUmEyNDlY?=
 =?utf-8?B?bjVvZEdXM0YxUTlLMHVJVFA5VHdLNVROeWZUSk4zU21rYUtpL3dlb0UySGRW?=
 =?utf-8?B?RWR1ZmhCYjZsNFd1SEM5NWVCVlIwaTZPUE5xWTlkYmw1SW93S0Y0T2k0MG5S?=
 =?utf-8?B?RFlUZnlMNEhsSHhjK2hwZWY5Q3J1YzBBcjA5SUZ3VlE2WDNCRGo4M0t0THEr?=
 =?utf-8?B?T0krc2tpQ0d0bnA5TVlaWHYxQUtYWi9XTHIzbDFOL3J2dk9GZWhEeVFra2h0?=
 =?utf-8?B?QnVUdGtmK0JZbFFSWGsxTzMvMEpmZEdMd2F3b0dzeEcyQm4xdGpydzhKYWN4?=
 =?utf-8?B?SSswME1JQmVhejdOZXNjS1ZBUHNpU2dGSlBOaEM0VzUvUk0xM0ozeHZHWmxq?=
 =?utf-8?B?TWEzVHg4Q1VzREt0WW5GMzlDME1YTGs1ZlFSdHRqWXJQS0VOMzVoVUg1Nkpx?=
 =?utf-8?B?WUhPTFJYdjg3VzFOVFBsYnlDRnBOb3VINDd2eXo2VWV3Y2l0cVhyMGNXenUx?=
 =?utf-8?B?NUxNREIvem4zK0tnUlM3c01RaEpFdkN1cFFwYnlqVmtZbjNWTU1xMWgrRnpr?=
 =?utf-8?B?alYrcG5Qd1VvMEFsdnp1TTNKa2M1OTkxdWNOVERJYkh5YjY0aGdYVEkxdnVy?=
 =?utf-8?B?S0kwaUw2MmFGL3BVSXdadGVhRTZ5S2htTTFhNFQzZHZ6WnhML3JnalBXOHZw?=
 =?utf-8?B?Wmd1eTBGSTJ2Tjc2QnNpSDBTblZXVkNJSkEwVXlrS3lOZGJhN1hScnp3bEc1?=
 =?utf-8?B?K2F1K3RSbkxEN2tDaVpDUEFxaGdnK1l0c0prTk1iMElmUUN6OEZHZ2xNQmpz?=
 =?utf-8?B?RzZPSjBUWDJyUUhMRTRGZHcwNHBwMXRRaG5xOTlWY1ZFRWpjbDg4Y3hnZlE1?=
 =?utf-8?B?Tjh6Zm8zN2lOV0dYYkZWeEc2dXFHMktsemJKNG1rMS9uK3pPSEp6SUUwbzNE?=
 =?utf-8?Q?8dN0=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8bcc0839-f959-48d1-4fc8-08dac0f0f886
X-MS-Exchange-CrossTenant-AuthSource: MW3PR12MB4553.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Nov 2022 18:50:42.3394
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: J/kAvOlXtB6651EbcPSZhqg8+vNXeMqSebTHdLFNSIF3BnNzWlV8r2avX1IxpHXG
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN2PR12MB4517
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Peter,

On 11/7/22 04:21, Peter Newman wrote:
> Hi Babu,
>
> On Fri, Nov 04, 2022 at 03:01:09PM -0500, Babu Moger wrote:
>> +	/*
>> +	 * When an Event Configuration is changed, the bandwidth counters
>> +	 * for all RMIDs and Events will be cleared by the hardware. The
>> +	 * hardware also sets MSR_IA32_QM_CTR.Unavailable (bit 62) for
>> +	 * every RMID on the next read to any event for every RMID.
>> +	 * Subsequent reads will have MSR_IA32_QM_CTR.Unavailable (bit 62)
>> +	 * cleared while it is tracked by the hardware. Clear the
>> +	 * mbm_local and mbm_total counts for all the RMIDs.
>> +	 */
>> +	memset(d->mbm_local, 0, sizeof(struct mbm_state) * r->num_rmid);
>> +	memset(d->mbm_total, 0, sizeof(struct mbm_state) * r->num_rmid);
> Looking around, I can't find a reader for mbm_total anymore. It looks
> like the last place it was used went away in James's recent change:
>
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fall%2F20220902154829.30399-19-james.morse%40arm.com&amp;data=05%7C01%7Cbabu.moger%40amd.com%7C84a9d0f934894a3031a608dac0a9db33%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C638034133003350939%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&amp;sdata=i3isjCzxnBp4b2VblC7ZpH3hShUEe7unKiKfngG1kzE%3D&amp;reserved=0
>
> Are we supposed to be clearing arch_mbm_total now?

Yes. You are right. We should be using resctrl_arch_reset_rmid to reset
the rmids here.

This patch should work. Will fix it in next revision after the other comments.

diff --git a/arch/x86/kernel/cpu/resctrl/rdtgroup.c
b/arch/x86/kernel/cpu/resctrl/rdtgroup.c             │
index 6b222f8e58ae..28d9d99a639e
100644                                                                  │
---
a/arch/x86/kernel/cpu/resctrl/rdtgroup.c                                                            
│
+++
b/arch/x86/kernel/cpu/resctrl/rdtgroup.c                                                            
│
@@ -1517,7 +1517,7 @@ static int mbm_config_write(struct rdt_resource *r,
struct rdt_domain *d,          │
                            u32 evtid, u32
val)                                                          │
 {                                                                                                      
│
        struct mon_config_info mon_info =
{0};                                                           │
-       int ret =
0;                                                                                    
│
+       int ret = 0,
i;                                                                                 
│
                                                                                                        
│
       
rdt_last_cmd_clear();                                                                           
│
                                                                                                        
│
@@ -1557,8 +1557,10 @@ static int mbm_config_write(struct rdt_resource *r,
struct rdt_domain *d,         │
         * cleared while it is tracked by the hardware. Clear
the                                        │
         * mbm_local and mbm_total counts for all the
RMIDs.                                             │
        
*/                                                                                             
│
-       memset(d->mbm_local, 0, sizeof(struct mbm_state) *
r->num_rmid);                                 │
-       memset(d->mbm_total, 0, sizeof(struct mbm_state) *
r->num_rmid);                                 │
+       for (i = 0; i < r->num_rmid; i++)
{                                                              │
+               resctrl_arch_reset_rmid(r, d, i,
QOS_L3_MBM_TOTAL_EVENT_ID);                             │
+               resctrl_arch_reset_rmid(r, d, i,
QOS_L3_MBM_LOCAL_EVENT_ID);                             │
+      
}                                                                                               
│
                                                                                                        
│
 write_exit:                                                                                            
│
        return ret;                                   


Thanks

Babu