Received-SPF: pass (google.com: domain of linux-kernel+bounces-88135-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <cc8699be-3aae-42aa-9c70-f8b6a9728ee3@amd.com>
Date: Fri, 1 Mar 2024 14:30:51 +0530
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] KVM: x86/svm/pmu: Set PerfMonV2 global control bits
 correctly
To: Like Xu <like.xu.linux@gmail.com>
Cc: seanjc@google.com, pbonzini@redhat.com, dapeng1.mi@linux.intel.com,
 mizhang@google.com, jmattson@google.com, ravi.bangoria@amd.com,
 nikunj.dadhania@amd.com, santosh.shukla@amd.com, manali.shukla@amd.com,
 babu.moger@amd.com, kvm list <kvm@vger.kernel.org>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
References: <20240301075007.644152-1-sandipan.das@amd.com>
 <06061a28-88c0-404b-98a6-83cc6cc8c796@gmail.com>
Content-Language: en-US
From: Sandipan Das <sandipan.das@amd.com>
In-Reply-To: <06061a28-88c0-404b-98a6-83cc6cc8c796@gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UFpON2pYdWpzNzBSOE4vUEVWSW1ZQ0JwTXpkb1FqdDF0MDhHREN6MXNXT3pa?=
 =?utf-8?B?UmZwNjQrU3UweW9PNStEdDNzbG1aZlhGS0dCVUhRM1crSnhaS3pHS2xBRkJQ?=
 =?utf-8?B?dGNZU3R6Y0ZmcFNnNkdpcmNxcTdKbWF6d1hSU01YRWZUd1NOQ0Z3MGRHN203?=
 =?utf-8?B?Vi9rRWlwNnJZWGcrRjF4cWpqTTBQL2N2cTYzeDhCWHV4OFZGb29DRWlyYm4v?=
 =?utf-8?B?U01uRXdjM1pFcnl3QTBJUUxBT0VnOUsyck9XbEJIUHpVMnFpSURDSEwyZmFQ?=
 =?utf-8?B?OXpqc3JWekRPUjVhcGZuMTZ3dThiTngrVHY4YndzWFFidkpybjFQYmZrQWhF?=
 =?utf-8?B?MWl6M2tJdG81QlcwS1ZSWndoNXVHNkZ5Z2dLdXN2eDdGSEhxeWs4R3BiQXgy?=
 =?utf-8?B?TW45YjNjWjdlclZiYWxZdk5OWWU1cmJkYkNuTXFMeVlWV3drL3F1NEpwelNR?=
 =?utf-8?B?Z1E0bnFBN21pQTFiZHE0TExWRFVndVJreHYzL3NYYzgzRVN4OTJkeDhBLzA2?=
 =?utf-8?B?d083WmtnNVloQ0RlcU5JQ0JuNkpGTVJMcGhVTkRlWG5RVlZnYkVQdTlBTEV2?=
 =?utf-8?B?NWNoRVNrUk1rWTY5OFRDaGlvOWFwLzNhUllHRjRMbHJEL3BCZGRNZHJlZllG?=
 =?utf-8?B?VzVPYzNRaGlydGZrdUxTM3owbVBpMDVhS1JncU1NQnFqaEh6L2t5cTFRTW5H?=
 =?utf-8?B?QjlEaFJndDZ1SUd2NnM0QkM5QlFmaEFuYm1UQ0tSUng0MloxcHl3YnhhbUZj?=
 =?utf-8?B?S1VjeUdZZU1iVmNTVjRIaitBSDR4L1JOL003aytZTzlqWklWZmk0ZGpJeGh6?=
 =?utf-8?B?WHRvcktuQnp5K3EyVUpOcHROaVNISnJTOThEM1pST3QxdnBaMkFsTUNWYkF1?=
 =?utf-8?B?c2t0TDBpM1NPeEFEUlNWeXlXQnBVM1dkeWtQdlQ1NGdKdnp3aUhrWkNCVzU2?=
 =?utf-8?B?WjJTZGNaYWY2anEvYWR4SWppWTZubHBxN0I0NDU3a00zT1BYNmp4Mk80NkQr?=
 =?utf-8?B?TS80Vkk2MXQ1RGVoMjlVMUdqRkd6M3QrK1hyT0trRjBJaWtlUHN0MXRva1NF?=
 =?utf-8?B?QkRhbFAvaVlUV3d0M3JhSHJOMHlDcFBCVHpCQVQvT1lycFlLS3hzK0lQQ01H?=
 =?utf-8?B?ZFVLNlgwcWVzc3NBajQzNFdGRUZjZk9uUlJiUUhnTTNvaDd4TkNzYWhtdmsv?=
 =?utf-8?B?SmR1M3dGRTVLQVAyT29QRkJoQk50YzNiVy9kdWJkbnVZdjhBTTZlb2Rtckwv?=
 =?utf-8?B?bGRmR0U3NHJ3UXFrQk96b2V0M2hFZjh6TW54NVBTUDhmVjR1N0FWb3BTM05I?=
 =?utf-8?B?UlROclpJRlhxMndSMXd3M0luazV2S0hTTjVubmpaYTdXTDhxeXVTbXZiS0M4?=
 =?utf-8?B?bTRnendYOEhZSEp3YWV2dCtNcTNiTFN1amNqaVpYZE9KNHdBK2VoNEtGenlR?=
 =?utf-8?B?YmYyUjllOUN3NHZVcmtpMTh6YXV0VkFFT05tdU01bHBZMi9nRklqZWhoYnBB?=
 =?utf-8?B?MStLUUV0aFNMQ0svQzFuMkZGUTd2NjJKN0h4MkNYM2R5eGFEMjErSkRvb0U1?=
 =?utf-8?B?d1VKR0hVNUhOU1hEc0hZTTI4VGxoK2JzclVjd3JqN090YWVCaFNOY3R6R2lH?=
 =?utf-8?B?M3FCcTVGYkRTVVpETFJULzFEM3FDeUpFTUxiOHdqalBpSDVTWW54OXNmd2J4?=
 =?utf-8?B?NGYzY2N6Y21kMUhRVFpOck1sYTZyQmFMUzRxcHFFelhHeHRDNGFhUlZxU0E1?=
 =?utf-8?B?S0JvZHZGVUR6bU92TEdVL1JUQlFMOWpac3o3TmZLWE9XWUVSeTFPQlE0UmRv?=
 =?utf-8?B?ZlVsVmRuazJVVVEwYUNOOUZDOU82bDJCMk1Zb281WXlrcEgxNEtYV0h4TENm?=
 =?utf-8?B?QlVhUUxWbWVoOUFoaXRzamg4RlJzOG9qdllmd0w1dXNhOFBnQi9CU2RsK0xF?=
 =?utf-8?B?Qm9MWUF3VnFyL1lzNFZPS1p4bVhVZTE5VlFyVE96UWNMVHhXekhVb05VYXE5?=
 =?utf-8?B?MUxSbjRvOXlXR1FacW94L3N0YlZ2aUovOGZRM3R0cjJEMWRJTkVTdDZna2lF?=
 =?utf-8?B?ejZnbkJ6Y3Q5TkNwWjR4MEM4MVBTQWRKd250Y2JJUjU1MVFSU0w2Q2wzODNj?=
 =?utf-8?Q?jm6NKEMeZnOyejymHUo9ri8xS?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4f258b00-cb75-4bb1-7e7b-08dc39ce1e1c
X-MS-Exchange-CrossTenant-AuthSource: PH7PR12MB5712.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Mar 2024 09:01:01.4514
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: llkK92KRf++xty+z6RSQBBgB7Dz5t/OYIowzUeDlMZ03GVF0H/CZ1zLie9GnEkq6e4St1fDQgmoUQbZbv0EQdg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM4PR12MB6397

On 3/1/2024 2:07 PM, Like Xu wrote:
> On 1/3/2024 3:50 pm, Sandipan Das wrote:
>> With PerfMonV2, a performance monitoring counter will start operating
>> only when both the PERF_CTLx enable bit as well as the corresponding
>> PerfCntrGlobalCtl enable bit are set.
>>
>> When the PerfMonV2 CPUID feature bit (leaf 0x80000022 EAX bit 0) is set
>> for a guest but the guest kernel does not support PerfMonV2 (such as
>> kernels older than v5.19), the guest counters do not count since the
>> PerfCntrGlobalCtl MSR is initialized to zero and the guest kernel never
>> writes to it.
> 
> If the vcpu has the PerfMonV2 feature, it should not work the way legacy
> PMU does. Users need to use the new driver to operate the new hardware,
> don't they ? One practical approach is that the hypervisor should not set
> the PerfMonV2 bit for this unpatched 'v5.19' guest.
> 

My understanding is that the legacy method of managing the counters should
still work because the enable bits in PerfCntrGlobalCtl are expected to be
set. The AMD PPR does mention that the PerfCntrEn bitfield of PerfCntrGlobalCtl
is set to 0x3f after a system reset. That way, the guest kernel can use either
the new or legacy method.

>>
>> This is not observed on bare-metal as the default value of the
>> PerfCntrGlobalCtl MSR after a reset is 0x3f (assuming there are six
>> counters) and the counters can still be operated by using the enable
>> bit in the PERF_CTLx MSRs. Replicate the same behaviour in guests for
>> compatibility with older kernels.
>>
>> Before:
>>
>>    $ perf stat -e cycles:u true
>>
>>     Performance counter stats for 'true':
>>
>>                     0      cycles:u
>>
>>           0.001074773 seconds time elapsed
>>
>>           0.001169000 seconds user
>>           0.000000000 seconds sys
>>
>> After:
>>
>>    $ perf stat -e cycles:u true
>>
>>     Performance counter stats for 'true':
>>
>>               227,850      cycles:u
>>
>>           0.037770758 seconds time elapsed
>>
>>           0.000000000 seconds user
>>           0.037886000 seconds sys
>>
>> Reported-by: Babu Moger <babu.moger@amd.com>
>> Fixes: 4a2771895ca6 ("KVM: x86/svm/pmu: Add AMD PerfMonV2 support")
>> Signed-off-by: Sandipan Das <sandipan.das@amd.com>
>> ---
>>   arch/x86/kvm/svm/pmu.c | 1 +
>>   1 file changed, 1 insertion(+)
>>
>> diff --git a/arch/x86/kvm/svm/pmu.c b/arch/x86/kvm/svm/pmu.c
>> index b6a7ad4d6914..14709c564d6a 100644
>> --- a/arch/x86/kvm/svm/pmu.c
>> +++ b/arch/x86/kvm/svm/pmu.c
>> @@ -205,6 +205,7 @@ static void amd_pmu_refresh(struct kvm_vcpu *vcpu)
>>       if (pmu->version > 1) {
>>           pmu->global_ctrl_mask = ~((1ull << pmu->nr_arch_gp_counters) - 1);
>>           pmu->global_status_mask = pmu->global_ctrl_mask;
>> +        pmu->global_ctrl = ~pmu->global_ctrl_mask;
>>       }
>>         pmu->counter_bitmask[KVM_PMC_GP] = ((u64)1 << 48) - 1;