Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Huang Shijie <shijie@os.amperecomputing.com>
To:     maz@kernel.org
Cc:     oliver.upton@linux.dev, james.morse@arm.com,
        suzuki.poulose@arm.com, yuzenghui@huawei.com,
        catalin.marinas@arm.com, will@kernel.org, pbonzini@redhat.com,
        peterz@infradead.org, ingo@redhat.com, acme@kernel.org,
        mark.rutland@arm.com, alexander.shishkin@linux.intel.com,
        jolsa@kernel.org, namhyung@kernel.org, irogers@google.com,
        linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev,
        linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
        linux-perf-users@vger.kernel.org, patches@amperecomputing.com,
        zwang@amperecomputing.com,
        Huang Shijie <shijie@os.amperecomputing.com>
Subject: [PATCH] perf/core: fix the bug in the event multiplexing
Date:   Wed,  9 Aug 2023 09:39:53 +0800
Message-Id: <20230809013953.7692-1-shijie@os.amperecomputing.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?/yl5zHIoDSMhGOQiK2uyTd9uVzpe0Mqr9K+FBQubuQzSKpZbYeG9ChMZ5XLi?=
 =?us-ascii?Q?5VBT5AvNhXTsDllHj/sppide9TnMQ1FbXZwSv5lH+C4RsPbHmTyGAHvKRvY+?=
 =?us-ascii?Q?HJ6mbqZQTo59THuJbCd3GSfwRsWOVxb5EzSsE7f4UrbFYDqBbDZpA2rom24A?=
 =?us-ascii?Q?aD91vH1wNjn1GDS1qq/yQzfe5LoejYa0Gku4toudsLFoj2TwUwbb50c4pmuP?=
 =?us-ascii?Q?MHb+hAJKf51EmKH2Vh0omK8k6lQtmRVyE3BZEjy4USyVYfYUEnKjtheSg6sD?=
 =?us-ascii?Q?CSKU7Bq7QVI2Nh1hF0H9wZ2+xv87mLyhjQQf+Ifod4tyEka0W7bNecjvQO+/?=
 =?us-ascii?Q?bohN7AYbwYTRxV4Hi7IvQPw3Zif+smyFrn9gizVqFiKq/5/kAKIdi5H6KyS6?=
 =?us-ascii?Q?uexOiGMEdqubhroDW/O7eMyPb9KU3yZB7xhA2QSFk3KfurZLmD8sCmkiw4T2?=
 =?us-ascii?Q?sM9Va3BmSCnxkShPGQsFHqMCdH9F7sX+hLjntzPnfwtl0vUPtTIQJAD5Hilw?=
 =?us-ascii?Q?OKPceh1VKW4wKB0gR2O9LRm7olG1qP+5iWbWudH+UC2IcH//fjz+mAdUBQ5I?=
 =?us-ascii?Q?2PQsYA/kUrUFZPmZJ6QaiLEW7B/XmoDkFefjtg16QloMW71JtE6YabKjSUlH?=
 =?us-ascii?Q?LMy5HTjgtQ2reNvcRcE4EyUDx3wnWqHpWJRQ1q36/izW8HXXehvVSyunfyxl?=
 =?us-ascii?Q?2xuUliuipHLlJP9xKCGSjTyLRnWCxlDCPwRCeU4e/MoOVnpS5VIjmWhUdv8T?=
 =?us-ascii?Q?4dZd6YUCwpxALBdm46TqXzRaXU0CPH+H8XwmWMGhMeWRKi3ddi+o5AzDBv7H?=
 =?us-ascii?Q?uP0Pjl1MxxULJCyvjEfyYlo3jQBXBQL8GJaN//Pil3Ztu2t71uR3AFC7OgFg?=
 =?us-ascii?Q?sm+HC1FbK9G3t5n/L0doJg+oHkM/BwrepEx1bI/Hv47+ZnWHSntHKj+MceB5?=
 =?us-ascii?Q?yLVuNS8V5PX10mTxazTrM2862enPAOCIWj59CLyfzGMoL/F1+/clxIX7bUrc?=
 =?us-ascii?Q?ZrCD14ziBr1AnToqSo/RwrZtEUvAMmfzgJ4gNGFUC1VTeoCB31fZFG64cKIr?=
 =?us-ascii?Q?AHUVNAPoSInlaENzAY0AYBJuIDQiKUw2l4Pi2phzDe+1Har67Qegmua6sQMR?=
 =?us-ascii?Q?Wa2N/v2pgdwWalJ2aZmf8q5H18ay2zZWsqPWAbMo68b9dvO9iGgcNIdqZjY+?=
 =?us-ascii?Q?Y3rRpaDi9/Y5Jy+0EBTNTCRL1jKwzjrwBqFpvVp/ACFgW3KsaDYQQYCVFoCA?=
 =?us-ascii?Q?HKyAD3toGj774eDxPl70pCi7X1UnBsXAAtP2N/qSIJ6XFgFAeeJy3efyHLF4?=
 =?us-ascii?Q?XDn9YYLywzCYZmu6bixS6iQWlb20U+0EFsJ1uIlL2KDAzO9AnXaqr9lfE5dj?=
 =?us-ascii?Q?4GWmN5J7Fs0SxudjKy+jcmJky6xQtTwlhbFT3vGMjtZDH7PyBJydynna0UtM?=
 =?us-ascii?Q?T8ydmzMjVBrb0KuKtUuh7Sj7yEMnz7qzxLK86j0nh082qZZtaOeR7NhLDz42?=
 =?us-ascii?Q?kyDztckT2/ICs13fSqJ+skGfdyi8WCo7KH52EVkoVlvSABAdaWtcJeVW8P3N?=
 =?us-ascii?Q?bng5edOeQnBSRXXy8DFblkNBVM6RDAz+KlgFe005f9LWofI97sKwZ4IenqP4?=
 =?us-ascii?Q?iBYu18zJCkZaoPBT9qlGaUQ=3D?=
X-OriginatorOrg: os.amperecomputing.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7a487444-e968-4013-b558-08db987a397d
X-MS-Exchange-CrossTenant-AuthSource: DM8PR01MB6824.prod.exchangelabs.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 Aug 2023 01:44:52.7626
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3bc2b170-fd94-476d-b0ce-4229bdc904a7
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3cA2jnJxtuh0IGxpxwcF9Vj8PMFbV4wo4NiBNS9cnq5cQlrAAFmGshitJ2gb7I/7cRJfbiKrBhzrfHLlaR2cqx+ME+ppQB3i/YlZyf2oFcP0b9oUfIFNzRUcr5N0SULc
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR01MB8118
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H2,
        SPF_HELO_PASS,SPF_PASS,URIBL_BLOCKED autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

1.) Background.
   1.1) In arm64, run a virtual guest with Qemu, and bind the guest
        to core 33 and run program "a" in guest.
        The code of "a" shows below:
   	----------------------------------------------------------
		#include <stdio.h>

		int main()
		{
			unsigned long i = 0;

			for (;;) {
				i++;
			}

			printf("i:%ld\n", i);
			return 0;
		}
   	----------------------------------------------------------

   1.2) Use the following perf command in host:
      #perf stat -e cycles:G,cycles:H -C 33 -I 1000 sleep 1
          #           time             counts unit events
               1.000817400      3,299,471,572      cycles:G
               1.000817400          3,240,586      cycles:H

       This result is correct, my cpu's frequency is 3.3G.

   1.3) Use the following perf command in host:
      #perf stat -e cycles:G,cycles:H -C 33 -d -d  -I 1000 sleep 1
            time             counts unit events
     1.000831480        153,634,097      cycles:G                                                                (70.03%)
     1.000831480      3,147,940,599      cycles:H                                                                (70.03%)
     1.000831480      1,143,598,527      L1-dcache-loads                                                         (70.03%)
     1.000831480              9,986      L1-dcache-load-misses            #    0.00% of all L1-dcache accesses   (70.03%)
     1.000831480    <not supported>      LLC-loads
     1.000831480    <not supported>      LLC-load-misses
     1.000831480        580,887,696      L1-icache-loads                                                         (70.03%)
     1.000831480             77,855      L1-icache-load-misses            #    0.01% of all L1-icache accesses   (70.03%)
     1.000831480      6,112,224,612      dTLB-loads                                                              (70.03%)
     1.000831480             16,222      dTLB-load-misses                 #    0.00% of all dTLB cache accesses  (69.94%)
     1.000831480        590,015,996      iTLB-loads                                                              (59.95%)
     1.000831480                505      iTLB-load-misses                 #    0.00% of all iTLB cache accesses  (59.95%)

       This result is wrong. The "cycle:G" should be nearly 3.3G.

2.) Root cause.
	There is only 7 counters in my arm64 platform:
	  (one cycle counter) + (6 normal counters)

	In 1.3 above, we will use 10 event counters.
	Since we only have 7 counters, the perf core will trigger
       	event multiplexing in hrtimer:
	     merge_sched_in() -->perf_mux_hrtimer_restart() -->
	     perf_rotate_context().

       In the perf_rotate_context(), it does not restore some PMU registers
       as context_switch() does.  In context_switch():
             kvm_sched_in()  --> kvm_vcpu_pmu_restore_guest()
             kvm_sched_out() --> kvm_vcpu_pmu_restore_host()

       So we got wrong result.

3.) About this patch.
        3.1) Add arch_perf_rotate_pmu_set()
        3.2) Add is_guest().
	     Check the context for hrtimer.
	3.3) In arm64's arch_perf_rotate_pmu_set(),
       	     set the PMU registers by the context.

4.) Test result of this patch:
      #perf stat -e cycles:G,cycles:H -C 33 -d -d  -I 1000 sleep 1
            time             counts unit events
     1.000817360      3,297,898,244      cycles:G                                                                (70.03%)
     1.000817360          2,719,941      cycles:H                                                                (70.03%)
     1.000817360            883,764      L1-dcache-loads                                                         (70.03%)
     1.000817360             17,517      L1-dcache-load-misses            #    1.98% of all L1-dcache accesses   (70.03%)
     1.000817360    <not supported>      LLC-loads
     1.000817360    <not supported>      LLC-load-misses
     1.000817360          1,033,816      L1-icache-loads                                                         (70.03%)
     1.000817360            103,839      L1-icache-load-misses            #   10.04% of all L1-icache accesses   (70.03%)
     1.000817360            982,401      dTLB-loads                                                              (70.03%)
     1.000817360             28,272      dTLB-load-misses                 #    2.88% of all dTLB cache accesses  (69.94%)
     1.000817360            972,072      iTLB-loads                                                              (59.95%)
     1.000817360                772      iTLB-load-misses                 #    0.08% of all iTLB cache accesses  (59.95%)

    The result is correct. The "cycle:G" is nearly 3.3G now.

Signed-off-by: Huang Shijie <shijie@os.amperecomputing.com>
---
 arch/arm64/kvm/pmu.c     | 8 ++++++++
 include/linux/kvm_host.h | 1 +
 kernel/events/core.c     | 5 +++++
 virt/kvm/kvm_main.c      | 9 +++++++++
 4 files changed, 23 insertions(+)

diff --git a/arch/arm64/kvm/pmu.c b/arch/arm64/kvm/pmu.c
index 121f1a14c829..a6815c3f0c4e 100644
--- a/arch/arm64/kvm/pmu.c
+++ b/arch/arm64/kvm/pmu.c
@@ -210,6 +210,14 @@ void kvm_vcpu_pmu_restore_host(struct kvm_vcpu *vcpu)
 	kvm_vcpu_pmu_disable_el0(events_guest);
 }
 
+void arch_perf_rotate_pmu_set(void)
+{
+	if (is_guest())
+		kvm_vcpu_pmu_restore_guest(NULL);
+	else
+		kvm_vcpu_pmu_restore_host(NULL);
+}
+
 /*
  * With VHE, keep track of the PMUSERENR_EL0 value for the host EL0 on the pCPU
  * where PMUSERENR_EL0 for the guest is loaded, since PMUSERENR_EL0 is switched
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 9d3ac7720da9..e350cbc8190f 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -931,6 +931,7 @@ void kvm_destroy_vcpus(struct kvm *kvm);
 
 void vcpu_load(struct kvm_vcpu *vcpu);
 void vcpu_put(struct kvm_vcpu *vcpu);
+bool is_guest(void);
 
 #ifdef __KVM_HAVE_IOAPIC
 void kvm_arch_post_irq_ack_notifier_list_update(struct kvm *kvm);
diff --git a/kernel/events/core.c b/kernel/events/core.c
index 6fd9272eec6e..fe78f9d17eba 100644
--- a/kernel/events/core.c
+++ b/kernel/events/core.c
@@ -4229,6 +4229,10 @@ ctx_event_to_rotate(struct perf_event_pmu_context *pmu_ctx)
 	return event;
 }
 
+void __weak arch_perf_rotate_pmu_set(void)
+{
+}
+
 static bool perf_rotate_context(struct perf_cpu_pmu_context *cpc)
 {
 	struct perf_cpu_context *cpuctx = this_cpu_ptr(&perf_cpu_context);
@@ -4282,6 +4286,7 @@ static bool perf_rotate_context(struct perf_cpu_pmu_context *cpc)
 	if (task_event || (task_epc && cpu_event))
 		__pmu_ctx_sched_in(task_epc->ctx, pmu);
 
+	arch_perf_rotate_pmu_set();
 	perf_pmu_enable(pmu);
 	perf_ctx_unlock(cpuctx, cpuctx->task_ctx);
 
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index dfbaafbe3a00..a77d336552be 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -218,6 +218,15 @@ void vcpu_load(struct kvm_vcpu *vcpu)
 }
 EXPORT_SYMBOL_GPL(vcpu_load);
 
+/* Do we in the guest? */
+bool is_guest(void)
+{
+	struct kvm_vcpu *vcpu;
+
+	vcpu = __this_cpu_read(kvm_running_vcpu);
+	return !!vcpu;
+}
+
 void vcpu_put(struct kvm_vcpu *vcpu)
 {
 	preempt_disable();
-- 
2.39.2