Received: by 2002:a05:7412:da14:b0:e2:908c:2ebd with SMTP id fe20csp2186582rdb; Mon, 9 Oct 2023 16:13:21 -0700 (PDT) X-Google-Smtp-Source: AGHT+IEln3O80YVBTLBpna5Uwu/+UCD64Gurrw/6zAQLQAOlFUw/Vj6Bj/1A5Af5p10fLi26YB/1 X-Received: by 2002:a17:90b:4a81:b0:278:fcac:7e0 with SMTP id lp1-20020a17090b4a8100b00278fcac07e0mr21235394pjb.16.1696893201388; Mon, 09 Oct 2023 16:13:21 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1696893201; cv=none; d=google.com; s=arc-20160816; b=E+BpHns/5Zv9pvrRpcnhmKWYuuYKzc6vdWkWkyEwuWXHTm+vms+pclB73Sg8aGbShx hr1njDMkjczlxk0rjbAWHRfxxzAZyIhTMmGQLmYOibiPXOvKLceC5moF5st4XhhsQCXD PW98xPTBvk2Fob9YUNMBMawqrtoUo+p0O/p55sNUPAARfjCyBHpL967l3Yn+VyX0agwT WbYI3+9pD+LMfT3XP4XpqlDQreloCnJebtuUewcXb3RVkCsTpm/D3XODQwmE3Y2sBcCe maltRBIWw+vqnoJtL1+QGi0PdCWkbLmChkCXlMxTFyJhee3sHZzinAgqN5xFMyL96brn x6sA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:cc:to:from:subject:message-id:mime-version:date :dkim-signature; bh=ZLpiPFI3YPzDm01qV1AAC709jBrt9uw5FfnkB7hlp+4=; fh=nSGf6E5bqeZ4+WG1wtUl4dHMM2T+/oX4atvC7v39NTU=; b=h2S+JxKJ1m0YAb5v3zpNFLkIkhZbHaJev/5TGbuKB8SjZ9HSID0D+P/QKKIhArBc6S agOThwbPEklBuQgZozr601fXJQdyaVttKv4VKhY3AiTm2hbephYzIldsm7RTPBvp7PQw jP889H0YUhrRtvOxPrsYkXh1tlbcKl69rk/v0oTAHcyQYYKI7P9e6mNKSYgRQZy5ghcy Tpc16z+zvIZthnqljqWctJ/F4YFMZF6+SZh0KgHP72BXDSUwZgNRZYKu6NoOlut6T7f8 0DYi0VwCUNU4rQAB3bcNwCaLqu/QB2QAa6F7+h1usDpBIdLkNnW8ADkj16c32wQDtgZA D+yw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=RnhMNZqC; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from groat.vger.email (groat.vger.email. [2620:137:e000::3:5]) by mx.google.com with ESMTPS id b6-20020a17090acc0600b002568a675b65si12548869pju.141.2023.10.09.16.13.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 09 Oct 2023 16:13:21 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) client-ip=2620:137:e000::3:5; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20230601 header.b=RnhMNZqC; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:5 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by groat.vger.email (Postfix) with ESMTP id 3C8AA80BB57B; Mon, 9 Oct 2023 16:13:07 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at groat.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1379068AbjJIXKZ (ORCPT + 99 others); Mon, 9 Oct 2023 19:10:25 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60148 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1379142AbjJIXKE (ORCPT ); Mon, 9 Oct 2023 19:10:04 -0400 Received: from mail-yw1-x1149.google.com (mail-yw1-x1149.google.com [IPv6:2607:f8b0:4864:20::1149]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 31A6CD58 for ; Mon, 9 Oct 2023 16:09:04 -0700 (PDT) Received: by mail-yw1-x1149.google.com with SMTP id 00721157ae682-5a7be940fe1so1678547b3.2 for ; Mon, 09 Oct 2023 16:09:04 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1696892943; x=1697497743; darn=vger.kernel.org; h=cc:to:from:subject:message-id:mime-version:date:from:to:cc:subject :date:message-id:reply-to; bh=ZLpiPFI3YPzDm01qV1AAC709jBrt9uw5FfnkB7hlp+4=; b=RnhMNZqCu/vFzYR2jIJbcYut1ZVGEcKxN1W7CSAtD9FmKjroNHZLtU3NW2CZ0AWCrn yxIxAPaA1OInJyyFvHPWnJ3JqZhOpQ8RmTHsPStluQUP1I5Qf0grrM2C8DAI6ClGlrXu VFdLK37k2c/Kwl0mga+BxiHSs9/ZOO3+MZzqnlhyyHQR8wgyQg693Fme5cPPpfSKj1XC pWOMS3oIcN/ySCHv9RjbfeUy0BaZmxYQIX1EFLaUNdVcfgCYA29YO86poyR/BOQPK8Kg aba2lLMCsErPayf6FYmpywyXTgT+FORwkifcor1TV3jKfbAKJ9YJqpWtU71GSz1SY+sc aIDw== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1696892943; x=1697497743; h=cc:to:from:subject:message-id:mime-version:date:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=ZLpiPFI3YPzDm01qV1AAC709jBrt9uw5FfnkB7hlp+4=; b=lpnyoHSQ9vQ17rhqj2DrUCXC5D7CvmJwcSkt5mWhN6h5xMbNEIkLqE5HAHiKkQWX3/ E9+arhPgzxGtJDGI6+KcTXtmfcFfoCHHO3MEnOz2hgLxJ+BjYNJVowOfEvVHqQ9E7qwI eEOlvSJT2x4Wy1mkCiLYI7wyr+ERFm8mUJGzRDgxcrlkFGoW3eAXIE63fUpcOuDu22+N Gl20Zg2VyiIFzjCOXtRGtBoS2OglQFyLQQGL+FzmkxpEJS/DtIgYViZATNQLM2T97EWK OFk+F6RWZUiFX0UWWmtz06ZjKfdwZYqv3fGHCUifVm2r7AUkYi+TYv4TOY9HFwmd1LrO jM/Q== X-Gm-Message-State: AOJu0YxakptT+IRaswG5v+rYnuFYo2RZqK1gdE7Qos5WUzyzcUCCGQF8 sZdyDD0KesFCckVqH6itsl8LuLfTkyaX X-Received: from rananta-linux.c.googlers.com ([fda3:e722:ac3:cc00:2b:ff92:c0a8:20a1]) (user=rananta job=sendgmr) by 2002:a25:d4d2:0:b0:d9a:38e4:78b5 with SMTP id m201-20020a25d4d2000000b00d9a38e478b5mr40510ybf.5.1696892943019; Mon, 09 Oct 2023 16:09:03 -0700 (PDT) Date: Mon, 9 Oct 2023 23:08:46 +0000 Mime-Version: 1.0 X-Mailer: git-send-email 2.42.0.609.gbb76f46606-goog Message-ID: <20231009230858.3444834-1-rananta@google.com> Subject: [PATCH v7 00/12] KVM: arm64: PMU: Allow userspace to limit the number of PMCs on vCPU From: Raghavendra Rao Ananta To: Oliver Upton , Marc Zyngier Cc: Alexandru Elisei , James Morse , Suzuki K Poulose , Paolo Bonzini , Zenghui Yu , Shaoqin Huang , Jing Zhang , Reiji Watanabe , Colton Lewis , Raghavendra Rao Anata , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-4.8 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_SBL_CSS,SPF_HELO_NONE,SPF_PASS, USER_IN_DEF_DKIM_WL autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Mon, 09 Oct 2023 16:13:07 -0700 (PDT) Hello, With permission from Reiji Watanabe , the original author of the series, I'm posting the v6 with necessary alterations. The goal of this series is to allow userspace to limit the number of PMU event counters on the vCPU. We need this to support migration across systems that implement different numbers of counters. The number of PMU event counters is indicated in PMCR_EL0.N. For a vCPU with PMUv3 configured, its value will be the same as the current PE by default. Userspace can set PMCR_EL0.N for the vCPU to any value even with the current KVM using KVM_SET_ONE_REG. However, it is practically unsupported, as KVM resets PMCR_EL0.N to the host value on vCPU reset and some KVM code uses the host value to identify (un)implemented event counters on the vCPU. This series will ensure that the PMCR_EL0.N value is preserved on vCPU reset and that KVM doesn't use the host value to identify (un)implemented event counters on the vCPU. This allows userspace to limit the number of the PMU event counters on the vCPU. The series is based on kvmarm/next @7e6587baafc0 to include the vCPU reset and feature flags cleanup/fixes series [1]. Patch 1 adds helper functions to set a PMU for the guest. This helper will make it easier for the following patches to add modify codes for that process. Patch 2 makes the default PMU for the guest set before the first vCPU reset. Patch 3 fixes reset_pmu_reg() to ensure that (RAZ) bits of PMCNTEN{SET,CLR}_EL0, PMINTEN{SET,CLR}_EL1, and PMOVS{SET,CLR}_EL1 corresponding to unimplemented event counters on the vCPU are reset to zero. Patch 4 is a minor refactoring to use the default PMU register reset function for PMUSERENR_EL0 and PMCCFILTR_EL0. Patch 5,6 adds a helper to read vCPU's PMCR_EL0 and the number of counters, respectively. Patch 7 changes the code to use the guest's PMCR_EL0.N, instead of the PE's PMCR_EL0.N. Patch 8 adds support userspace modifying PMCR_EL0.N. Patch 9-12 adds a selftest to verify reading and writing PMU registers for implemented or unimplemented PMU event counters on the vCPU. v7: Thanks, Oliver for the suggestions - Rebase the series onto kvmarm/next. - Move the logic to set the default PMU for the guest from kvm_reset_vcpu() to __kvm_vcpu_set_target() to deal with the error returned. - Add a helper, kvm_arm_get_num_counters(), to read the number of general-purpose counters. - Use this helper to fix the error reported by kernel test robot [2]. v6: Thanks, Oliver and Shaoqin for the suggestions - Split the previously defined kvm_arm_set_vm_pmu() into separate functions: default arm_pmu and a caller requested arm_pmu. - Send -EINVAL from kvm_reset_vcpu(), instead of -ENODEV for the case where KVM fails to set a default arm_pmu, to remain consistent with the existing behavior. - Drop the v5 patch-5/12 that removes ARMV8_PMU_PMCR_N_MASK and adds ARMV8_PMU_PMCR_N. Make corresponding changes to v5 patch-6/12. - Disregard introducing 'pmcr_n_limit' in kvm->arch as a member to be accessed later in 'set_pmcr()'. Instead, directly obtain the value by accessing the saved 'arm_pmu'. - 'set_pmcr()' ignores the error when userspace tries to set PMCR.N greater than the hardware limit to keep the existing API behavior. - 'set_pmcr()' ignores modifications to the register after the VM has started and returns a success to userspace. - Introduce [get|set]_pmcr_n() helpers in the selftest to make modifications to the field easier. - Define the 'vpmu_vm' globally in the selftest, instead of allocating it every time a VM is created. - Use the new printf style __GUEST_ASSERT()s in the selftest. v5: https://lore.kernel.org/all/20230817003029.3073210-1-rananta@google.com/ - Drop the patches (v4 3,4) related to PMU version fixes as it's now being handled in a separate series [3]. - Switch to config_lock, instead of kvm->lock, while configuring the guest PMU. - Instead of continuing after a WARN_ON() for the return value of kvm_arm_set_vm_pmu() in kvm_arm_pmu_v3_set_pmu(), patch-1 now returns from the function immediately with the error code. - Fix WARN_ON() logic in kvm_host_pmu_init() (patch v4 9/14). - Instead of returning 0, return -ENODEV from the kvm_arm_set_vm_pmu() stub function. - Do not define the PMEVN_CASE() and PMEVN_SWITCH() macros in the selftest code as they are now included in the imported arm_pmuv3.h header. - Since the (initial) purpose of the selftest is to test the accessibility of the counter registers, remove the functional test at the end of test_access_pmc_regs(). It'll be added later in a separate series. - Introduce additional helper functions (destroy_vpmu_vm(), PMC_ACC_TO_IDX()) in the selftest for ease of maintenance and debugging. v4: https://lore.kernel.org/all/20230211031506.4159098-1-reijiw@google.com/ - Fix the selftest bug in patch 13 (Have test_access_pmc_regs() to specify pmc index for test_bitmap_pmu_regs() instead of bit-shifted value (Thank you Raghavendra for the reporting the issue!). v3: https://lore.kernel.org/all/20230203040242.1792453-1-reijiw@google.com/ - Remove reset_pmu_reg(), and use reset_val() instead. [Marc] - Fixed the initial value of PMCR_EL0.N on heterogeneous PMU systems. [Oliver] - Fixed PMUVer issues on heterogeneous PMU systems. - Fixed typos [Shaoqin] v2: https://lore.kernel.org/all/20230117013542.371944-1-reijiw@google.com/ - Added the sys_reg's set_user() handler for the PMCR_EL0 to disallow userspace to set PMCR_EL0.N for the vCPU to a value that is greater than the host value (and added a new test case for this behavior). [Oliver] - Added to the commit log of the patch 2 that PMUSERENR_EL0 and PMCCFILTR_EL0 have UNKNOWN reset values. v1: https://lore.kernel.org/all/20221230035928.3423990-1-reijiw@google.com/ Thank you. Raghavendra [1]: https://lore.kernel.org/all/20230920195036.1169791-1-oliver.upton@linux.dev/ [2]: https://lore.kernel.org/all/202309290607.Qgg05wKw-lkp@intel.com/ [3]: https://lore.kernel.org/all/20230728181907.1759513-1-reijiw@google.com/ Raghavendra Rao Ananta (2): KVM: arm64: PMU: Add a helper to read the number of counters tools: Import arm_pmuv3.h Reiji Watanabe (10): KVM: arm64: PMU: Introduce helpers to set the guest's PMU KVM: arm64: PMU: Set the default PMU for the guest before vCPU reset KVM: arm64: PMU: Clear PM{C,I}NTEN{SET,CLR} and PMOVS{SET,CLR} on vCPU reset KVM: arm64: PMU: Don't define the sysreg reset() for PM{USERENR,CCFILTR}_EL0 KVM: arm64: PMU: Add a helper to read a vCPU's PMCR_EL0 KVM: arm64: PMU: Set PMCR_EL0.N for vCPU based on the associated PMU KVM: arm64: PMU: Allow userspace to limit PMCR_EL0.N for the guest KVM: selftests: aarch64: Introduce vpmu_counter_access test KVM: selftests: aarch64: vPMU register test for implemented counters KVM: selftests: aarch64: vPMU register test for unimplemented counters arch/arm64/include/asm/kvm_host.h | 3 + arch/arm64/kvm/arm.c | 23 +- arch/arm64/kvm/pmu-emul.c | 102 ++- arch/arm64/kvm/sys_regs.c | 101 ++- include/kvm/arm_pmu.h | 18 + tools/include/perf/arm_pmuv3.h | 308 +++++++++ tools/testing/selftests/kvm/Makefile | 1 + .../kvm/aarch64/vpmu_counter_access.c | 590 ++++++++++++++++++ .../selftests/kvm/include/aarch64/processor.h | 1 + 9 files changed, 1087 insertions(+), 60 deletions(-) create mode 100644 tools/include/perf/arm_pmuv3.h create mode 100644 tools/testing/selftests/kvm/aarch64/vpmu_counter_access.c base-commit: 7e6587baafc0054bd32d9ca5f72af36e36ff1d05 -- 2.42.0.609.gbb76f46606-goog