Received: by 2002:a05:6a10:9afc:0:0:0:0 with SMTP id t28csp2460825pxm; Fri, 25 Feb 2022 02:21:05 -0800 (PST) X-Google-Smtp-Source: ABdhPJwfM9ngKzil2VODIZrNclPwKrcq6d3Dy+8s4RGr+rbNZwAaHhuXsBCg4BkRysOaTqYsg9OZ X-Received: by 2002:a50:e08c:0:b0:407:cdc6:809e with SMTP id f12-20020a50e08c000000b00407cdc6809emr6362990edl.162.1645784464909; Fri, 25 Feb 2022 02:21:04 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1645784464; cv=none; d=google.com; s=arc-20160816; b=o6vOdVLmj3rLzmaO7oRsC0LaEGFubdf+FTAv7G2T0TAwzD2Qqw9Y6oUYNYB/rwlAZs i5G/MsRpJj5HvXJ3BpKonfGPNiqfeFp6uEtkSba8CUqnSaDI0QXwhrS85ThNFU6XBVwN oPYLz/iGjS4E/c+bNliYLIjZY2ijFa3tXqdjATMAfSl7LArdiOj7HTaGMyAh5zIxVekd SrwJFmXKIdGh+8iff3KfAfthachFdUIIVg/02hfGhoqwpXcKVKnoMprY6dHx03EUAtUL K0ryL54zR6j36swry8jQSJkgNLeRNfSeyD5XiWxb2KkGxjG7PmAjdts3H9N6VNGxiXwN IO8g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:in-reply-to:content-disposition:mime-version :references:message-id:subject:cc:to:from:date:dkim-signature; bh=3WKwDOfvIucdHL9pK/JQ4fGfUfoOR9rKDu+vz9mXHr0=; b=bVwRQvq2/6zP0kkL4qkyg9+4PsRADJMzbFrf56ervO5PPBMyU7PPABR/cri9Hc65wG ul5I4XxormzgoE7IAOdVbyn8P0l7P4IgobhyMrX9SvasYXZbFAOL32dWnxFiwFdca4Kh F6ii4jzI/RfN+nUGQB1vhMU5ogNqzNf7qrZUZ1E4sS8MmfxWAnp6C//9QjJV/FOx1rH1 ClEiGlav/MSgRppy8GLHu0ZWKlt92MrIsqYCLhpwlrhFgXKeTHde6D5OaxuoymELzMT/ dHGtF80JxDKrOdJOf/bNKCMfhDLSyqEZQx0rhU7nS+Ct9sa9EEMZw69kipl03EAvbQTS 4G/A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=llpmptyH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id di3-20020a170906730300b006bde46ff2bcsi1453480ejc.140.2022.02.25.02.20.23; Fri, 25 Feb 2022 02:21:04 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20210112 header.b=llpmptyH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S237894AbiBYGne (ORCPT + 99 others); Fri, 25 Feb 2022 01:43:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34432 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232861AbiBYGnc (ORCPT ); Fri, 25 Feb 2022 01:43:32 -0500 Received: from mail-il1-x134.google.com (mail-il1-x134.google.com [IPv6:2607:f8b0:4864:20::134]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4E1487DAB2 for ; Thu, 24 Feb 2022 22:43:01 -0800 (PST) Received: by mail-il1-x134.google.com with SMTP id i1so3589895ila.7 for ; Thu, 24 Feb 2022 22:43:01 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20210112; h=date:from:to:cc:subject:message-id:references:mime-version :content-disposition:in-reply-to; bh=3WKwDOfvIucdHL9pK/JQ4fGfUfoOR9rKDu+vz9mXHr0=; b=llpmptyH/Oo89tS4JSWIPeZ4BB1PE4VD/2fgg5tQ89Ip2J0B0xoM/23xz9EidgWjBZ IAj8+D07jmB0szwX5VJkQ8Wmz5xMZIE/R2h0qXtROL2dcfEKWumRAPDHUvIIIHUjsLiD +1YQnChbotcSy7w5cIcUW+I1n3xixvcc/BlPksoFenBb7ZMLb2Oauh3pdirk7dbV1oT2 N5IbtmwoelB+UsN+RnchWIYH4S89d5kxFkBfi0N0apDI2TcIfEq7gqpyHg8dHWl+DFBv 4t9cP2AcffdiUalQ26Z/jg9PfsSL2/bPia3CQzhq1rvlL+MY1Rvus/Jmq9vWTig79XsD 6BiA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:date:from:to:cc:subject:message-id:references :mime-version:content-disposition:in-reply-to; bh=3WKwDOfvIucdHL9pK/JQ4fGfUfoOR9rKDu+vz9mXHr0=; b=y6yoGTIwgDCquuOwviZQR/ybzUq0owQnPbogcyGnJ5qCRMi3ifgKHqaR9VDqy9+IBi eql4YLK/ypOn7cfpate3KQ1HKqabyrau1Vzx7I0N+jWqDGUwMB/Og4o3Pn8Nipykdrz1 nex0jG+ZQrklTCQvT7wp3RTSs3oIK3ZLVwnLlmmIbzjSX1h9BabXp1YxRTOb7AGu6xkZ xP3rBufEAKOFZIJrviLnZ+yaSi5qLzvJPME27rxL64FJBOyjB8DzKXp6EwEY8/S/xBY3 qqjmXsxdfPEyDhW8+vJgmon6csIG97p1lkrXvGdkcm5J23dfu+IxmsAFz3bD1d9c6yGk 1Xvg== X-Gm-Message-State: AOAM533ik1Pjm0VBqM4+nfptIbg7xpHc+YpOj+Y+4FOsdnjzgKN3tE2R XfHUdS+FgTcSICudfqxYybjTng== X-Received: by 2002:a05:6e02:54e:b0:2b8:b9af:731a with SMTP id i14-20020a056e02054e00b002b8b9af731amr5349458ils.205.1645771380280; Thu, 24 Feb 2022 22:43:00 -0800 (PST) Received: from google.com (194.225.68.34.bc.googleusercontent.com. [34.68.225.194]) by smtp.gmail.com with ESMTPSA id y4-20020a056e02118400b002c274deca4esm1158065ili.0.2022.02.24.22.42.59 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 24 Feb 2022 22:42:59 -0800 (PST) Date: Fri, 25 Feb 2022 06:42:56 +0000 From: Oliver Upton To: Raghavendra Rao Ananta Cc: Marc Zyngier , Andrew Jones , James Morse , Alexandru Elisei , Suzuki K Poulose , Paolo Bonzini , Catalin Marinas , Will Deacon , Peter Shier , Ricardo Koller , Reiji Watanabe , Jing Zhang , linux-arm-kernel@lists.infradead.org, kvmarm@lists.cs.columbia.edu, linux-kernel@vger.kernel.org, kvm@vger.kernel.org Subject: Re: [PATCH v4 02/13] KVM: arm64: Introduce KVM_CAP_ARM_REG_SCOPE Message-ID: References: <20220224172559.4170192-1-rananta@google.com> <20220224172559.4170192-3-rananta@google.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20220224172559.4170192-3-rananta@google.com> X-Spam-Status: No, score=-17.6 required=5.0 tests=BAYES_00,DKIMWL_WL_MED, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF, ENV_AND_HDR_SPF_MATCH,RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS, T_SCC_BODY_TEXT_LINE,USER_IN_DEF_DKIM_WL,USER_IN_DEF_SPF_WL autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Feb 24, 2022 at 05:25:48PM +0000, Raghavendra Rao Ananta wrote: > KVM_[GET|SET]_ONE_REG act on per-vCPU basis. Currently certain > ARM64 registers, such as KVM_REG_ARM_SMCCC_ARCH_WORKAROUND_[1|2], > are accessed via this interface even though the effect that > they have are really per-VM. As a result, userspace could just > waste cycles to read/write the same information for every vCPU > that it spawns, only to realize that there's absolutely no change > in the VM's state. The problem gets worse in proportion to the > number of vCPUs created. > > As a result, to avoid this redundancy, introduce the capability > KVM_CAP_ARM_REG_SCOPE. If enabled, KVM_GET_REG_LIST will advertise > the registers that are VM-scoped by dynamically modifying the > register encoding. KVM_REG_ARM_SCOPE_* helper macros are introduced > to decode the same. By learning this, userspace can access such > registers only once. > > Signed-off-by: Raghavendra Rao Ananta > --- > Documentation/virt/kvm/api.rst | 16 ++++++++++++++++ > arch/arm64/include/asm/kvm_host.h | 3 +++ > arch/arm64/include/uapi/asm/kvm.h | 6 ++++++ > arch/arm64/kvm/arm.c | 13 +++++++------ > include/uapi/linux/kvm.h | 1 + > 5 files changed, 33 insertions(+), 6 deletions(-) > > diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst > index a4267104db50..7e7b3439f540 100644 > --- a/Documentation/virt/kvm/api.rst > +++ b/Documentation/virt/kvm/api.rst > @@ -7561,3 +7561,19 @@ The argument to KVM_ENABLE_CAP is also a bitmask, and must be a subset > of the result of KVM_CHECK_EXTENSION. KVM will forward to userspace > the hypercalls whose corresponding bit is in the argument, and return > ENOSYS for the others. > + > +8.34 KVM_CAP_ARM_REG_SCOPE > +-------------------------- > + > +:Architectures: arm64 > + > +The capability, if enabled, amends the existing register encoding > +with additional information to the userspace if a particular register > +is scoped per-vCPU or per-VM via KVM_GET_REG_LIST. KVM provides > +KVM_REG_ARM_SCOPE_* helper macros to decode the same. Userspace can > +use this information from the register encoding to access a VM-scopped > +regiser only once, as opposed to accessing it for every vCPU for the > +same effect. > + Could you describe the encoding changes in 4.68 'KVM_SET_ONE_REG', along with the other ARM encoding details? > +On the other hand, if the capability is disabled, all the registers > +remain vCPU-scopped by default, retaining backward compatibility. typo: vCPU-scoped That said, I don't believe we need to document behavior if the CAP is disabled, as the implicated ioctls should continue to work the same. > diff --git a/arch/arm64/include/asm/kvm_host.h b/arch/arm64/include/asm/kvm_host.h > index 5bc01e62c08a..8132de6bd718 100644 > --- a/arch/arm64/include/asm/kvm_host.h > +++ b/arch/arm64/include/asm/kvm_host.h > @@ -136,6 +136,9 @@ struct kvm_arch { > > /* Memory Tagging Extension enabled for the guest */ > bool mte_enabled; > + > + /* Register scoping enabled for KVM registers */ > + bool reg_scope_enabled; > }; > > struct kvm_vcpu_fault_info { > diff --git a/arch/arm64/include/uapi/asm/kvm.h b/arch/arm64/include/uapi/asm/kvm.h > index b3edde68bc3e..c35447cc0e0c 100644 > --- a/arch/arm64/include/uapi/asm/kvm.h > +++ b/arch/arm64/include/uapi/asm/kvm.h > @@ -199,6 +199,12 @@ struct kvm_arm_copy_mte_tags { > #define KVM_REG_ARM_COPROC_MASK 0x000000000FFF0000 > #define KVM_REG_ARM_COPROC_SHIFT 16 > > +/* Defines if a KVM register is one per-vCPU or one per-VM */ > +#define KVM_REG_ARM_SCOPE_MASK 0x0000000010000000 > +#define KVM_REG_ARM_SCOPE_SHIFT 28 Thinking about the advertisement of VM- and vCPU-scoped registers, this could be generally useful. Might it make sense to add such an encoding to the arch-generic register definitions? If that is the case, we may want to snap up a few more bits (a nybble) for future expansion. > +#define KVM_REG_ARM_SCOPE_VCPU 0 > +#define KVM_REG_ARM_SCOPE_VM 1 > + > /* Normal registers are mapped as coprocessor 16. */ > #define KVM_REG_ARM_CORE (0x0010 << KVM_REG_ARM_COPROC_SHIFT) > #define KVM_REG_ARM_CORE_REG(name) (offsetof(struct kvm_regs, name) / sizeof(__u32)) > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > index ecc5958e27fe..107977c82c6c 100644 > --- a/arch/arm64/kvm/arm.c > +++ b/arch/arm64/kvm/arm.c > @@ -81,26 +81,26 @@ int kvm_arch_check_processor_compat(void *opaque) > int kvm_vm_ioctl_enable_cap(struct kvm *kvm, > struct kvm_enable_cap *cap) > { > - int r; > + int r = 0; > > if (cap->flags) > return -EINVAL; > > switch (cap->cap) { > case KVM_CAP_ARM_NISV_TO_USER: > - r = 0; > kvm->arch.return_nisv_io_abort_to_user = true; > break; > case KVM_CAP_ARM_MTE: > mutex_lock(&kvm->lock); > - if (!system_supports_mte() || kvm->created_vcpus) { > + if (!system_supports_mte() || kvm->created_vcpus) > r = -EINVAL; > - } else { > - r = 0; > + else > kvm->arch.mte_enabled = true; > - } > mutex_unlock(&kvm->lock); > break; Hmm.. these all look like cleanups. If you want to propose these, could you do it in a separate patch? > + case KVM_CAP_ARM_REG_SCOPE: > + WRITE_ONCE(kvm->arch.reg_scope_enabled, true); > + break; > default: > r = -EINVAL; > break; > @@ -209,6 +209,7 @@ int kvm_vm_ioctl_check_extension(struct kvm *kvm, long ext) > case KVM_CAP_SET_GUEST_DEBUG: > case KVM_CAP_VCPU_ATTRIBUTES: > case KVM_CAP_PTP_KVM: > + case KVM_CAP_ARM_REG_SCOPE: It is a bit odd to advertise a capability (and allow userspace to enable it), despite the fact that the feature itself hasn't yet been implemented. Is it possible to fold the feature in to the patch that exposes it to userspace? Otherwise, you could punt advertisement of the CAP until it is actually implemented in kernel. -- Oliver