Received: by 2002:a05:6a10:22f:0:0:0:0 with SMTP id 15csp40168pxk; Tue, 1 Sep 2020 15:36:26 -0700 (PDT) X-Google-Smtp-Source: ABdhPJzaozwJ4KkJc+LPn8KOwOEawBeJeSrtfbHSrtIAufW9igfDa0ppjUaJGKL6vapzsHGbRLvF X-Received: by 2002:a17:906:2a0d:: with SMTP id j13mr3766884eje.474.1598999786744; Tue, 01 Sep 2020 15:36:26 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1598999786; cv=none; d=google.com; s=arc-20160816; b=fp0MvbnDe88pbLK9xXNfiz/quUf450doY6U55hli2sg69jO8cHlP/NoyfVaZsPwp5+ s5wdJtV66n5yuEqZ+M5zF/6mDKisscmvnSrTkbU6cw07DXAuTemHMJYACzdsVLhFzbzu iLG+ZuryjWIJBlOKcDr+nSGbG9LWVQNHZpNNvnkawxsU0Fy9Ayg3YowdSE2g3Gpey8yw v/WyPKE71lYTOtnNlt03P7tg0YakotB153dJMWAEjg5SlWlxDsWJpwqVsi7RgxJbRtVI pgsVZuxhpcyRg0EzX5oH9ifZEopWgSjRWotzVUSZw8stTS/5Jygy8/MQxQ25H6prtiRW 4MIw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=Y8Kqs1FhyFxYDeoHU6Zu55afEGALtbuNFoge6VETFuE=; b=MECBxrDjUvYZYxhlDP204UkigMECq6X9KMWMv4f5egbTeij+/QJ2gScLdAqEc0GmXk RiSeEVyZkPHih8qqnfJ0hMZCySxEFKNPicXc+3DQ1xu38WyRmrzIZ+84WbujcsiYne77 kcavLJx6hSHt/afkZ1PGVhitL8ocnGCR2VmCeFZcUctSJ6gcPuIoTZBF8/Zdv9Rj4p89 Pmq6lhfNeKrya8bmMVYXgxbfdIV5yYzqhaDiJZpAgBrk0r7dVuJTPUqB0wx2Uop8PBh2 7trR5GhiWcMGz+8VvqWpoq8Yzw8UxYWp1b5/9QZNR7CxIAAjB0BSi7LGHOYVuCG3wQQu q0yA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=aKWdEI0n; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id x6si697325edv.132.2020.09.01.15.36.03; Tue, 01 Sep 2020 15:36:26 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=aKWdEI0n; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727115AbgIAWc0 (ORCPT + 99 others); Tue, 1 Sep 2020 18:32:26 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:34770 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1728309AbgIAWcW (ORCPT ); Tue, 1 Sep 2020 18:32:22 -0400 Received: from mail-ed1-x542.google.com (mail-ed1-x542.google.com [IPv6:2a00:1450:4864:20::542]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E82EAC061246 for ; Tue, 1 Sep 2020 15:32:21 -0700 (PDT) Received: by mail-ed1-x542.google.com with SMTP id l17so2992666edq.12 for ; Tue, 01 Sep 2020 15:32:21 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=Y8Kqs1FhyFxYDeoHU6Zu55afEGALtbuNFoge6VETFuE=; b=aKWdEI0nW4hHs1YnMgp3+Hrkr++hc9rpCLsA/xs2TXIbOrgye7EGHDWiTjH5JePfL3 Wcr9IQwQcPco2v37BDFuyQ+GOA/wM4GTM6Vzi+y6aiUXO0unFhYlQzdRDEH/A2niaF2h UxUo9uWDPQwYdCroC7inEuxXANNyMfw2bj3hPPRXHJ/doLXiR/833fmPUMAenEGWXicA 9hlvZcn4JQYBvDFeW5le1YTyjQTnRO9LR/tfK1AVeRZB3Uw6D9tiQIowJVWgmGcERZot Da7c85cogEK0mEyM2zWWAcYk1G0nIwYmbJQdKXTjGHZeCS/2+u1uHtdGcIbjyOajxnlI ZOVg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=Y8Kqs1FhyFxYDeoHU6Zu55afEGALtbuNFoge6VETFuE=; b=e/HTf9x57fFPDVx1zp+f4Wjckxc5POMo81HnMbLMnfchl9fbKj0OP9CiJ18MTr/aeN AO6ZWXUMKc4iBzOWNueApIkxmT7S1pOo6+CPdTBlfoBkZ4QJp+olRnKJr2FlsMGMI7Yo 92DpwTs/LPhCowlQI4LTZ47MOOPUH2Am1VHSTqXmnEhKdkj2leKnB4QhYB0/6cpvjdzc XvOI/XdBnu7JPHgSh2+wwY2lgs9KVwQnDEqVsdL2efR2lhnmCxaSGN6Wlyk9219lu6/b aX23m4z+HlgoafRjb7cizgYJDJfk9sXIOT592wqJi/vO4w1pXk4gLpETr/4vzfogInso koYQ== X-Gm-Message-State: AOAM530xynWKgmHIDDanvrjWjbEdAy01MD+bTyxNFKr7rym2JuYocUiY 2MY9PXYkxNiiC4Q+qKwl5Op6pT5ajKJUW56psCl83Q== X-Received: by 2002:aa7:d68c:: with SMTP id d12mr3939682edr.274.1598999540116; Tue, 01 Sep 2020 15:32:20 -0700 (PDT) MIME-Version: 1.0 References: <20200901201517.29086-1-graf@amazon.com> In-Reply-To: <20200901201517.29086-1-graf@amazon.com> From: Aaron Lewis Date: Tue, 1 Sep 2020 15:32:08 -0700 Message-ID: Subject: Re: [PATCH v6 0/7] Allow user space to restrict and augment MSR emulation To: Alexander Graf Cc: Paolo Bonzini , Jonathan Corbet , Sean Christopherson , Vitaly Kuznetsov , Wanpeng Li , Jim Mattson , Joerg Roedel , KarimAllah Raslan , Dan Carpenter , kvm list , linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Tue, Sep 1, 2020 at 1:15 PM Alexander Graf wrote: > > While tying to add support for the MSR_CORE_THREAD_COUNT MSR in KVM, > I realized that we were still in a world where user space has no control > over what happens with MSR emulation in KVM. > > That is bad for multiple reasons. In my case, I wanted to emulate the > MSR in user space, because it's a CPU specific register that does not > exist on older CPUs and that really only contains informational data that > is on the package level, so it's a natural fit for user space to provide > it. > > However, it is also bad on a platform compatibility level. Currrently, > KVM has no way to expose different MSRs based on the selected target CPU > type. > > This patch set introduces a way for user space to indicate to KVM which > MSRs should be handled in kernel space. With that, we can solve part of > the platform compatibility story. Or at least we can not handle AMD specific > MSRs on an Intel platform and vice versa. > > In addition, it introduces a way for user space to get into the loop > when an MSR access would generate a #GP fault, such as when KVM finds an > MSR that is not handled by the in-kernel MSR emulation or when the guest > is trying to access reserved registers. > > In combination with filtering, user space trapping allows us to emulate > arbitrary MSRs in user space, paving the way for target CPU specific MSR > implementations from user space. > > v1 -> v2: > > - s/ETRAP_TO_USER_SPACE/ENOENT/g > - deflect all #GP injection events to user space, not just unknown MSRs. > That was we can also deflect allowlist errors later > - fix emulator case > - new patch: KVM: x86: Introduce allow list for MSR emulation > - new patch: KVM: selftests: Add test for user space MSR handling > > v2 -> v3: > > - return r if r == X86EMUL_IO_NEEDED > - s/KVM_EXIT_RDMSR/KVM_EXIT_X86_RDMSR/g > - s/KVM_EXIT_WRMSR/KVM_EXIT_X86_WRMSR/g > - Use complete_userspace_io logic instead of reply field > - Simplify trapping code > - document flags for KVM_X86_ADD_MSR_ALLOWLIST > - generalize exit path, always unlock when returning > - s/KVM_CAP_ADD_MSR_ALLOWLIST/KVM_CAP_X86_MSR_ALLOWLIST/g > - Add KVM_X86_CLEAR_MSR_ALLOWLIST > - Add test to clear whitelist > - Adjust to reply-less API > - Fix asserts > - Actually trap on MSR_IA32_POWER_CTL writes > > v3 -> v4: > > - Mention exit reasons in re-enter mandatory section of API documentation > - Clear padding bytes > - Generalize get/set deflect functions > - Remove redundant pending_user_msr field > - lock allow check and clearing > - free bitmaps on clear > > v4 -> v5: > > - use srcu > > v5 -> v6: > > - Switch from allow list to filtering API with explicit fallback option > - Support and test passthrough MSR filtering > - Check for filter exit reason > - Add .gitignore > - send filter change notification > - change to atomic set_msr_filter ioctl with fallback flag > - use EPERM for filter blocks > - add bit for MSR user space deflection > - check for overflow of BITS_TO_LONGS (thanks Dan Carpenter!) > - s/int i;/u32 i;/ > - remove overlap check > - Introduce exit reason mask to allow for future expansion and filtering > - s/emul_to_vcpu(ctxt)/vcpu/ > - imported patch: KVM: x86: Prepare MSR bitmaps for userspace tracked MSRs > - new patch: KVM: x86: Add infrastructure for MSR filtering > - new patch: KVM: x86: SVM: Prevent MSR passthrough when MSR access is denied > - new patch: KVM: x86: VMX: Prevent MSR passthrough when MSR access is denied > > Aaron Lewis (1): > KVM: x86: Prepare MSR bitmaps for userspace tracked MSRs > > Alexander Graf (6): > KVM: x86: Deflect unknown MSR accesses to user space > KVM: x86: Add infrastructure for MSR filtering > KVM: x86: SVM: Prevent MSR passthrough when MSR access is denied > KVM: x86: VMX: Prevent MSR passthrough when MSR access is denied > KVM: x86: Introduce MSR filtering > KVM: selftests: Add test for user space MSR handling > > Documentation/virt/kvm/api.rst | 176 +++++++++- > arch/x86/include/asm/kvm_host.h | 18 ++ > arch/x86/include/uapi/asm/kvm.h | 19 ++ > arch/x86/kvm/emulate.c | 18 +- > arch/x86/kvm/svm/svm.c | 122 +++++-- > arch/x86/kvm/svm/svm.h | 7 + > arch/x86/kvm/vmx/nested.c | 2 +- > arch/x86/kvm/vmx/vmx.c | 303 ++++++++++++------ > arch/x86/kvm/vmx/vmx.h | 9 +- > arch/x86/kvm/x86.c | 267 ++++++++++++++- > arch/x86/kvm/x86.h | 1 + > include/trace/events/kvm.h | 2 +- > include/uapi/linux/kvm.h | 17 + > tools/testing/selftests/kvm/.gitignore | 1 + > tools/testing/selftests/kvm/Makefile | 1 + > .../selftests/kvm/x86_64/user_msr_test.c | 224 +++++++++++++ > 16 files changed, 1055 insertions(+), 132 deletions(-) > create mode 100644 tools/testing/selftests/kvm/x86_64/user_msr_test.c > > -- > 2.17.1 > > > > > Amazon Development Center Germany GmbH > Krausenstr. 38 > 10117 Berlin > Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss > Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B > Sitz: Berlin > Ust-ID: DE 289 237 879 > > > Hi Alex, I'm only seeing 4 commits. Are you planning on sending the remaining 3? Thanks, Aaron