Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <d1891527-1d22-f1c3-a1f6-b4a1a3f42051@redhat.com>
Date:   Fri, 24 Jun 2022 19:04:53 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.10.0
Subject: Re: [PATCH v6 00/17] Introducing AMD x2AVIC and hybrid-AVIC modes
Content-Language: en-US
To:     Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>,
        linux-kernel@vger.kernel.org, kvm@vger.kernel.org
Cc:     mlevitsk@redhat.com, seanjc@google.com, joro@8bytes.org,
        jon.grimm@amd.com, wei.huang2@amd.com, terry.bowman@amd.com
References: <20220519102709.24125-1-suravee.suthikulpanit@amd.com>
From:   Paolo Bonzini <pbonzini@redhat.com>
In-Reply-To: <20220519102709.24125-1-suravee.suthikulpanit@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 5/19/22 12:26, Suravee Suthikulpanit wrote:
> Introducing support for AMD x2APIC virtualization. This feature is
> indicated by the CPUID Fn8000_000A EDX[14], and it can be activated
> by setting bit 31 (enable AVIC) and bit 30 (x2APIC mode) of VMCB
> offset 60h.
> 
> With x2AVIC support, the guest local APIC can be fully virtualized in
> both xAPIC and x2APIC modes, and the mode can be changed during runtime.
> For example, when AVIC is enabled, the hypervisor set VMCB bit 31
> to activate AVIC for each vCPU. Then, it keeps track of each vCPU's
> APIC mode, and updates VMCB bit 30 to enable/disable x2APIC
> virtualization mode accordingly.
> 
> Besides setting bit VMCB bit 30 and 31, for x2AVIC, kvm_amd driver needs
> to disable interception for the x2APIC MSR range to allow AVIC hardware
> to virtualize register accesses.
> 
> This series also introduce a partial APIC virtualization (hybrid-AVIC)
> mode, where APIC register accesses are trapped (i.e. not virtualized
> by hardware), but leverage AVIC doorbell for interrupt injection.
> This eliminates need to disable x2APIC in the guest on system without
> x2AVIC support. (Note: suggested by Maxim)
> 
> Testing for v5:
>    * Test partial AVIC mode by launching a VM with x2APIC mode
>    * Tested booting a Linux VM with x2APIC physical and logical modes upto 512 vCPUs.
>    * Test the following nested SVM test use cases:
> 
>               L0     |    L1   |   L2
>         ----------------------------------
>                 AVIC |    APIC |    APIC
>                 AVIC |    APIC |  x2APIC
>          hybrid-AVIC |  x2APIC |    APIC
>          hybrid-AVIC |  x2APIC |  x2APIC
>               x2AVIC |    APIC |    APIC
>               x2AVIC |    APIC |  x2APIC
>               x2AVIC |  x2APIC |    APIC
>               x2AVIC |  x2APIC |  x2APIC
> 
> Changes from v5:
> (https://lore.kernel.org/lkml/20220518162652.100493-1-suravee.suthikulpanit@amd.com/T/#t)
>    * Re-order patch 16 to 10
>    * Patch 11: Update commit message
> 
> Changes from v4:
> (https://lore.kernel.org/lkml/20220508023930.12881-5-suravee.suthikulpanit@amd.com/T/)
>    * Patch  3: Move enum_avic_modes definition to svm.h
>    * Patch 10: Rename avic_set_x2apic_msr_interception to
>                svm_set_x2apic_msr_interception and move it to svm.c
>                to simplify the struct svm_direct_access_msrs declaration.
>    * Patch 16: New from Maxim
>    * Patch 17: New from Maxim
> 
> Best Regards,
> Suravee
> 
> Maxim Levitsky (2):
>    KVM: x86: nSVM: always intercept x2apic msrs
>    KVM: x86: nSVM: optimize svm_set_x2apic_msr_interception
> 
> Suravee Suthikulpanit (15):
>    x86/cpufeatures: Introduce x2AVIC CPUID bit
>    KVM: x86: lapic: Rename [GET/SET]_APIC_DEST_FIELD to
>      [GET/SET]_XAPIC_DEST_FIELD
>    KVM: SVM: Detect X2APIC virtualization (x2AVIC) support
>    KVM: SVM: Update max number of vCPUs supported for x2AVIC mode
>    KVM: SVM: Update avic_kick_target_vcpus to support 32-bit APIC ID
>    KVM: SVM: Do not support updating APIC ID when in x2APIC mode
>    KVM: SVM: Adding support for configuring x2APIC MSRs interception
>    KVM: x86: Deactivate APICv on vCPU with APIC disabled
>    KVM: SVM: Refresh AVIC configuration when changing APIC mode
>    KVM: SVM: Introduce logic to (de)activate x2AVIC mode
>    KVM: SVM: Do not throw warning when calling avic_vcpu_load on a
>      running vcpu
>    KVM: SVM: Introduce hybrid-AVIC mode
>    KVM: x86: Warning APICv inconsistency only when vcpu APIC mode is
>      valid
>    KVM: SVM: Use target APIC ID to complete x2AVIC IRQs when possible
>    KVM: SVM: Add AVIC doorbell tracepoint
> 
>   arch/x86/hyperv/hv_apic.c          |   2 +-
>   arch/x86/include/asm/apicdef.h     |   4 +-
>   arch/x86/include/asm/cpufeatures.h |   1 +
>   arch/x86/include/asm/kvm_host.h    |   1 -
>   arch/x86/include/asm/svm.h         |  16 ++-
>   arch/x86/kernel/apic/apic.c        |   2 +-
>   arch/x86/kernel/apic/ipi.c         |   2 +-
>   arch/x86/kvm/lapic.c               |   6 +-
>   arch/x86/kvm/svm/avic.c            | 178 ++++++++++++++++++++++++++---
>   arch/x86/kvm/svm/nested.c          |   5 +
>   arch/x86/kvm/svm/svm.c             |  75 ++++++++----
>   arch/x86/kvm/svm/svm.h             |  25 +++-
>   arch/x86/kvm/trace.h               |  18 +++
>   arch/x86/kvm/x86.c                 |   8 +-
>   14 files changed, 291 insertions(+), 52 deletions(-)
> 

I haven't quite finished reviewing this, but it passes both 
kvm-unit-tests and selftests so I pushed it to kvm/queue.

Paolo