Received-SPF: pass (google.com: domain of linux-kernel+bounces-199862-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Mon, 3 Jun 2024 17:14:11 -0700
In-Reply-To: <20240429155738.990025-5-alejandro.j.jimenez@oracle.com>
Precedence: bulk
Mime-Version: 1.0
References: <20240429155738.990025-1-alejandro.j.jimenez@oracle.com> <20240429155738.990025-5-alejandro.j.jimenez@oracle.com>
Message-ID: <Zl5cUwGiMrng2zcc@google.com>
Subject: Re: [PATCH 4/4] KVM: x86: Add vCPU stat for APICv interrupt
 injections causing #VMEXIT
From: Sean Christopherson <seanjc@google.com>
To: Alejandro Jimenez <alejandro.j.jimenez@oracle.com>
Cc: kvm@vger.kernel.org, pbonzini@redhat.com, linux-kernel@vger.kernel.org, 
	suravee.suthikulpanit@amd.com, vashegde@amd.com, mlevitsk@redhat.com, 
	joao.m.martins@oracle.com, boris.ostrovsky@oracle.com, mark.kanda@oracle.com
Content-Type: text/plain; charset="us-ascii"

On Mon, Apr 29, 2024, Alejandro Jimenez wrote:
> Even when APICv/AVIC is active, certain guest accesses to its local APIC(s)
> cannot be fully accelerated, and cause a #VMEXIT to allow the VMM to
> emulate the behavior and side effects. Expose a counter stat for these
> specific #VMEXIT types.
> 
> Suggested-by: Paolo Bonzini <pbonzini@redhat.com>
> Signed-off-by: Alejandro Jimenez <alejandro.j.jimenez@oracle.com>
> ---
>  arch/x86/include/asm/kvm_host.h | 1 +
>  arch/x86/kvm/svm/avic.c         | 7 +++++++
>  arch/x86/kvm/vmx/vmx.c          | 2 ++
>  arch/x86/kvm/x86.c              | 1 +
>  4 files changed, 11 insertions(+)
> 
> diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
> index e7e3213cefae..388979dfe9f3 100644
> --- a/arch/x86/include/asm/kvm_host.h
> +++ b/arch/x86/include/asm/kvm_host.h
> @@ -1576,6 +1576,7 @@ struct kvm_vcpu_stat {
>  	u64 guest_mode;
>  	u64 notify_window_exits;
>  	u64 apicv_active;
> +	u64 apicv_unaccelerated_inj;

The stat name doesn't match the changelog or the code.  The AVIC updates in
avic_incomplete_ipi_interception() are unaccelerated _injection_, they're
unaccelarated _delivery_.  And in those cases, the fact that delivery wasn't
accelerated is relatively uninteresting in most cases.

And avic_unaccelerated_access_interception() and handle_apic_write() don't
necessarily have anything to do with injection.

On the flip side, the slow paths for {svm,vmx}_deliver_interrupt() are very
explicitly unnaccelerated injection.

It's not entirely clear from the changelog what the end goal of this stat is.
A singular stat for all APICv/AVIC access VM-Exits seems uninteresting, as such
a stat essentially just captures that the guest is active.  Maaaybe someone could
glean info from comparing two VMs, but even that is dubious.  E.g. if a guest is
doing something function and generating a lot of avic_incomplete_ipi_interception()
exits, those will likely be in the noise due to the total volume of other AVIC
exits.

>  };
>  
>  struct x86_instruction_info;
> diff --git a/arch/x86/kvm/svm/avic.c b/arch/x86/kvm/svm/avic.c
> index 4b74ea91f4e6..274041d3cf66 100644
> --- a/arch/x86/kvm/svm/avic.c
> +++ b/arch/x86/kvm/svm/avic.c
> @@ -517,6 +517,8 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu)
>  			kvm_apic_write_nodecode(vcpu, APIC_ICR);
>  		else
>  			kvm_apic_send_ipi(apic, icrl, icrh);
> +
> +		++vcpu->stat.apicv_unaccelerated_inj;
>  		break;
>  	case AVIC_IPI_FAILURE_TARGET_NOT_RUNNING:
>  		/*
> @@ -525,6 +527,8 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu)
>  		 * vcpus. So, we just need to kick the appropriate vcpu.
>  		 */
>  		avic_kick_target_vcpus(vcpu->kvm, apic, icrl, icrh, index);
> +
> +		++vcpu->stat.apicv_unaccelerated_inj;
>  		break;
>  	case AVIC_IPI_FAILURE_INVALID_BACKING_PAGE:
>  		WARN_ONCE(1, "Invalid backing page\n");
> @@ -704,6 +708,9 @@ int avic_unaccelerated_access_interception(struct kvm_vcpu *vcpu)
>  
>  	trace_kvm_avic_unaccelerated_access(vcpu->vcpu_id, offset,
>  					    trap, write, vector);
> +
> +	++vcpu->stat.apicv_unaccelerated_inj;
> +
>  	if (trap) {
>  		/* Handling Trap */
>  		WARN_ONCE(!write, "svm: Handling trap read.\n");
> diff --git a/arch/x86/kvm/vmx/vmx.c b/arch/x86/kvm/vmx/vmx.c
> index f10b5f8f364b..a7487f12ded1 100644
> --- a/arch/x86/kvm/vmx/vmx.c
> +++ b/arch/x86/kvm/vmx/vmx.c
> @@ -5657,6 +5657,8 @@ static int handle_apic_write(struct kvm_vcpu *vcpu)
>  {
>  	unsigned long exit_qualification = vmx_get_exit_qual(vcpu);
>  
> +	++vcpu->stat.apicv_unaccelerated_inj;
> +
>  	/*
>  	 * APIC-write VM-Exit is trap-like, KVM doesn't need to advance RIP and
>  	 * hardware has done any necessary aliasing, offset adjustments, etc...
> diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
> index 03cb933920cb..c8730b0fac87 100644
> --- a/arch/x86/kvm/x86.c
> +++ b/arch/x86/kvm/x86.c
> @@ -307,6 +307,7 @@ const struct _kvm_stats_desc kvm_vcpu_stats_desc[] = {
>  	STATS_DESC_IBOOLEAN(VCPU, guest_mode),
>  	STATS_DESC_COUNTER(VCPU, notify_window_exits),
>  	STATS_DESC_IBOOLEAN(VCPU, apicv_active),
> +	STATS_DESC_COUNTER(VCPU, apicv_unaccelerated_inj),
>  };
>  
>  const struct kvm_stats_header kvm_vcpu_stats_header = {
> -- 
> 2.39.3
>