Received-SPF: pass (google.com: domain of linux-kernel+bounces-147486-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Precedence: bulk
MIME-Version: 1.0
References: <20240215160136.1256084-1-alejandro.j.jimenez@oracle.com> <Zh6-h0lBCpYBahw7@google.com>
In-Reply-To: <Zh6-h0lBCpYBahw7@google.com>
From: Paolo Bonzini <pbonzini@redhat.com>
Date: Tue, 16 Apr 2024 21:51:10 +0200
Message-ID: <CABgObfZ4kqaXLaOAOj4aGB5GAe9GxOmJmOP+7kdke6OqA35HzA@mail.gmail.com>
Subject: Re: [RFC 0/3] Export APICv-related state via binary stats interface
To: Sean Christopherson <seanjc@google.com>
Cc: Alejandro Jimenez <alejandro.j.jimenez@oracle.com>, kvm@vger.kernel.org, 
	linux-kernel@vger.kernel.org, joao.m.martins@oracle.com, 
	boris.ostrovsky@oracle.com, mark.kanda@oracle.com, 
	suravee.suthikulpanit@amd.com, mlevitsk@redhat.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Apr 16, 2024 at 8:08=E2=80=AFPM Sean Christopherson <seanjc@google.=
com> wrote:
>
> On Thu, Feb 15, 2024, Alejandro Jimenez wrote:
> > The goal of this RFC is to agree on a mechanism for querying the state =
(and
> > related stats) of APICv/AVIC. I clearly have an AVIC bias when approach=
ing this
> > topic since that is the side that I have mostly looked at, and has the =
greater
> > number of possible inhibits, but I believe the argument applies for bot=
h
> > vendor's technologies.
> >
> > Currently, a user or monitoring app trying to determine if APICv is act=
ually
> > being used needs implementation-specific knowlegde in order to look for=
 specific
> > types of #VMEXIT (i.e. AVIC_INCOMPLETE_IPI/AVIC_NOACCEL), checking GALo=
g events
> > by watching /proc/interrupts for AMD-Vi*-GA, etc. There are existing tr=
acepoints
> > (e.g. kvm_apicv_accept_irq, kvm_avic_ga_log) that make this task easier=
, but
> > tracefs is not viable in some scenarios. Adding kvm debugfs entries has=
 similar
> > downsides. Suravee has previously proposed a new IOCTL interface[0] to =
expose
> > this information, but there has not been any development in that direct=
ion.
> > Sean has mentioned a preference for using BPF to extract info from the =
current
> > tracepoints, which would require reworking existing structs to access s=
ome
> > desired data, but as far as I know there isn't any work done on that ap=
proach
> > yet.
> >
> > Recently Joao mentioned another alternative: the binary stats framework=
 that is
> > already supported by kernel[1] and QEMU[2].
>
> The hiccup with stats are that they are ABI, e.g. we can't (easily) ditch=
 stats
> once they're added, and KVM needs to maintain the exact behavior.

Stats are not ABI---why would they be? They have an established
meaning and it's not a good idea to change it, but it's not an
absolute no-no(*); and removing them is not a problem at all.

For example, if stats were ABI, there would be no need for the
introspection mechanism, you could just use a struct like ethtool
stats (which *are* ABO).

Not everything makes a good stat but, if in doubt and it's cheap
enough to collect it, go ahead and add it. Cheap collection is the
important point, because tracepoints in a hot path can be so expensive
as to slow down the guest substantially, at least in microbenchmarks.

In this case I'm not sure _all_ inhibits makes sense and I certainly
wouldn't want a bitmask, but a generic APICv-enabled stat certainly
makes sense, and perhaps another for a weirdly-configured local APIC.

Paolo

(*) you have to draw a line somewhere. New processor models may
virtualize parts of the CPU in such a way that some stats become
meaningless or just stay at zero. Should KVM not support those
features because it is not possible anymore to introspect the guest
through stat?

> Tracepoints are explicitly not ABI, and so we can be much more permissive=
 when it
> comes to adding/expanding tracepoints, specifically because there are no =
guarantees
> provided to userspace.
>