Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:7 as permitted sender) client-ip=2620:137:e000::3:7;
Date:   Tue, 26 Sep 2023 13:29:36 -0700
In-Reply-To: <21C2A5D8-66D9-4EF0-A416-4B1049C91E83@infradead.org>
Mime-Version: 1.0
References: <1b52b557beb6606007f7ec5672eab0adf1606a34.camel@infradead.org>
 <CABgObfZgYXaXqP=6s53=+mYWvOnbgYJiCRct-0ob444sK9SvGw@mail.gmail.com>
 <faec494b6df5ebee5644017c9415e747bd34952b.camel@infradead.org>
 <3dc66987-49c7-abda-eb70-1898181ef3fe@redhat.com> <d3e0c3e9-4994-4808-a8df-3d23487ff9c4@amazon.de>
 <CABgObfZb4CvzpnSJxz9saw8PJeo1Y2=0uB9y4_K+Cu9P9FpF6g@mail.gmail.com> <21C2A5D8-66D9-4EF0-A416-4B1049C91E83@infradead.org>
Message-ID: <ZRM_MHBpePiAQ__1@google.com>
Subject: Re: [RFC] KVM: x86: Allow userspace exit on HLT and MWAIT, else yield
 on MWAIT
From:   Sean Christopherson <seanjc@google.com>
To:     David Woodhouse <dwmw2@infradead.org>
Cc:     Paolo Bonzini <pbonzini@redhat.com>,
        Alexander Graf <graf@amazon.de>, kvm@vger.kernel.org,
        Peter Zijlstra <peterz@infradead.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>, linux-kernel@vger.kernel.org,
        Nicolas Saenz Julienne <nsaenz@amazon.es>,
        Fred Griffoul <fgriffo@amazon.com>
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Tue, Sep 26, 2023, David Woodhouse wrote:
>=20
>=20
> On 26 September 2023 19:20:24 CEST, Paolo Bonzini <pbonzini@redhat.com> w=
rote:
> >On Sat, Sep 23, 2023 at 6:44=E2=80=AFPM Alexander Graf <graf@amazon.de> =
wrote:
> >> On 23.09.23 11:24, Paolo Bonzini wrote:
> >> > Why do you need it?  You can just use KVM_RUN to go to sleep, and if=
 you
> >> > get another job you kick out the vCPU with pthread_kill.  (I also di=
dn't
> >> > get the VSM reference).
> >>
> >> With the original VSM patches, we used to make a vCPU aware of the fac=
t
> >> that it can morph into one of many VTLs. That approach turned out to b=
e
> >> insanely intrusive and fragile and so we're currently reimplementing
> >> everything as VTLs as vCPUs. That allows us to move the majority of VS=
M
> >> functionality to user space. Everything we've seen so far looks as if
> >> there is no real performance loss with that approach.
> >
> >Yes, that was also what I remember, sharing the FPU somehow while
> >having separate vCPU file descriptors.
> >
> >> One small problem with that is that now user space is responsible for
> >> switching between VTLs: It determines which VTL is currently running a=
nd
> >> leaves all others (read: all other vCPUs) as stopped. That means if yo=
u
> >> are running happily in KVM_RUN in VTL0 and VTL1 gets an interrupt, use=
r
> >> space needs to stop VTL0 and unpause VTL1 until it triggers VTL_RETURN
> >> at which point VTL1 stops execution and VTL0 runs again.
> >
> >That's with IPIs in VTL1, right? I understand now. My idea was, since
> >we need a link from VTL1 to VTL0 for the FPU, to use the same link to
> >trigger a vmexit to userspace if source VTL > destination VTL. I am
> >not sure how you would handle the case where the destination vCPU is
> >not running; probably by detecting the IPI when VTL0 restarts on the
> >destination vCPU?
> >
> >In any case, making vCPUs poll()-able is sensible.
>=20
> Thinking about this a bit more, even for HLT it probably isn't just as si=
mple
> as checking for mp_state changes. If there's a REQ_EVENT outstanding for
> something like a timer delivery, that won't get handled and the IRQ actua=
lly
> delivered to the local APIC until the vCPU is actually *run*, will it?

I haven't been following this conversation, just reacting to seeing "HLT" a=
nd
"mp_state", which is a bit of a mess:

https://lore.kernel.org/all/ZMgIQ5m1jMSAogT4@google.com