Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33;
Message-ID: <3dc66987-49c7-abda-eb70-1898181ef3fe@redhat.com>
Date:   Sat, 23 Sep 2023 11:24:44 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.13.0
Content-Language: en-US
To:     David Woodhouse <dwmw2@infradead.org>
Cc:     kvm@vger.kernel.org, Peter Zijlstra <peterz@infradead.org>,
        Sean Christopherson <seanjc@google.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, x86@kernel.org,
        "H. Peter Anvin" <hpa@zytor.com>, linux-kernel@vger.kernel.org,
        graf@amazon.de, Nicolas Saenz Julienne <nsaenz@amazon.es>,
        "Griffoul, Fred" <fgriffo@amazon.com>
References: <1b52b557beb6606007f7ec5672eab0adf1606a34.camel@infradead.org>
 <CABgObfZgYXaXqP=6s53=+mYWvOnbgYJiCRct-0ob444sK9SvGw@mail.gmail.com>
 <faec494b6df5ebee5644017c9415e747bd34952b.camel@infradead.org>
From:   Paolo Bonzini <pbonzini@redhat.com>
Subject: Re: [RFC] KVM: x86: Allow userspace exit on HLT and MWAIT, else yield
 on MWAIT
In-Reply-To: <faec494b6df5ebee5644017c9415e747bd34952b.camel@infradead.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 9/23/23 09:22, David Woodhouse wrote:
> On Fri, 2023-09-22 at 14:00 +0200, Paolo Bonzini wrote:
>> To avoid races you need two flags though; there needs to be also a
>> kernel->userspace communication of whether the vCPU is currently in
>> HLT or MWAIT, using the "flags" field for example. If it was HLT only,
>> moving the mp_state in kvm_run would seem like a good idea; but not if
>> MWAIT or PAUSE are also included.
> 
> Right. When work is added to an empty workqueue, the VMM will want to
> hunt for a vCPU which is currently idle and then signal it to exit.
> 
> As you say, for HLT it's simple enough to look at the mp_state, and we
> can move that into kvm_run so it doesn't need an ioctl...

Looking at it again: not so easy because the mpstate is changed in the 
vCPU thread by vcpu_block() itself.

> although it
> would also be nice to get an *event* on an eventfd when the vCPU
> becomes runnable (as noted, we want that for VSM anyway). Or perhaps
> even to be able to poll() on the vCPU fd.

Why do you need it?  You can just use KVM_RUN to go to sleep, and if you 
get another job you kick out the vCPU with pthread_kill.  (I also didn't 
get the VSM reference).

An interesting quirk is that kvm_run->immediate_exit is processed before 
kvm_vcpu_block(), but TIF_SIGPENDING is processed afterwards.  This 
means that you can force an mpstate update with pthread_kill + KVM_RUN. 
It's not going to be a speed demon, but it's worth writing a selftest 
for it.

> But MWAIT (as currently not-really-emulated) and PAUSE are both just
> transient states with nothing you can really *wait* for, which is why
> they're such fun to deal with.

PAUSE is easier because it is just momentary and you stick it inside 
what's already a busy wait.  MWAIT is less fun because you don't really 
want to busy wait.

Paolo