Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.38 as permitted sender) client-ip=23.128.96.38;
Date:   Wed, 25 Oct 2023 13:42:30 +0100
Message-ID: <86cyx250w9.wl-maz@kernel.org>
From:   Marc Zyngier <maz@kernel.org>
To:     Jan Henrik Weinstock <jan@mwa.re>
Cc:     oliver.upton@linux.dev, james.morse@arm.com,
        suzuki.poulose@arm.com, yuzenghui@huawei.com,
        catalin.marinas@arm.com, will@kernel.org,
        linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev,
        linux-kernel@vger.kernel.org,
        Lukas =?UTF-8?B?SsO8bmdlcg==?= <lukas@mwa.re>
Subject: Re: KVM exit to userspace on WFI
In-Reply-To: <CANi1PHieGooO0DK=6BPwq0UknHzsn9QM3rFQkh3HLMfWxDseUQ@mail.gmail.com>
References: <CANi1PHhzk80HvwQbBM46gpJ6_AA_P6+m5Jo0Nuy_MAdA4C2BhQ@mail.gmail.com>
        <87ttql5aq7.wl-maz@kernel.org>
        <CANi1PHieGooO0DK=6BPwq0UknHzsn9QM3rFQkh3HLMfWxDseUQ@mail.gmail.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.1
 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk

On Wed, 25 Oct 2023 13:12:14 +0100,
Jan Henrik Weinstock <jan@mwa.re> wrote:
> 
> Hi Marc,
> 
> Thanks for your feedback. I understand that request_interrupt_window
> is not to be used. I assume a setting a flag is a better way,
> something similar to KVM_ARCH_FLAG_RETURN_NISV_IO_ABORT_TO_USER, e.g.
> KVM_ARCH_FLAG_WFX_EXIT_TO_USER.
> 
> I will also check that WFx traps are always enabled while this mode is
> active to make sure userspace does not get blocked/scheduled out.

Why would that be an acceptable behaviour?

> The reason for this is that we cannot have the thread that executes
> KVM_RUN to be blocked or scheduled out whenever it hits a WFI.

Why? If that's not acceptable, how do you even cope with the basic
preemption?

> Nop-WFIs are not a problem, since the PE will just continue executing
> instructions, which is fine. We are currently using a timeout signal
> that kicks KVM_RUN back into userspace, but we are seeing a lot of
> time wasted because our KVM thread hangs in WFI/WFEs. It would be
> better if we could just return from KVM_RUN immediately if the thread
> would otherwise be blocked.

On the face of it, this makes little sense:

- While in userspace, no interrupt source that normally delivered
  without any userpsace intervention will be blocked (timers,
  VLPIs...). I cannot how this can be a good idea.

- Trapping WFE is an important scheduling hint, and returning to
  userspace defeats it. Contended spinlocks, for example, will be even
  slower to acquire.

I'm sure you have a particular use case for such a degraded behaviour,
but since you are not describing it, I'm not at all inclined to
actively break KVM's performance and scalability.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.