Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <ed71d0967113a35f670a9625a058b8e6e0b2f104.1583547991.git.luto@kernel.org>
 <CALCETrVmsF9JSMLSd44-3GGWEz6siJQxudeaYiVnvv__YDT1BQ@mail.gmail.com> <87ftek9ngq.fsf@nanos.tec.linutronix.de>
In-Reply-To: <87ftek9ngq.fsf@nanos.tec.linutronix.de>
From:   Andy Lutomirski <luto@kernel.org>
Date:   Sat, 7 Mar 2020 07:59:22 -0800
Message-ID: <CALCETrVsc-t=tDRPbCg5dWHDY0NFv2zjz12ahD-vnGPn8T+RXA@mail.gmail.com>
Subject: Re: [PATCH v2] x86/kvm: Disable KVM_ASYNC_PF_SEND_ALWAYS
To:     Thomas Gleixner <tglx@linutronix.de>
Cc:     Andy Lutomirski <luto@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>, X86 ML <x86@kernel.org>,
        kvm list <kvm@vger.kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        stable <stable@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Sat, Mar 7, 2020 at 7:47 AM Thomas Gleixner <tglx@linutronix.de> wrote:
>
> Andy Lutomirski <luto@kernel.org> writes:
> > On Fri, Mar 6, 2020 at 6:26 PM Andy Lutomirski <luto@kernel.org> wrote:
> >> +               /*
> >> +                * We do not set KVM_ASYNC_PF_SEND_ALWAYS.  With the current
> >> +                * KVM paravirt ABI, the following scenario is possible:
> >> +                *
> >> +                * #PF: async page fault (KVM_PV_REASON_PAGE_NOT_PRESENT)
> >> +                *  NMI before CR2 or KVM_PF_REASON_PAGE_NOT_PRESENT
> >> +                *   NMI accesses user memory, e.g. due to perf
> >> +                *    #PF: normal page fault
> >> +                *     #PF reads CR2 and apf_reason -- apf_reason should be 0
> >> +                *
> >> +                *  outer #PF reads CR2 and apf_reason -- apf_reason should be
> >> +                *  KVM_PV_REASON_PAGE_NOT_PRESENT
> >> +                *
> >> +                * There is no possible way that both reads of CR2 and
> >> +                * apf_reason get the correct values.  Fixing this would
> >> +                * require paravirt ABI changes.
> >> +                */
> >> +
> >
> > Upon re-reading my own comment, I think the problem is real, but I
> > don't think my patch fixes it.  The outer #PF could just as easily
> > have come from user mode.  We may actually need the NMI code (and
> > perhaps MCE and maybe #DB too) to save, clear, and restore apf_reason.
> > If we do this, then maybe CPL0 async PFs are actually okay, but the
> > semantics are so poorly defined that I'm not very confident about
> > that.
>
> I think even with the current mode this is fixable on the host side when
> it keeps track of the state.
>
> The host knows exactly when it injects a async PF and it can store CR2
> and reason of that async PF in flight.
>
> On the next VMEXIT it checks whether apf_reason is 0. If apf_reason is 0
> then it knows that the guest has read CR2 and apf_reason. All good
> nothing to worry about.
>
> If not it needs to be careful.
>
> As long as the apf_reason of the last async #PF is not cleared by the
> guest no new async #PF can be injected. That's already correct because
> in that case IF==0 which prevents a nested async #PF.
>
> If MCE, NMI trigger a real pagefault then the #PF injection needs to
> clear apf_reason and set the correct CR2. When that #PF returns then the
> old CR2 and apf_reason need to be restored.

How is the host supposed to know when the #PF returns?  Intercepting
IRET sounds like a bad idea and, in any case, is not actually a
reliable indication that #PF returned.