Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <cbd4709bb499874c60986083489e17c93b48d003.camel@redhat.com>
Subject: Re: [PATCH] KVM: x86/mmu: Do not create SPTEs for GFNs that exceed
 host.MAXPHYADDR
From:   Maxim Levitsky <mlevitsk@redhat.com>
To:     Sean Christopherson <seanjc@google.com>
Cc:     Paolo Bonzini <pbonzini@redhat.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Jim Mattson <jmattson@google.com>,
        Joerg Roedel <joro@8bytes.org>, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, Ben Gardon <bgardon@google.com>,
        David Matlack <dmatlack@google.com>
Date:   Tue, 03 May 2022 18:12:10 +0300
In-Reply-To: <e11c21e99e7c4ac758b4417e0ae66d3a2f1fe663.camel@redhat.com>
References: <337332ca-835c-087c-c99b-92c35ea8dcd3@redhat.com>
         <Ymv1I5ixX1+k8Nst@google.com>
         <20e1e7b1-ece7-e9e7-9085-999f7a916ac2@redhat.com>
         <Ymv5TR76RNvFBQhz@google.com>
         <e5864cb4-cce8-bd32-04b0-ecb60c058d0b@redhat.com>
         <YmwL87h6klEC4UKV@google.com>
         <ac2001e66957edc8a3af2413b78478c15898f86c.camel@redhat.com>
         <f3ffad3aa8476156f369ff1d4c33f3e127b47d0c.camel@redhat.com>
         <82d1a5364f1cc479da3762b046d22f136db167e3.camel@redhat.com>
         <af15fd31f73e8a956da50db6104e690f9d308dad.camel@redhat.com>
         <YnAMKtfAeoydHr3x@google.com>
         <e11c21e99e7c4ac758b4417e0ae66d3a2f1fe663.camel@redhat.com>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.36.5 (3.36.5-2.fc32) 
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Precedence: bulk

On Tue, 2022-05-03 at 12:12 +0300, Maxim Levitsky wrote:
> On Mon, 2022-05-02 at 16:51 +0000, Sean Christopherson wrote:
> > On Mon, May 02, 2022, Maxim Levitsky wrote:
> > > On Mon, 2022-05-02 at 10:59 +0300, Maxim Levitsky wrote:
> > > > > > Also I can reproduce it all the way to 5.14 kernel (last kernel I have installed in this VM).
> > > > > > 
> > > > > > I tested kvm/queue as of today, sadly I still see the warning.
> > > > > 
> > > > > Due to a race, the above statements are out of order ;-)
> > > > 
> > > > So futher investigation shows that the trigger for this *is* cpu_pm=on :(
> > > > 
> > > > So this is enough to trigger the warning when run in the guest:
> > > > 
> > > > qemu-system-x86_64  -nodefaults  -vnc none -serial stdio -machine accel=kvm
> > > > -kernel x86/dummy.flat -machine kernel-irqchip=on -smp 8 -m 1g -cpu host
> > > > -overcommit cpu-pm=on
> > > > 
> > > > 
> > > > '-smp 8' is needed, and the more vCPUs the more often the warning appears.
> > > > 
> > > > 
> > > > Due to non atomic memslot update bug, I use patched qemu version, with an
> > > > attached hack, to pause/resume vcpus around the memslot update it does, but
> > > > even without this hack, you can just ctrl+c the test after it gets the KVM
> > > > internal error, and then tdp mmu memory leak warning shows up (not always
> > > > but very often).
> > > > 
> > > > 
> > > > Oh, and if I run the above command on the bare metal, it  never terminates.
> > > > Must be due to preemption, qemu shows beeing stuck in kvm_vcpu_block. AVIC
> > > > disabled, kvm/queue.  Bugs, bugs, and features :)
> > > 
> > > All right, at least that was because I removed the '-device isa-debug-exit,iobase=0xf4,iosize=0x4',
> > > which is apparently used by KVM unit tests to signal exit from the VM.
> > 
> > Can you provide your QEMU command line for running your L1 VM?  And your L0 and L1
> > Kconfigs too?  I've tried both the dummy and ipi_stress tests on a variety of hardware,
> > kernels, QEMUs, etc..., with no luck.
> > 
> 
> So now both L0 and L1 run almost pure kvm/queue)
> (commit 2764011106d0436cb44702cfb0981339d68c3509)
> 
> I have some local patches but they are not relevant to KVM at all, more
> like various tweaks to sensors, a sad hack for yet another regression
> in AMDGPU, etc.
> 
> The config and qemu command line attached.
> 
> AVIC disabled in L0, L0 qemu is from master upstream.
> Bug reproduces too well IMHO, almost always.
> 
> For reference the warning is printed in L1's dmesg.

Tested this without any preemption in L0 and L1 - bug still reproduces just fine.
(kvm/queue)

Best regards,
	Maxim Levitsky

> 
> 
> 
> Best regards,
> 	Maxim Levitsky