Date:   Tue, 21 Mar 2023 13:40:21 -0700
In-Reply-To: <ZBoMIJipRtmvsNXg@google.com>
Mime-Version: 1.0
References: <20230211014626.3659152-1-vipinsh@google.com> <ZBTwX5790zwl5721@google.com>
 <ZBj9L2VUjEbWbgcS@google.com> <CAHVum0feM8hnD-+dXF4jiug8tmpm9GBAh619Xf279LNSm=Jozw@mail.gmail.com>
 <ZBoMIJipRtmvsNXg@google.com>
Message-ID: <ZBoWNdwGho5bZ+Kz@google.com>
Subject: Re: [Patch v3 0/7] Optimize clear dirty log
From:   Sean Christopherson <seanjc@google.com>
To:     Vipin Sharma <vipinsh@google.com>
Cc:     pbonzini@redhat.com, bgardon@google.com, dmatlack@google.com,
        kvm@vger.kernel.org, linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Tue, Mar 21, 2023, Sean Christopherson wrote:
> On Tue, Mar 21, 2023, Vipin Sharma wrote:
> > On Mon, Mar 20, 2023 at 5:41=E2=80=AFPM Sean Christopherson <seanjc@goo=
gle.com> wrote:
> > >
> > > On Fri, Mar 17, 2023, Sean Christopherson wrote:
> > > > Did a cursory glance, looks good.  I'll do a more thorough pass nex=
t week and get
> > > > it queued up if all goes well.  No need for a v4 at this point, I'l=
l fixup David's
> > > > various nits when applying.
> > >
> > > Ooof, that ended up being painful.  In hindsight, I should have asked=
 for a v4,
> > > but damage done, and it's my fault for throwing you a big blob of cod=
e in the
> > > first place.
> > >
> > > I ended up splitting the "interesting" patches into three each:
> > >
> > >   1. Switch to the atomic-AND
> > >   2. Drop the access-tracking / dirty-logging (as appropriate)
> > >   3. Drop the call to __handle_changed_spte()
> > >
> > > because logically they are three different things (although obviously=
 related).
> > >
> > > I have pushed the result to kvm-x86/mmu, but haven't merged to kvm-x8=
6/next or
> > > sent thanks because it's not yet tested.  I'll do testing tomorrow, b=
ut if you
> > > can take a look in the meantime to make sure I didn't do something co=
mpletely
> > > boneheaded, it'd be much appreciated.
> >=20
> >=20
> > Thanks for refactoring the patches. I reviewed the commits, no obvious
> > red flags from my side. Few small nits I found:
> >=20
> > commit e534a94eac07 ("KVM: x86/mmu: Use kvm_ad_enabled() to determine
> > if TDP MMU SPTEs need wrprot")
> >  - kvm_ad_enabled() should be outside the loop.
>=20
> Hmm, I deliberately left it inside the loop, but I agree that it would be=
 better
> to hoist it out in that commit.
>=20
> > commit 69032b5d71ef (" KVM: x86/mmu: Atomically clear SPTE dirty state
> > in the clear-dirty-log flow")
> >  - MMU_WARN_ON(kvm_ad_enabled() &&
> > spte_ad_need_write_protect(iter.old_spte) should be after
> > if(iter.level > PG_LEVEL_4k...)
>=20
> Ah, hrm.  This was also deliberate, but looking at the diff I agree that =
relative
> to the diff, it's an unnecessary/unrelated change.  I think what I'll do =
is
> land the assertion above the "if (iter.level > PG_LEVEL_4K ||" in the abo=
ve
> commit that switches to kvm_ad_enabled().  That way there shouldn't be an=
y change
> for the assertion in this commit.

Aha!  Even better, split this into yet one more patch to dedup the guts bef=
ore
switching to the atomic-AND, and give clear_dirty_gfn_range() the same trea=
tment.
That further isolates the changes, provides solid justification for hoistin=
g the
kvm_ad_enabled() check out of the loop (it's basically guaranteed to be a s=
ingle
memory read that hits the L1), and keeps clear_dirty_gfn_range() and
clear_dirty_pt_masked() as similar as is reasonably possible.

Speaking of which, I'll send a patch to remove the redundant is_shadow_pres=
ent_pte()
check in clear_dirty_gfn_range(), that's already handled by tdp_root_for_ea=
ch_leaf_pte().