Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Fri, 13 Aug 2021 16:13:22 +0000
From:   Sean Christopherson <seanjc@google.com>
To:     Paolo Bonzini <pbonzini@redhat.com>
Cc:     Ben Gardon <bgardon@google.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Jim Mattson <jmattson@google.com>,
        Joerg Roedel <joro@8bytes.org>, kvm <kvm@vger.kernel.org>,
        LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH 2/2] KVM: x86/mmu: Don't step down in the TDP iterator
 when zapping all SPTEs
Message-ID: <YRaaIi9Go38E3mUh@google.com>
References: <20210812050717.3176478-1-seanjc@google.com>
 <20210812050717.3176478-3-seanjc@google.com>
 <CANgfPd8HSYZbqmi21XQ=XeMCndXJ0+Ld0eZNKPWLa1fKtutiBA@mail.gmail.com>
 <YRVVWC31fuZiw9tT@google.com>
 <928be04d-e60e-924c-1f3a-cb5fef8b0042@redhat.com>
 <YRVbamoQhvPmrEgK@google.com>
 <7a95b2f6-a7ad-5101-baa5-6a19194695a3@redhat.com>
 <YRVebIjxEv87I55b@google.com>
 <b08a7751-20c3-26fc-522e-c4cf274d9a6c@redhat.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <b08a7751-20c3-26fc-522e-c4cf274d9a6c@redhat.com>
Precedence: bulk

On Fri, Aug 13, 2021, Paolo Bonzini wrote:
> On 12/08/21 19:46, Sean Christopherson wrote:
> > > > 	if (iter->level == iter->min_level)
> > > > 		return false;
> > > > 
> > > > 	/*
> > > > 	 * Reread the SPTE before stepping down to avoid traversing into page
> > > > 	 * tables that are no longer linked from this entry.
> > > > 	 */
> > > > 	iter->old_spte = READ_ONCE(*rcu_dereference(iter->sptep));  \
> > > >                                                                        ---> this is the code that is avoided
> > > > 	child_pt = spte_to_child_pt(iter->old_spte, iter->level);   /
> > > > 	if (!child_pt)
> > > > 		return false;
> > > Ah, right - so I agree with Ben that it's not too important.
> > Ya.  There is a measurable performance improvement, but it's really only
> > meaningful when there aren't many SPTEs to zap, otherwise the cost of zapping
> > completely dominates the time.
> 
> I don't understand.  When try_step_down is called by tdp_iter_next, all it
> does is really just the READ_ONCE, because spte_to_child_pt will see a
> non-present PTE and return immediately.  Why do two, presumably cache hot,
> reads cause a measurable performance improvement?

It's entirely possible my measurements were bad and/or noisy.  Ah, and my kernel
was running with CONFIG_PROVE_RCU=y, which makes the rcu_dereference() quite a bit
more expensive.