2020-06-22 19:23:02

by Sean Christopherson

Subject: [PATCH] KVM: x86/mmu: Don't put invalid SPs back on the list of active pages

Delete a shadow page from the invalidation list instead of throwing it
back on the list of active pages when it's a root shadow page with
active users. Invalid active root pages will be explicitly freed by
mmu_free_root_page() when the root_count hits zero, i.e. they don't need
to be put on the active list to avoid leakage.

Use sp->role.invalid to detect that a shadow page has already been
zapped, i.e. is not on a list.

WARN if an invalid page is encountered when zapping pages, as it should
now be impossible.

Signed-off-by: Sean Christopherson <[email protected]>
---
arch/x86/kvm/mmu/mmu.c | 18 ++++++++++--------
1 file changed, 10 insertions(+), 8 deletions(-)

diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
index fdd05c233308..fa5bd3f987dd 100644
--- a/arch/x86/kvm/mmu/mmu.c
+++ b/arch/x86/kvm/mmu/mmu.c
@@ -2757,10 +2757,13 @@ static bool __kvm_mmu_prepare_zap_page(struct kvm *kvm,
if (!sp->root_count) {
/* Count self */
(*nr_zapped)++;
- list_move(&sp->link, invalid_list);
+ if (sp->role.invalid)
+ list_add(&sp->link, invalid_list);
+ else
+ list_move(&sp->link, invalid_list);
kvm_mod_used_mmu_pages(kvm, -1);
} else {
- list_move(&sp->link, &kvm->arch.active_mmu_pages);
+ list_del(&sp->link);

/*
* Obsolete pages cannot be used on any vCPUs, see the comment
@@ -5732,12 +5735,11 @@ static void kvm_zap_obsolete_pages(struct kvm *kvm)
break;

/*
- * Skip invalid pages with a non-zero root count, zapping pages
- * with a non-zero root count will never succeed, i.e. the page
- * will get thrown back on active_mmu_pages and we'll get stuck
- * in an infinite loop.
+ * Invalid pages should never land back on the list of active
+ * pages. Skip the bogus page, otherwise we'll get stuck in an
+ * infinite loop if the page gets put back on the list (again).
*/
- if (sp->role.invalid && sp->root_count)
+ if (WARN_ON(sp->role.invalid))
continue;

/*
@@ -6015,7 +6017,7 @@ void kvm_mmu_zap_all(struct kvm *kvm)
spin_lock(&kvm->mmu_lock);
restart:
list_for_each_entry_safe(sp, node, &kvm->arch.active_mmu_pages, link) {
- if (sp->role.invalid && sp->root_count)
+ if (WARN_ON(sp->role.invalid))
continue;
if (__kvm_mmu_prepare_zap_page(kvm, sp, &invalid_list, &ign))
goto restart;
--
2.26.0

2020-06-23 00:28:30

by Paolo Bonzini

Subject: Re: [PATCH] KVM: x86/mmu: Don't put invalid SPs back on the list of active pages

On 22/06/20 21:18, Sean Christopherson wrote:
> diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> index fdd05c233308..fa5bd3f987dd 100644
> --- a/arch/x86/kvm/mmu/mmu.c
> +++ b/arch/x86/kvm/mmu/mmu.c
> @@ -2757,10 +2757,13 @@ static bool __kvm_mmu_prepare_zap_page(struct kvm *kvm,
> if (!sp->root_count) {
> /* Count self */
> (*nr_zapped)++;
> - list_move(&sp->link, invalid_list);
> + if (sp->role.invalid)
> + list_add(&sp->link, invalid_list);
> + else
> + list_move(&sp->link, invalid_list);

It's late here, but I think this part needs a comment anyway...

Paolo

2020-06-23 01:19:18

by Sean Christopherson

Subject: Re: [PATCH] KVM: x86/mmu: Don't put invalid SPs back on the list of active pages

On Tue, Jun 23, 2020 at 02:23:53AM +0200, Paolo Bonzini wrote:
> On 22/06/20 21:18, Sean Christopherson wrote:
> > diff --git a/arch/x86/kvm/mmu/mmu.c b/arch/x86/kvm/mmu/mmu.c
> > index fdd05c233308..fa5bd3f987dd 100644
> > --- a/arch/x86/kvm/mmu/mmu.c
> > +++ b/arch/x86/kvm/mmu/mmu.c
> > @@ -2757,10 +2757,13 @@ static bool __kvm_mmu_prepare_zap_page(struct kvm *kvm,
> > if (!sp->root_count) {
> > /* Count self */
> > (*nr_zapped)++;
> > - list_move(&sp->link, invalid_list);
> > + if (sp->role.invalid)
> > + list_add(&sp->link, invalid_list);
> > + else
> > + list_move(&sp->link, invalid_list);
>
> It's late here, but I think this part needs a comment anyway...

No argument here. I'll spin a v2, I just realized there is a separate
optimization that can build on this patch. I was planning on sending it
separately, but I misread the loop in make_mmu_pages_available().