Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20230306224127.1689967-1-vipinsh@google.com> <20230306224127.1689967-16-vipinsh@google.com>
 <ZBzS9WnZu9VtSVAt@google.com> <CAHVum0dOP6kR83Uuha=Ka7=enfEdrTTX7ZtR0nhXrkJDCo9dKQ@mail.gmail.com>
In-Reply-To: <CAHVum0dOP6kR83Uuha=Ka7=enfEdrTTX7ZtR0nhXrkJDCo9dKQ@mail.gmail.com>
From:   David Matlack <dmatlack@google.com>
Date:   Tue, 28 Mar 2023 16:24:36 -0700
Message-ID: <CALzav=fgek=jgtaytz6ZEUgBzoHBKAgLk53n9w0LGZrJkMBg-A@mail.gmail.com>
Subject: Re: [Patch v4 15/18] KVM: mmu: Add NUMA node support in struct kvm_mmu_memory_cache{}
To:     Vipin Sharma <vipinsh@google.com>
Cc:     seanjc@google.com, pbonzini@redhat.com, bgardon@google.com,
        jmattson@google.com, mizhang@google.com, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Tue, Mar 28, 2023 at 10:51=E2=80=AFAM Vipin Sharma <vipinsh@google.com> =
wrote:
>
> On Thu, Mar 23, 2023 at 3:30=E2=80=AFPM David Matlack <dmatlack@google.co=
m> wrote:
> >
> > On Mon, Mar 06, 2023 at 02:41:24PM -0800, Vipin Sharma wrote:
> > > Add NUMA node id variable in struct kvm_mmu_memory_cache{}. This
> > > variable denotes preferable NUMA node from which memory will be
> > > allocated under this memory cache.
> > >
> > > Set this variable to NUMA_NO_NODE if there is no preferred node.
> > >
> > > MIPS doesn't do any sort of initializatino of struct
> > > kvm_mmu_memory_cache{}. Keep things similar in MIPS by setting gfp_ze=
ro
> > > to 0 as INIT_KVM_MMU_MEMORY_CACHE() will initialize it to __GFP_ZERO.
> > >
> > > "node" cannot be left as 0, as 0 is a valid NUMA node value.
> > >
> > > Signed-off-by: Vipin Sharma <vipinsh@google.com>
> > > ---
> > >  arch/mips/kvm/mips.c      | 3 +++
> > >  include/linux/kvm_types.h | 3 +++
> > >  2 files changed, 6 insertions(+)
> > >
> > > diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
> > > index 36c8991b5d39..5ec5ce919918 100644
> > > --- a/arch/mips/kvm/mips.c
> > > +++ b/arch/mips/kvm/mips.c
> > > @@ -294,6 +294,9 @@ int kvm_arch_vcpu_create(struct kvm_vcpu *vcpu)
> > >                    HRTIMER_MODE_REL);
> > >       vcpu->arch.comparecount_timer.function =3D kvm_mips_comparecoun=
t_wakeup;
> > >
> > > +     INIT_KVM_MMU_MEMORY_CACHE(&vcpu->arch.mmu_page_cache);
> > > +     vcpu->arch.mmu_page_cache.gfp_zero =3D 0;
> >
> > Oh MIPS is here. Why isn't MIPS covered in the previous commits?
>
> Because this is the patch where MIPS get impacted. MIPS doesn't
> initialize gfp_zero, so there was no need to change the code in MIPS.
> However, with the addition of "node" in kvm_mmu_memory_cache{} in this
> patch, we need initialization in MIPS to (1) Set  node to NUMA_NO_NODE
> as 0 is now a valid value, and (2) INIT_KVM_MMU_MEMORY_CACHE() will
> set gfp_zero to __GFP_ZERO which is different than existing code in
> MIPS to keep it 0.
>
> I asked MIPS maintainers in the previous version to see if GFP_ZERO
> can be added but didn't get any response.
> https://lore.kernel.org/lkml/CAHVum0c+17Z-RbGAFdU-xmRejDjDQ+MKOfN4XaObh2S=
wgWAjLg@mail.gmail.com/

I see. IMO it's more logical to convert the MIPS cache to
INIT_KVM_MMU_MEMORY_CACHE() in patch 13, along with all the other
users of struct kvm_mmu_memory_cache. Then in patch 14, add the line
to set gfp_zero to 0 for MIPS to preserve the existing behavior. That
produces a very simple chain of changes:

Patch 13: Convert all users of struct kvm_mmu_memory_cache to INIT()
Patch 14: Invert the default value of kvm_mmu_memory_cache.gfp_zero
Patch 15: Add node to kvm_mmu_memory_cache


>
> > > +
> > >       /*
> > >        * Allocate space for host mode exception handlers that handle
> > >        * guest mode exits
> > > diff --git a/include/linux/kvm_types.h b/include/linux/kvm_types.h
> > > index 5da7953532ce..b2a405c8e629 100644
> > > --- a/include/linux/kvm_types.h
> > > +++ b/include/linux/kvm_types.h
> > > @@ -97,10 +97,13 @@ struct kvm_mmu_memory_cache {
> > >       struct kmem_cache *kmem_cache;
> > >       int capacity;
> > >       void **objects;
> > > +     /* Preferred NUMA node of memory allocation. */
> > > +     int node;
> > >  };
> > >
> > >  #define KVM_MMU_MEMORY_CACHE_INIT() {        \
> > >       .gfp_zero =3D __GFP_ZERO,         \
> > > +     .node =3D NUMA_NO_NODE,           \
> > >  }
> > >
> > >  #define KVM_MMU_MEMORY_CACHE(_name) \
> > > --
> > > 2.40.0.rc0.216.gc4246ad0f0-goog
> > >