Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 03 Aug 2023 00:25:35 +0100
Message-ID: <86h6ph58nk.wl-maz@kernel.org>
From:   Marc Zyngier <maz@kernel.org>
To:     Raghavendra Rao Ananta <rananta@google.com>
Cc:     Oliver Upton <oliver.upton@linux.dev>,
        James Morse <james.morse@arm.com>,
        Suzuki K Poulose <suzuki.poulose@arm.com>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <seanjc@google.com>,
        Huacai Chen <chenhuacai@kernel.org>,
        Zenghui Yu <yuzenghui@huawei.com>,
        Anup Patel <anup@brainfault.org>,
        Atish Patra <atishp@atishpatra.org>,
        Jing Zhang <jingzhangos@google.com>,
        Reiji Watanabe <reijiw@google.com>,
        Colton Lewis <coltonlewis@google.com>,
        David Matlack <dmatlack@google.com>,
        linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev,
        linux-mips@vger.kernel.org, kvm-riscv@lists.infradead.org,
        linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org,
        kvm@vger.kernel.org, Gavin Shan <gshan@redhat.com>,
        Shaoqin Huang <shahuang@redhat.com>
Subject: Re: [PATCH v7 08/12] KVM: arm64: Define kvm_tlb_flush_vmid_range()
In-Reply-To: <CAJHc60xC-Bs-0uoqkytDVcwHTxVnW7eyVxfHhO0mRrupVbdaYg@mail.gmail.com>
References: <20230722022251.3446223-1-rananta@google.com>
        <20230722022251.3446223-9-rananta@google.com>
        <87o7jxr06t.wl-maz@kernel.org>
        <87lef1qzim.wl-maz@kernel.org>
        <CAJHc60xC-Bs-0uoqkytDVcwHTxVnW7eyVxfHhO0mRrupVbdaYg@mail.gmail.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2
 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Mon, 31 Jul 2023 19:01:53 +0100,
Raghavendra Rao Ananta <rananta@google.com> wrote:
>=20
> On Thu, Jul 27, 2023 at 6:01=E2=80=AFAM Marc Zyngier <maz@kernel.org> wro=
te:
> >
> > On Thu, 27 Jul 2023 13:47:06 +0100,
> > Marc Zyngier <maz@kernel.org> wrote:
> > >
> > > On Sat, 22 Jul 2023 03:22:47 +0100,
> > > Raghavendra Rao Ananta <rananta@google.com> wrote:
> > > >
> > > > Implement the helper kvm_tlb_flush_vmid_range() that acts
> > > > as a wrapper for range-based TLB invalidations. For the
> > > > given VMID, use the range-based TLBI instructions to do
> > > > the job or fallback to invalidating all the TLB entries.
> > > >
> > > > Signed-off-by: Raghavendra Rao Ananta <rananta@google.com>
> > > > Reviewed-by: Gavin Shan <gshan@redhat.com>
> > > > Reviewed-by: Shaoqin Huang <shahuang@redhat.com>
> > > > ---
> > > >  arch/arm64/include/asm/kvm_pgtable.h | 10 ++++++++++
> > > >  arch/arm64/kvm/hyp/pgtable.c         | 20 ++++++++++++++++++++
> > > >  2 files changed, 30 insertions(+)
> > > >
> > > > diff --git a/arch/arm64/include/asm/kvm_pgtable.h b/arch/arm64/incl=
ude/asm/kvm_pgtable.h
> > > > index 8294a9a7e566..5e8b1ff07854 100644
> > > > --- a/arch/arm64/include/asm/kvm_pgtable.h
> > > > +++ b/arch/arm64/include/asm/kvm_pgtable.h
> > > > @@ -754,4 +754,14 @@ enum kvm_pgtable_prot kvm_pgtable_stage2_pte_p=
rot(kvm_pte_t pte);
> > > >   *    kvm_pgtable_prot format.
> > > >   */
> > > >  enum kvm_pgtable_prot kvm_pgtable_hyp_pte_prot(kvm_pte_t pte);
> > > > +
> > > > +/**
> > > > + * kvm_tlb_flush_vmid_range() - Invalidate/flush a range of TLB en=
tries
> > > > + *
> > > > + * @mmu:   Stage-2 KVM MMU struct
> > > > + * @addr:  The base Intermediate physical address from which to in=
validate
> > > > + * @size:  Size of the range from the base to invalidate
> > > > + */
> > > > +void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu,
> > > > +                           phys_addr_t addr, size_t size);
> > > >  #endif     /* __ARM64_KVM_PGTABLE_H__ */
> > > > diff --git a/arch/arm64/kvm/hyp/pgtable.c b/arch/arm64/kvm/hyp/pgta=
ble.c
> > > > index aa740a974e02..5d14d5d5819a 100644
> > > > --- a/arch/arm64/kvm/hyp/pgtable.c
> > > > +++ b/arch/arm64/kvm/hyp/pgtable.c
> > > > @@ -670,6 +670,26 @@ static bool stage2_has_fwb(struct kvm_pgtable =
*pgt)
> > > >     return !(pgt->flags & KVM_PGTABLE_S2_NOFWB);
> > > >  }
> > > >
> > > > +void kvm_tlb_flush_vmid_range(struct kvm_s2_mmu *mmu,
> > > > +                           phys_addr_t addr, size_t size)
> > > > +{
> > > > +   unsigned long pages, inval_pages;
> > > > +
> > > > +   if (!system_supports_tlb_range()) {
> > > > +           kvm_call_hyp(__kvm_tlb_flush_vmid, mmu);
> > > > +           return;
> > > > +   }
> > > > +
> > > > +   pages =3D size >> PAGE_SHIFT;
> > > > +   while (pages > 0) {
> > > > +           inval_pages =3D min(pages, MAX_TLBI_RANGE_PAGES);
> > > > +           kvm_call_hyp(__kvm_tlb_flush_vmid_range, mmu, addr, inv=
al_pages);
> > > > +
> > > > +           addr +=3D inval_pages << PAGE_SHIFT;
> > > > +           pages -=3D inval_pages;
> > > > +   }
> > > > +}
> > > > +
> > >
> > > This really shouldn't live in pgtable.c. This code gets linked into
> > > the EL2 object. What do you think happens if, for some reason, this
> > > gets called *from EL2*?
> >
> > Ah, actually, nothing too bad would happen, as we convert the
> > kvm_call_hyp() into a function call.
> >
> > But still, we don't need two copies of this stuff, and it can live in
> > mmu.c.
> >
> But since we have a couple of references in pgtable.c to
> kvm_tlb_flush_vmid_range(), wouldn't that be an (linking) issue if we
> moved the definition to mmu.c?
>=20
> ld: error: undefined symbol: __kvm_nvhe_kvm_tlb_flush_vmid_range
> >>> referenced by pgtable.c:1148 (./arch/arm64/kvm/hyp/nvhe/../pgtable.c:=
1148)
> >>>               arch/arm64/kvm/hyp/nvhe/kvm_nvhe.o:(__kvm_nvhe_kvm_pgta=
ble_stage2_unmap) in archive vmlinux.a
> ...

Ah crap, I missed that. What a mess. Forget it then. It really is a
shame that all the neat separation between mmu.c and pgtable.c that we
were aiming for is ultimately lost.

	M.

--=20
Without deviation from the norm, progress is not possible.