Received-SPF: pass (google.com: domain of linux-kernel+bounces-149950-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Date: Thu, 18 Apr 2024 12:07:35 +0100
Message-ID: <86sezjq688.wl-maz@kernel.org>
From: Marc Zyngier <maz@kernel.org>
To: Catalin Marinas <catalin.marinas@arm.com>
Cc: Naresh Kamboju <naresh.kamboju@linaro.org>,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Mark Brown <broonie@kernel.org>,
	stable@vger.kernel.org,
	patches@lists.linux.dev,
	linux-kernel@vger.kernel.org,
	torvalds@linux-foundation.org,
	akpm@linux-foundation.org,
	linux@roeck-us.net,
	shuah@kernel.org,
	patches@kernelci.org,
	lkft-triage@lists.linaro.org,
	pavel@denx.de,
	jonathanh@nvidia.com,
	f.fainelli@gmail.com,
	sudipm.mukherjee@gmail.com,
	srw@sladewatkins.net,
	rwarsow@gmx.de,
	conor@kernel.org,
	allen.lkml@gmail.com,
	Yihuang Yu <yihyu@redhat.com>,
	Gavin Shan <gshan@redhat.com>,
	Ryan Roberts <ryan.roberts@arm.com>,
	Anshuman Khandual <anshuman.khandual@arm.com>,
	Shaoqin Huang <shahuang@redhat.com>,
	Will Deacon <will@kernel.org>,
	linux-arm-kernel@lists.infradead.org,
	Anders Roxell <anders.roxell@linaro.org>
Subject: Re: [PATCH 6.6 000/122] 6.6.28-rc1 review
In-Reply-To: <Zh61KobDt_y1O46-@arm.com>
References: <20240415141953.365222063@linuxfoundation.org>
	<Zh5UJh31PlBkpZWd@finisterre.sirena.org.uk>
	<CA+G9fYu-AjRm-BBA=8fWS8oCbBJ5W443JHPh3uddD7ea7MY-YA@mail.gmail.com>
	<86y19dqw74.wl-maz@kernel.org>
	<Zh61KobDt_y1O46-@arm.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.2
 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO)
Precedence: bulk
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=US-ASCII

On Tue, 16 Apr 2024 18:28:10 +0100,
Catalin Marinas <catalin.marinas@arm.com> wrote:
> 
> On Tue, Apr 16, 2024 at 02:22:07PM +0100, Marc Zyngier wrote:
> > On Tue, 16 Apr 2024 14:07:30 +0100,
> > Naresh Kamboju <naresh.kamboju@linaro.org> wrote:
> > > On Tue, 16 Apr 2024 at 16:04, Mark Brown <broonie@kernel.org> wrote:
> > > > On Mon, Apr 15, 2024 at 04:19:25PM +0200, Greg Kroah-Hartman wrote:
> > > > > This is the start of the stable review cycle for the 6.6.28 release.
> > > > > There are 122 patches in this series, all will be posted as a response
> > > > > to this one.  If anyone has any issues with these being applied, please
> > > > > let me know.
> > > >
> > > > The bisect of the boot issue that's affecting the FVP in v6.6 (only)
> > > > landed on c9ad150ed8dd988 (arm64: tlb: Fix TLBI RANGE operand),
> > > > e3ba51ab24fdd in mainline, as being the first bad commit - it's also in
> > > > the -rc for v6.8 but that seems fine.  I've done no investigation beyond
> > > > the bisect and looking at the commit log to pull out people to CC and
> > > > note that the fix was explicitly targeted at v6.6.
> > > 
> > > Anders investigated this reported issues and bisected and also found
> > > the missing commit for stable-rc 6.6 is
> > > e2768b798a19 ("arm64/mm: Modify range-based tlbi to decrement scale")
> > 
> > Which is definitely *not* stable candidate. We need to understand why
> > the invalidation goes south when the scale go up instead of down.
> 
> If you backport e3ba51ab24fd ("arm64: tlb: Fix TLBI RANGE operand")
> which fixes 117940aa6e5f ("KVM: arm64: Define
> kvm_tlb_flush_vmid_range()") but without the newer e2768b798a19
> ("arm64/mm: Modify range-based tlbi to decrement scale"), it looks like
> "scale" in __flush_tlb_range_op() goes out of range to 4. Tested on my
> CBMC model, not on the actual kernel. It may be worth adding some
> WARN_ONs in __flush_tlb_range_op() if scale is outside the 0..3 range or
> num greater than 31.
> 
> I haven't investigated properly (and I'm off tomorrow, back on Thu) but
> it's likely the original code was not very friendly to the maximum
> range, never tested. Anyway, if one figures out why it goes out of
> range, I think the solution is to also backport e2768b798a19 to stable.

I looked into this, and I came to the conclusion that this patch is
pretty much incompatible with the increasing scale (even if you cap
num to 30).

The number of pages to invalidate is a 20 bit quantity, a 5 bit slice
per scale. With the 6.6 approach (limit of num=30 and increasing
scale), we invalidate each 5 bit slice independently. After each
scale round, the corresponding slice is guaranteed to be 0.

With the 6.9 method, we invalidate the maximum possible for a given
scale. With a decreasing scale, we converge towards 0 or 1 on each
round.  With an increasing scale, this breaks spectacularly, because
the strong guarantee that the remaining page count is "aligned" to
2^(5*scale+1) is not valid anymore (the low bits may not be 0).

As a result, we don't converge because we never consider these low
bits anymore, the page count doesn't decrease, scale goes past 3, and
everything catches fire.

So despite my earlier comment, it looks like picking e2768b798a19 is
the right thing to do *if* we're taking e3ba51ab24fd into 6.6-stable.

Otherwise, we need a separate fix, which Ryan initially advocating for
initially.

Thanks,

	M.

-- 
Without deviation from the norm, progress is not possible.