2020-01-25 17:42:12

by Guenter Roeck

[permalink] [raw]
Subject: Re: [PATCH] ARM: 8936/1: decompressor: avoid CP15 barrier instructions in v7 cache setup code

Hi,

On Fri, Nov 08, 2019 at 01:44:32PM +0100, Ard Biesheuvel wrote:
> Commit e17b1af96b2afc38e684aa2f1033387e2ed10029
>
> "ARM: 8857/1: efi: enable CP15 DMB instructions before cleaning the cache"
>
> added some explicit handling of the CP15BEN bit in the SCTLR system
> register, to ensure that CP15 barrier instructions are enabled, even
> if we enter the decompressor via the EFI stub.
>
> However, as it turns out, there are other ways in which we may end up
> using CP15 barrier instructions without them being enabled. I.e., when
> the decompressor startup code skips the cache_on() initially, we end
> up calling cache_clean_flush() with the caches and MMU off, in which
> case the CP15BEN bit in SCTLR may not be programmed either. And in
> fact, cache_on() itself issues CP15 barrier instructions before actually
> enabling them by programming the new SCTLR value (and issuing an ISB)
>
> Since all these routines are specific to v7, let's clean this up by
> using the ordinary v7 barrier instructions in the v7 specific cache
> handling routines, so that we never rely on the CP15 ones. This also
> avoids the issue where a barrier is required between programming SCTLR
> and using the CP15 barrier instructions, which would result in two
> different kinds of barriers being used in the same function.
>
> Acked-by: Marc Zyngier <[email protected]>
> Signed-off-by: Ard Biesheuvel <[email protected]>
> Signed-off-by: Russell King <[email protected]>

This patch causes all qemu emulations for ARM1176 to fail hard (stall with
no console output even with earlycon enabled). This affects witherspoon-bmc,
ast2500-evb, romulus-bmc, and swift-bmc. It does not affect emulations
for other CPU types, even with the same kernel configuration (such as
ast2600-evb).

Reverting the patch fixes the problem.

Guenter

---
bisect log:

# bad: [702ccea170f07783bd002055a353a0866c062267] Add linux-next specific files for 20200124
# good: [def9d2780727cec3313ed3522d0123158d87224d] Linux 5.5-rc7
git bisect start 'HEAD' 'v5.5-rc7'
# bad: [70b03961bd9b391bd124731df1f92d3e6f1b91c5] Merge remote-tracking branch 'crypto/master'
git bisect bad 70b03961bd9b391bd124731df1f92d3e6f1b91c5
# bad: [fd2c516f2b10a23d2a86948b3fad0961038ce708] Merge remote-tracking branch 'v9fs/9p-next'
git bisect bad fd2c516f2b10a23d2a86948b3fad0961038ce708
# bad: [4c1bb3d9902fbbe24efea7bae47337e51f8f112a] Merge remote-tracking branch 'arm-soc/for-next'
git bisect bad 4c1bb3d9902fbbe24efea7bae47337e51f8f112a
# good: [9b0b308a1586f620a49c50605ae8abf509190661] Merge tag 'zynq-dt-for-v5.6-v2' of https://github.com/Xilinx/linux-xlnx into arm/dt
git bisect good 9b0b308a1586f620a49c50605ae8abf509190661
# bad: [fa534dfeb4e761e31ba833c6315e350e9ff4ba5f] Merge remote-tracking branch 'arm/for-next'
git bisect bad fa534dfeb4e761e31ba833c6315e350e9ff4ba5f
# good: [c6e2f4e0e8b40ac5a6c6084189ea3429c71b0b4c] Merge remote-tracking branch 'ide/master'
git bisect good c6e2f4e0e8b40ac5a6c6084189ea3429c71b0b4c
# good: [eecc70d22ae51225de1ef629c1159f7116476b2e] media: digitv: don't continue if remote control state can't be read
git bisect good eecc70d22ae51225de1ef629c1159f7116476b2e
# good: [2e30f8d480842313c3de748bedd0a469f131f12e] Merge remote-tracking branch 'pinctrl-intel-fixes/fixes'
git bisect good 2e30f8d480842313c3de748bedd0a469f131f12e
# good: [9c9aa8fdf306cd7329e0a68bbcbe2f71b397dac1] kbuild: remove 'Building modules, stage 2.' log
git bisect good 9c9aa8fdf306cd7329e0a68bbcbe2f71b397dac1
# good: [3460d0bc256a50b71dbdae8227c600761c502022] kconfig: distinguish between dependencies and visibility in help text
git bisect good 3460d0bc256a50b71dbdae8227c600761c502022
# bad: [290b9fa737f893f96602c158f22caacd942e1268] ARM: 8951/1: Fix Kexec compilation issue.
git bisect bad 290b9fa737f893f96602c158f22caacd942e1268
# bad: [44832877ea4c186460f42e48b1da98a08a29ffb5] ARM: 8942/1: Revert "8857/1: efi: enable CP15 DMB instructions before cleaning the cache"
git bisect bad 44832877ea4c186460f42e48b1da98a08a29ffb5
# bad: [dff03da7c2b0575b8112c2947f84521aad287ae7] ARM: 8941/1: ARM/decompressor: enable CP15 barrier instructions in v7 cache setup code
git bisect bad dff03da7c2b0575b8112c2947f84521aad287ae7
# bad: [bcb704e551fc4e5034b9fa319a3ef49d30cb84d9] ARM: 8936/1: decompressor: avoid CP15 barrier instructions in v7 cache setup code
git bisect bad bcb704e551fc4e5034b9fa319a3ef49d30cb84d9
# first bad commit: [bcb704e551fc4e5034b9fa319a3ef49d30cb84d9] ARM: 8936/1: decompressor: avoid CP15 barrier instructions in v7 cache setup code


2020-01-25 18:08:32

by Russell King (Oracle)

[permalink] [raw]
Subject: Re: [PATCH] ARM: 8936/1: decompressor: avoid CP15 barrier instructions in v7 cache setup code

On Sat, Jan 25, 2020 at 09:39:50AM -0800, Guenter Roeck wrote:
> Hi,
>
> On Fri, Nov 08, 2019 at 01:44:32PM +0100, Ard Biesheuvel wrote:
> > Commit e17b1af96b2afc38e684aa2f1033387e2ed10029
> >
> > "ARM: 8857/1: efi: enable CP15 DMB instructions before cleaning the cache"
> >
> > added some explicit handling of the CP15BEN bit in the SCTLR system
> > register, to ensure that CP15 barrier instructions are enabled, even
> > if we enter the decompressor via the EFI stub.
> >
> > However, as it turns out, there are other ways in which we may end up
> > using CP15 barrier instructions without them being enabled. I.e., when
> > the decompressor startup code skips the cache_on() initially, we end
> > up calling cache_clean_flush() with the caches and MMU off, in which
> > case the CP15BEN bit in SCTLR may not be programmed either. And in
> > fact, cache_on() itself issues CP15 barrier instructions before actually
> > enabling them by programming the new SCTLR value (and issuing an ISB)
> >
> > Since all these routines are specific to v7, let's clean this up by
> > using the ordinary v7 barrier instructions in the v7 specific cache
> > handling routines, so that we never rely on the CP15 ones. This also
> > avoids the issue where a barrier is required between programming SCTLR
> > and using the CP15 barrier instructions, which would result in two
> > different kinds of barriers being used in the same function.
> >
> > Acked-by: Marc Zyngier <[email protected]>
> > Signed-off-by: Ard Biesheuvel <[email protected]>
> > Signed-off-by: Russell King <[email protected]>
>
> This patch causes all qemu emulations for ARM1176 to fail hard (stall with
> no console output even with earlycon enabled). This affects witherspoon-bmc,
> ast2500-evb, romulus-bmc, and swift-bmc. It does not affect emulations
> for other CPU types, even with the same kernel configuration (such as
> ast2600-evb).

Hmm, looks like we're going to have to drop 8936/1, 8941/1 and 8942/1
in that case.

--
RMK's Patch system: https://www.armlinux.org.uk/developer/patches/
FTTC broadband for 0.8mile line in suburbia: sync at 12.1Mbps down 622kbps up
According to speedtest.net: 11.9Mbps down 500kbps up

2020-01-25 18:13:23

by Ard Biesheuvel

[permalink] [raw]
Subject: Re: [PATCH] ARM: 8936/1: decompressor: avoid CP15 barrier instructions in v7 cache setup code

On Sat, 25 Jan 2020 at 19:06, Russell King - ARM Linux admin
<[email protected]> wrote:
>
> On Sat, Jan 25, 2020 at 09:39:50AM -0800, Guenter Roeck wrote:
> > Hi,
> >
> > On Fri, Nov 08, 2019 at 01:44:32PM +0100, Ard Biesheuvel wrote:
> > > Commit e17b1af96b2afc38e684aa2f1033387e2ed10029
> > >
> > > "ARM: 8857/1: efi: enable CP15 DMB instructions before cleaning the cache"
> > >
> > > added some explicit handling of the CP15BEN bit in the SCTLR system
> > > register, to ensure that CP15 barrier instructions are enabled, even
> > > if we enter the decompressor via the EFI stub.
> > >
> > > However, as it turns out, there are other ways in which we may end up
> > > using CP15 barrier instructions without them being enabled. I.e., when
> > > the decompressor startup code skips the cache_on() initially, we end
> > > up calling cache_clean_flush() with the caches and MMU off, in which
> > > case the CP15BEN bit in SCTLR may not be programmed either. And in
> > > fact, cache_on() itself issues CP15 barrier instructions before actually
> > > enabling them by programming the new SCTLR value (and issuing an ISB)
> > >
> > > Since all these routines are specific to v7, let's clean this up by
> > > using the ordinary v7 barrier instructions in the v7 specific cache
> > > handling routines, so that we never rely on the CP15 ones. This also
> > > avoids the issue where a barrier is required between programming SCTLR
> > > and using the CP15 barrier instructions, which would result in two
> > > different kinds of barriers being used in the same function.
> > >
> > > Acked-by: Marc Zyngier <[email protected]>
> > > Signed-off-by: Ard Biesheuvel <[email protected]>
> > > Signed-off-by: Russell King <[email protected]>
> >
> > This patch causes all qemu emulations for ARM1176 to fail hard (stall with
> > no console output even with earlycon enabled). This affects witherspoon-bmc,
> > ast2500-evb, romulus-bmc, and swift-bmc. It does not affect emulations
> > for other CPU types, even with the same kernel configuration (such as
> > ast2600-evb).
>
> Hmm, looks like we're going to have to drop 8936/1, 8941/1 and 8942/1
> in that case.
>

8941 was intended as an alternative approach to 8936, as the latter is
flawed, given that the v7 cache maintenance routines are shared with
CPUID capable non-v7 CPUs such as the ARM1176. So it was never the
intention for both to be applied.

It should be sufficient to revert 8936. Apologies for the confusion.