Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Received-SPF: Fail (protection.outlook.com: domain of arri.de does not
 designate 217.111.95.7 as permitted sender) receiver=protection.outlook.com;
 client-ip=217.111.95.7; helo=mta.arri.de;
From:   Christian Eggers <ceggers@arri.de>
To:     Russell King <linux@armlinux.org.uk>,
        <linux-arm-kernel@lists.infradead.org>
CC:     Catalin Marinas <catalin.marinas@arm.com>,
        <linux-arm-kernel@lists.infradead.org>,
        <linux-kernel@vger.kernel.org>, <llvm@lists.linux.dev>,
        Nathan Chancellor <nathan@kernel.org>
Subject: Re: CONFIG_THUMB2_KERNEL=y boot failure after Spectre BHB fixes
Date:   Tue, 22 Mar 2022 18:49:17 +0100
Message-ID: <2109354.irdbgypaU6@localhost.localdomain>
Organization: Arnold & Richter Cine Technik GmbH & Co. Betriebs KG
In-Reply-To: <YipOoAaBIHjeCKOq@dev-arch.thelio-3990X>
References: <YipOoAaBIHjeCKOq@dev-arch.thelio-3990X>
MIME-Version: 1.0
Content-Transfer-Encoding: 7Bit
Content-Type: text/plain; charset="us-ascii"
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Mar 2022 17:49:18.8096
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: daec52f3-af3a-47b1-b7a5-08da0c2c4a2b
X-MS-Exchange-CrossTenant-Id: e6a73a5a-614d-4c51-b3e3-53b660a9433a
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=e6a73a5a-614d-4c51-b3e3-53b660a9433a;Ip=[217.111.95.7];Helo=[mta.arri.de]
X-MS-Exchange-CrossTenant-AuthSource: HE1EUR02FT060.eop-EUR02.prod.protection.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0702MB3582
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Nathan, hi Russel,

I stumbled today over the same problem (no output on serial console
with v5.15.28-rt36). During `git bisect`, I had also some commits
where a few lines of output were visible.

At commit 8d9d651ff227 ("ARM: use LOADADDR() to get load address of
sections"), the system boots up to here:

start_kernel()
+--setup_arch()
   +--paging_init()
      +--devicemaps_init()
         +--eary_trap_init(vectors_base = 0xC7FFE000)
            +--copy_from_lma(vectors_base = 0xC7FFE000, __vectors_start=0x0, __vectors_end=0x20)
               +--__memcpy()

               copy_template.S:113 
               ldr8w   r1, r3, r4, r5, r6, r7, r8, ip, lr, abort=20f
               r1 = 0
              

With the final v5.15.28-rt36 I found out that the system boots fine
after disabling CONFIG_HARDEN_BRANCH_HISTORY.

Is there anything else I could analyze? My SoC system is a NXP i.MX6LL
with a ARMv7 core. I have access to a JTAG debugger.

regards
Christian


On Thursday, 10 March 2022, 20:16:48 CET, Nathan Chancellor wrote:
> Hi Russell,
> 
> Apologies if this has already been reported, I did not see anything when
> sifting through lore.kernel.org and I can still reproduce this with
> current mainline (1db333d9a51f).
> 
> I noticed a QEMU boot failure with multi_v7_defconfig with
> CONFIG_THUMB2_KERNEL=y in our continuous integration [1]. It does not
> appear to be compiler specific, as it reproduces with a bunch of
> different clang versions and GCC 11.2.0 (I didn't try other GCC
> versions).
> 
> At commit 04e91b732476 ("ARM: early traps initialisation"), everything
> boots fine.
> 
> At commit 8d9d651ff227 ("ARM: use LOADADDR() to get load address of
> sections"), there is no output from QEMU at all.
> 
> At commit b9baf5c8c5c3 ("ARM: Spectre-BHB workaround"), there is some
> output but the boot still hangs before init. I have included a log of
> the output of QEMU at this revision along with the command line I am
> using, which comes from [2]. If I disable CONFIG_HARDEN_BRANCH_HISTORY,
> the kernel boots.
> 
> If there is any further information I can provide or patches I can try,
> I am happy to do so.
> 
> [1]: https://github.com/ClangBuiltLinux/continuous-integration2/runs/5496036256?check_suite_focus=true
> [2]: https://github.com/ClangBuiltLinux/boot-utils
> 
> Cheers,
> Nathan
>