On Tue, May 17, 2022 at 11:14:10AM +0100, Andre Przywara wrote:
> When we boot a machine using a devicetree, the generic DT code goes
> through all nodes with a 'device_type = "memory"' property, and collects
> all memory banks mentioned there. However it does not check for the
> status property, so any nodes which are explicitly "disabled" will still
> be added as a memblock.
> This ends up badly for QEMU, when booting with secure firmware on
> arm/arm64 machines, because QEMU adds a node describing secure-only
> memory:
> ===================
> secram@e000000 {
BTW, 'memory' is the correct node name.
> secure-status = "okay";
> status = "disabled";
> reg = <0x00 0xe000000 0x00 0x1000000>;
> device_type = "memory";
> };
> ===================
>
> The kernel will eventually use that memory block (which is located below
> the main DRAM bank), but accesses to that will be answered with an
> SError:
> ===================
> [ 0.000000] Internal error: synchronous external abort: 96000050 [#1] PREEMPT SMP
> [ 0.000000] Modules linked in:
> [ 0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 5.18.0-rc6-00014-g10c8acb8b679 #524
> [ 0.000000] Hardware name: linux,dummy-virt (DT)
> [ 0.000000] pstate: 200000c5 (nzCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [ 0.000000] pc : new_slab+0x190/0x340
> [ 0.000000] lr : new_slab+0x184/0x340
> [ 0.000000] sp : ffff80000a4b3d10
> ....
> ==================
> The actual crash location and call stack will be somewhat random, and
> depend on the specific allocation of that physical memory range.
>
> As the DT spec[1] explicitly mentions standard properties, add a simple
> check to skip over disabled memory nodes, so that we only use memory
> that is meant for non-secure code to use.
>
> That fixes booting a QEMU arm64 VM with EL3 enabled ("secure=on"), when
> not using UEFI. In this case the QEMU generated DT will be handed on
> to the kernel, which will see the secram node.
> This issue is reproducible when using TF-A together with U-Boot as
> firmware, then booting with the "booti" command.
>
> When using U-Boot as an UEFI provider, the code there [2] explicitly
> filters for disabled nodes when generating the UEFI memory map, so we
> are safe.
> EDK/2 only reads the first bank of the first DT memory node [3] to learn
> about memory, so we got lucky there.
>
> [1] https://github.com/devicetree-org/devicetree-specification/blob/main/source/chapter3-devicenodes.rst#memory-node (after the table)
> [2] https://source.denx.de/u-boot/u-boot/-/blob/master/lib/fdtdec.c#L1061-1063
> [3] https://github.com/tianocore/edk2/blob/master/ArmVirtPkg/PrePi/FdtParser.c
>
> Reported-by: Ross Burton <[email protected]>
> Signed-off-by: Andre Przywara <[email protected]>
> ---
> drivers/of/fdt.c | 3 +++
> 1 file changed, 3 insertions(+)
Applied, thanks!
On Tue, 17 May 2022 at 16:34, Rob Herring <[email protected]> wrote:
>
> On Tue, May 17, 2022 at 11:14:10AM +0100, Andre Przywara wrote:
> > When we boot a machine using a devicetree, the generic DT code goes
> > through all nodes with a 'device_type = "memory"' property, and collects
> > all memory banks mentioned there. However it does not check for the
> > status property, so any nodes which are explicitly "disabled" will still
> > be added as a memblock.
> > This ends up badly for QEMU, when booting with secure firmware on
> > arm/arm64 machines, because QEMU adds a node describing secure-only
> > memory:
> > ===================
> > secram@e000000 {
>
> BTW, 'memory' is the correct node name.
We already have a 'memory' node, which is for the NS
memory. This one's for the secure-only RAM block,
which is why I gave it a name that hopefully helps in
spotting that when a human is reading the DT.
I'm not really sure to what extent node names in device trees are
"this is just an identifying textual label" and to what extent
they are "this is really ABI and you need to follow the standard",
though -- nothing in practice seems to care what they are,
suggesting the "textual label" theory, but some bits of tooling
complain if you do things like forget the address value or use the
same address for two different nodes, suggesting the "really ABI"
theory.
thanks
-- PMM