2021-06-08 19:57:50

by Sean Quinn

[permalink] [raw]
Subject: Reserving memory

Hi,

Kernel newb here, sorry if this is could be sent to a more suitable list.

I have an embedded project using a Zynq based SoM (Enclustra
MA-ZX2-10-2I-D9 SOM). We have a device using a DMA that writes data
into system SDRAM shared by the processing system/OS.

Using Xilinx tools ("2019.2"), which are based on kernel 4.19 I am
able to build an image and use the "mem=384M" boot arg kernel command
to prevent overlap with a reserved region (top 128M of 512M) where our
device is writing data to a ring buffer.

I am still debugging, but unfortunately, this same tactic seems to
fail with newer tools ("2020.1") based on kernel 5.4. That boot arg
results in a kernel panic. I have also tried using a device tree
approach, e.g.

reserved-memory {
#address-cells = <1>;
#size-cells = <1>;
ranges;

reserved: rambuffer@0x18000000 {
no-map;
reg = <0x18000000 0x8000000>;
};
};

also results in kernel panic during boot (pasted at end of msg)

Perhaps my understanding of what mem does is poor and got lucky the
first time. If the kernel error is the correct behavior, could I ask
what is the proper way to configure reserved regions? Any
feedback/criticisms/links welcome

Cheers,

U-Boot 2020.01 (Jun 08 2021 - 01:46:49 +0000)

CPU: Zynq 7z010
Silicon: v3.1
Model: Enclustra MA-ZX2-10-2I-D9 SOM
DRAM: ECC disabled 512 MiB
Flash: 0 Bytes
NAND: 0 MiB
MMC: mmc@e0100000: 0
Loading Environment from FAT... *** Warning - bad CRC, using default environment

Loading Environment from SPI Flash... SF: Detected s25fl512s_256k with
page size 256 Bytes, erase size 256 KiB, total 64 MiB
*** Warning - bad CRC, using default environment

In: serial@e0000000
Out: serial@e0000000
Err: serial@e0000000
Net:
ZYNQ GEM: e000b000, mdio bus e000b000, phyaddr 3, interface rgmii-id
eth0: ethernet@e000b000
Hit any key to stop autoboot: 0
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot.scr
ERROR: reserving fdt memory region failed (addr=18000000 size=8000000)
2010 bytes read in 19 ms (102.5 KiB/s)
## Executing script at 03000000
ERROR: reserving fdt memory region failed (addr=18000000 size=8000000)
4349960 bytes read in 328 ms (12.6 MiB/s)
## Loading kernel from FIT Image at 10000000 ...
Using '[email protected]' configuration
Verifying Hash Integrity ... OK
Trying 'kernel@1' kernel subimage
Description: Linux kernel
Type: Kernel Image
Compression: uncompressed
Data Start: 0x100000f8
Data Size: 4328048 Bytes = 4.1 MiB
Architecture: ARM
OS: Linux
Load Address: 0x00200000
Entry Point: 0x00200000
Hash algo: sha256
Hash value:
efedbd67e6d5dc525817819550a182c9650143669af52c31fa7d49cf57156a25
Verifying Hash Integrity ... sha256+ OK
## Loading fdt from FIT Image at 10000000 ...
Using '[email protected]' configuration
Verifying Hash Integrity ... OK
Trying '[email protected]' fdt subimage
Description: Flattened Device Tree blob
Type: Flat Device Tree
Compression: uncompressed
Data Start: 0x10420c74
Data Size: 20014 Bytes = 19.5 KiB
Architecture: ARM
Hash algo: sha256
Hash value:
640e1e4ba0f8d7d0a696e3cffd4de243b1f193782df8dfb57ab64ddcebb2d2ef
Verifying Hash Integrity ... sha256+ OK
Booting using the fdt blob at 0x10420c74
Loading Kernel Image
ERROR: reserving fdt memory region failed (addr=18000000 size=8000000)
Loading Device Tree to 1eafe000, end 1eb05e2d ... OK

Starting kernel ...

Booting Linux on physical CPU 0x0
Linux version 5.4.0-xilinx-v2020.1 (oe-user@oe-host) (gcc version
9.2.0 (GCC)) #1 SMP PREEMPT Sat Jun 5 00:08:36 UTC 2021
CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=18c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
OF: fdt: Machine model: Enclustra MA-ZX2-10-2I-D9 SOM
earlycon: cdns0 at MMIO 0xe0000000 (options '115200n8')
printk: bootconsole [cdns0] enabled
Memory policy: Data cache writealloc
cma: Reserved 16 MiB at 0x17000000
8<--- cut here ---
Unable to handle kernel paging request at virtual address deafe000
pgd = (ptrval)
[deafe000] *pgd=00000000
Internal error: Oops - BUG: 5 [#1] PREEMPT SMP ARM
Modules linked in:
CPU: 0 PID: 0 Comm: swapper Not tainted 5.4.0-xilinx-v2020.1 #1
Hardware name: Xilinx Zynq Platform
PC is at fdt32_ld+0x0/0xc
LR is at fdt_check_header+0xc/0x124
pc : [<c06a2b30>] lr : [<c06a2bf4>] psr: a00000d3
sp : c0b01f20 ip : 00b582ff fp : d6ffff00
r10: c0b130f4 r9 : c0a1fe78 r8 : 00000000
r7 : deafe000 r6 : c0b51b58 r5 : c0b0833c r4 : c0a1fe78
r3 : deafe000 r2 : c0b51b58 r1 : 00000000 r0 : deafe000
Flags: NzCv IRQs off FIQs off Mode SVC_32 ISA ARM Segment none
Control: 18c5387d Table: 0000404a DAC: 00000051
Process swapper (pid: 0, stack limit = 0x(ptrval))
Stack: (0xc0b01f20 to 0xc0b02000)
1f20: c0a1fe78 c0b0833c c0b51b58 c055c45c c0a1fe78 c0b0833c c0b06f40 c0b37500
1f40: 00000000 c0b46f90 c0b130f4 c0a20fe0 00000000 0000000c c0a2b36c c0a041e8
1f60: ffffffff 18c5387d c0b03c40 c08694b4 00000000 c0b37380 00000001 18c5387d
1f80: 00000000 c0157f9c c07f9851 c0b03c48 00000001 c0a00330 c0b03c40 10c0387d
1fa0: 00000000 c0b37380 00000001 18c5387d 00000000 c0a00970 00000000 00000000
1fc0: 00000000 00000000 00000000 c0a2da40 00000000 c0b03c48 c0a00330 00000051
1fe0: 10c0387d 00000000 1eafe000 413fc090 18c5387d 00000000 00000000 00000000
[<c06a2b30>] (fdt32_ld) from [<c06a2bf4>] (fdt_check_header+0xc/0x124)
[<c06a2bf4>] (fdt_check_header) from [<c055c45c>]
(__unflatten_device_tree+0x28/0xfc)
[<c055c45c>] (__unflatten_device_tree) from [<c0a20fe0>]
(unflatten_device_tree+0x24/0x40)
[<c0a20fe0>] (unflatten_device_tree) from [<c0a041e8>] (setup_arch+0x63c/0xa6c)
[<c0a041e8>] (setup_arch) from [<c0a00970>] (start_kernel+0x58/0x440)
[<c0a00970>] (start_kernel) from [<00000000>] (0x0)
Code: ebf2196f e28dd014 e49df004 c06a2a28 (e5900000)
random: get_random_bytes called from init_oops_id+0x20/0x3c with crng_init=0
---[ end trace 0000000000000000 ]---
Kernel panic - not syncing: Attempted to kill the idle task!
---[ end Kernel panic - not syncing: Attempted to kill the idle task! ]---

--
Sean Quinn
Postdoctoral researcher, GAPS project

475 Portola Plaza
Los Angeles, CA 90095

[email protected] | Office: 310-825-0344 | Web: gaps1.astro.ucla.edu/gaps/


2021-06-09 23:56:12

by Austin Kim

[permalink] [raw]
Subject: Re: Reserving memory

Hello,

2021년 6월 9일 (수) 오후 5:33, Sean Quinn <[email protected]>님이 작성:
>
> Hi,
>
> Kernel newb here, sorry if this is could be sent to a more suitable list.
>
> I have an embedded project using a Zynq based SoM (Enclustra
> MA-ZX2-10-2I-D9 SOM). We have a device using a DMA that writes data
> into system SDRAM shared by the processing system/OS.
>
> [..]
> 8<--- cut here ---
> Unable to handle kernel paging request at virtual address deafe000

It seems that void *fdt contains deafe000, which is not handled by mmu.

https://elixir.bootlin.com/linux/v5.4.124/source/scripts/dtc/libfdt/fdt.c
int fdt_check_header(const void *fdt)
{
size_t hdrsize;

if (fdt_magic(fdt) != FDT_MAGIC) //<<-- exception point
return -FDT_ERR_BADMAGIC;

2021-06-10 21:31:19

by Sean Quinn

[permalink] [raw]
Subject: Re: Reserving memory

Hi Austin,

Thanks for the reply!

Yeah, I am not sure what's going on with that address.

At any rate, I tested some other configurations using only 5.4 now

If I use the updated device tree below, things seem to work alright!
It boots, it's stable, I can devmem stuff into that region etc. Still
have to test out if the DMA device writes will succeed.

/ {
model = "Enclustra MA-ZX2-10-2I-D9 SOM";
chosen {
bootargs = "console=ttyPS0,115200 earlycon vmalloc=512M
root=/dev/mmcblk0p2 rw rootwait";
};
reserved-memory {
#address-cells = <1>;
#size-cells = <1>;
ranges;

fpga_reserved1: rambuffer@0x4000000 {
reg = <0x4000000 0x8400000>;
};
};
};

Boot log

U-Boot 2020.01 (Jun 09 2021 - 21:37:56 +0000)

CPU: Zynq 7z010
Silicon: v3.1
Model: Enclustra MA-ZX2-10-2I-D9 SOM
DRAM: ECC disabled 512 MiB
Flash: 0 Bytes
NAND: 0 MiB
MMC: mmc@e0100000: 0
Loading Environment from FAT... *** Warning - bad CRC, using default environment

Loading Environment from SPI Flash... SF: Detected s25fl512s_256k with
page size 256 Bytes, erase size 256 KiB, total 64 MiB
*** Warning - bad CRC, using default environment

In: serial@e0000000
Out: serial@e0000000
Err: serial@e0000000
Net:
ZYNQ GEM: e000b000, mdio bus e000b000, phyaddr 3, interface rgmii-id
eth0: ethernet@e000b000
Hit any key to stop autoboot: 0
switch to partitions #0, OK
mmc0 is current device
Scanning mmc 0:1...
Found U-Boot script /boot.scr
2010 bytes read in 19 ms (102.5 KiB/s)
## Executing script at 03000000
4349956 bytes read in 328 ms (12.6 MiB/s)
## Loading kernel from FIT Image at 10000000 ...
Using '[email protected]' configuration
Verifying Hash Integrity ... OK
Trying 'kernel@1' kernel subimage
Description: Linux kernel
Type: Kernel Image
Compression: uncompressed
Data Start: 0x100000f8
Data Size: 4328048 Bytes = 4.1 MiB
Architecture: ARM
OS: Linux
Load Address: 0x00200000
Entry Point: 0x00200000
Hash algo: sha256
Hash value:
efedbd67e6d5dc525817819550a182c9650143669af52c31fa7d49cf57156a25
Verifying Hash Integrity ... sha256+ OK
## Loading fdt from FIT Image at 10000000 ...
Using '[email protected]' configuration
Verifying Hash Integrity ... OK
Trying '[email protected]' fdt subimage
Description: Flattened Device Tree blob
Type: Flat Device Tree
Compression: uncompressed
Data Start: 0x10420c74
Data Size: 20009 Bytes = 19.5 KiB
Architecture: ARM
Hash algo: sha256
Hash value:
141f7ed36d3e8e1bc1d72085104e7bc77fa0658be04e427b4c0413ddd3a757c0
Verifying Hash Integrity ... sha256+ OK
Booting using the fdt blob at 0x10420c74
Loading Kernel Image
Loading Device Tree to 1eafe000, end 1eb05e28 ... OK

Starting kernel ...

Booting Linux on physical CPU 0x0
Linux version 5.4.0-xilinx-v2020.1 (oe-user@oe-host) (gcc version
9.2.0 (GCC)) #1 SMP PREEMPT Sat Jun 5 00:08:36 UTC 2021
CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=18c5387d
CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache
OF: fdt: Machine model: Enclustra MA-ZX2-10-2I-D9 SOM
earlycon: cdns0 at MMIO 0xe0000000 (options '115200n8')
printk: bootconsole [cdns0] enabled
Memory policy: Data cache writealloc
cma: Reserved 16 MiB at 0x1d800000
percpu: Embedded 15 pages/cpu s31948 r8192 d21300 u61440
Built 1 zonelists, mobility grouping on. Total pages: 129938
Kernel command line: console=ttyPS0,115200 earlycon vmalloc=512M
root=/dev/mmcblk0p2 rw rootwait
Dentry cache hash table entries: 65536 (order: 6, 262144 bytes, linear)
Inode-cache hash table entries: 32768 (order: 5, 131072 bytes, linear)
mem auto-init: stack:off, heap alloc:off, heap free:off
Memory: 356396K/524288K available (6144K kernel code, 220K rwdata,
1844K rodata, 1024K init, 131K bss, 151508K reserved, 16384K
cma-reserved, 8192K highmem)
rcu: Preemptible hierarchical RCU implementation.
rcu: RCU restricting CPUs from NR_CPUS=4 to nr_cpu_ids=2.
Tasks RCU enabled.
rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=2
NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16
efuse mapped to (ptrval)
slcr mapped to (ptrval)
L2C: platform modifies aux control register: 0x72360000 -> 0x72760000
L2C: DT/platform modifies aux control register: 0x72360000 -> 0x72760000
L2C-310 erratum 769419 enabled
L2C-310 enabling early BRESP for Cortex-A9
L2C-310 full line of zeros enabled for Cortex-A9
L2C-310 ID prefetch enabled, offset 1 lines
L2C-310 dynamic clock gating enabled, standby mode enabled
L2C-310 cache controller enabled, 8 ways, 512 kB
L2C-310: CACHE_ID 0x410000c8, AUX_CTRL 0x76760001
random: get_random_bytes called from start_kernel+0x260/0x440 with crng_init=0
zynq_clock_init: clkc starts at (ptrval)
Zynq clock init
sched_clock: 64 bits at 333MHz, resolution 3ns, wraps every 4398046511103ns
clocksource: arm_global_timer: mask: 0xffffffffffffffff max_cycles:
0x4ce07af025, max_idle_ns: 440795209040 ns
Switching to timer-based delay loop, resolution 3ns
Console: colour dummy device 80x30
Calibrating delay loop (skipped), value calculated using timer
frequency.. 666.66 BogoMIPS (lpj=3333333)
pid_max: default: 32768 minimum: 301
Mount-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
Mountpoint-cache hash table entries: 1024 (order: 0, 4096 bytes, linear)
CPU: Testing write buffer coherency: ok
CPU0: Spectre v2: using BPIALL workaround
CPU0: thread -1, cpu 0, socket 0, mpidr 80000000
Setting up static identity map for 0x100000 - 0x100060
rcu: Hierarchical SRCU implementation.
smp: Bringing up secondary CPUs ...
CPU1: thread -1, cpu 1, socket 0, mpidr 80000001
CPU1: Spectre v2: using BPIALL workaround
smp: Brought up 1 node, 2 CPUs
SMP: Total of 2 processors activated (1333.33 BogoMIPS).
CPU: All CPU(s) started in SVC mode.
devtmpfs: initialized
VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 4
clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff,
max_idle_ns: 19112604462750000 ns
futex hash table entries: 512 (order: 3, 32768 bytes, linear)
pinctrl core: initialized pinctrl subsystem
NET: Registered protocol family 16
DMA: preallocated 256 KiB pool for atomic coherent allocations
cpuidle: using governor menu
hw-breakpoint: found 5 (+1 reserved) breakpoint and 1 watchpoint registers.
hw-breakpoint: maximum watchpoint size is 4 bytes.
zynq-ocm f800c000.ocmc: ZYNQ OCM pool: 256 KiB @ 0x(ptrval)
e0000000.serial: ttyPS0 at MMIO 0xe0000000 (irq = 25, base_baud =
6249999) is a xuartps
printk: console [ttyPS0] enabled
printk: console [ttyPS0] enabled
printk: bootconsole [cdns0] disabled
printk: bootconsole [cdns0] disabled
vgaarb: loaded
SCSI subsystem initialized
usbcore: registered new interface driver usbfs
usbcore: registered new interface driver hub
usbcore: registered new device driver usb
mc: Linux media interface: v0.10
videodev: Linux video capture interface: v2.00
pps_core: LinuxPPS API ver. 1 registered
pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti
<[email protected]>
PTP clock support registered
EDAC MC: Ver: 3.0.0
FPGA manager framework
Advanced Linux Sound Architecture Driver Initialized.
clocksource: Switched to clocksource arm_global_timer
thermal_sys: Registered thermal governor 'step_wise'
NET: Registered protocol family 2
tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 6144 bytes, linear)
TCP established hash table entries: 4096 (order: 2, 16384 bytes, linear)
TCP bind hash table entries: 4096 (order: 3, 32768 bytes, linear)
TCP: Hash tables configured (established 4096 bind 4096)
UDP hash table entries: 256 (order: 1, 8192 bytes, linear)
UDP-Lite hash table entries: 256 (order: 1, 8192 bytes, linear)
NET: Registered protocol family 1
RPC: Registered named UNIX socket transport module.
RPC: Registered udp transport module.
RPC: Registered tcp transport module.
RPC: Registered tcp NFSv4.1 backchannel transport module.
PCI: CLS 0 bytes, default 64
hw perfevents: no interrupt-affinity property for /pmu@f8891000, guessing.
hw perfevents: enabled with armv7_cortex_a9 PMU driver, 7 counters available
workingset: timestamp_bits=14 max_order=17 bucket_order=3
jffs2: version 2.2. (NAND) (SUMMARY) © 2001-2006 Red Hat, Inc.
bounce: pool size: 64 pages
io scheduler mq-deadline registered
io scheduler kyber registered
zynq-pinctrl 700.pinctrl: zynq pinctrl initialized
dma-pl330 f8003000.dmac: Loaded driver for PL330 DMAC-241330
dma-pl330 f8003000.dmac: DBUFF-128x8bytes Num_Chans-8 Num_Peri-4 Num_Events-16
brd: module loaded
loop: module loaded
spi_master spi0: cannot find modalias for /amba/spi@e000d000/flash@0
spi_master spi0: Failed to create SPI device for /amba/spi@e000d000/flash@0
libphy: Fixed MDIO Bus: probed
CAN device driver interface
libphy: MACB_mii_bus: probed
Micrel KSZ9031 Gigabit PHY e000b000.ethernet-ffffffff:03: attached PHY
driver [Micrel KSZ9031 Gigabit PHY]
(mii_bus:phy_addr=e000b000.ethernet-ffffffff:03, irq=POLL)
macb e000b000.ethernet eth0: Cadence GEM rev 0x00020118 at 0xe000b000
irq 27 (XX:XX:XX:XX:XX:XX)
e1000e: Intel(R) PRO/1000 Network Driver - 3.2.6-k
e1000e: Copyright(c) 1999 - 2015 Intel Corporation.
ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
ehci-pci: EHCI PCI platform driver
usbcore: registered new interface driver usb-storage
i2c /dev entries driver
cdns-i2c e0004000.i2c: 100 kHz mmio e0004000 irq 22
cdns-wdt f8005000.watchdog: Xilinx Watchdog Timer with timeout 10s
EDAC MC: ECC not enabled
Xilinx Zynq CpuIdle Driver started
sdhci: Secure Digital Host Controller Interface driver
sdhci: Copyright(c) Pierre Ossman
sdhci-pltfm: SDHCI platform and OF driver helper
mmc0: SDHCI controller on e0100000.mmc [e0100000.mmc] using ADMA
ledtrig-cpu: registered to indicate activity on CPUs
clocksource: ttc_clocksource: mask: 0xffff max_cycles: 0xffff,
max_idle_ns: 537538477 ns
timer #0 at (ptrval), irq=40
usbcore: registered new interface driver usbhid
usbhid: USB HID core driver
fpga_manager fpga0: Xilinx Zynq FPGA Manager registered
NET: Registered protocol family 10
Segment Routing with IPv6
sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver
NET: Registered protocol family 17
can: controller area network core (rev 20170425 abi 9)
NET: Registered protocol family 29
can: raw protocol (rev 20170425)
can: broadcast manager protocol (rev 20170425 t)
can: netlink gateway (rev 20190810) max_hops=1
Registering SWP/SWPB emulation handler
of-fpga-region fpga-full: FPGA Region probed
hctosys: unable to open rtc device (rtc0)
of_cfs_init
of_cfs_init: OK
ALSA device list:
No soundcards found.
Waiting for root device /dev/mmcblk0p2...
mmc0: new high speed SDHC card at address e624
mmcblk0: mmc0:e624 SA08G 7.40 GiB
mmcblk0: p1 p2
EXT4-fs (mmcblk0p2): mounted filesystem with ordered data mode. Opts: (null)
VFS: Mounted root (ext4 filesystem) on device 179:2.
devtmpfs: mounted
Freeing unused kernel memory: 1024K
Run /sbin/init as init process
random: fast init done
systemd[1]: System time before build time, advancing clock.
systemd[1]: Failed to look up module alias 'autofs4': Function not implemented
systemd[1]: systemd 245.4-4ubuntu3.6 running in system mode. (+PAM
+AUDIT +SELINUX +IMA +APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP
+GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2
-IDN +PCRE2 default-hierarchy=hybrid)
systemd[1]: Detected architecture arm.

On Wed, Jun 9, 2021 at 4:51 PM Austin Kim <[email protected]> wrote:
>
> Hello,
>
> 2021년 6월 9일 (수) 오후 5:33, Sean Quinn <[email protected]>님이 작성:
> >
> > Hi,
> >
> > Kernel newb here, sorry if this is could be sent to a more suitable list.
> >
> > I have an embedded project using a Zynq based SoM (Enclustra
> > MA-ZX2-10-2I-D9 SOM). We have a device using a DMA that writes data
> > into system SDRAM shared by the processing system/OS.
> >
> > [..]
> > 8<--- cut here ---
> > Unable to handle kernel paging request at virtual address deafe000
>
> It seems that void *fdt contains deafe000, which is not handled by mmu.
>
> https://elixir.bootlin.com/linux/v5.4.124/source/scripts/dtc/libfdt/fdt.c
> int fdt_check_header(const void *fdt)
> {
> size_t hdrsize;
>
> if (fdt_magic(fdt) != FDT_MAGIC) //<<-- exception point
> return -FDT_ERR_BADMAGIC;