The arm64 WinLink E850-96 Board boot failed with 16K and 64K page size builds
Please find the below warning log on Linux next-20240229.
First noticed on the next-20240220 tag.
This issue arises only when one of these Kconfig options is enabled.
CONFIG_ARM64_16K_PAGES=y
CONFIG_ARM64_64K_PAGES=y
Reported-by: Linux Kernel Functional Testing <[email protected]>
Warning log:
-------
[ 2.231008] mmc_host mmc0: Bus speed (slot 0) = 49968750Hz (slot
req 52000000Hz, actual 49968750HZ div = 0)
[ 2.231714] mmc_host mmc0: Bus speed (slot 0) = 399750000Hz (slot
req 200000000Hz, actual 199875000HZ div = 1)
[ 2.241961] mmc0: new HS400 Enhanced strobe MMC card at address 0001
[ 2.249182] ------------[ cut here ]------------
[ 2.252371] WARNING: CPU: 3 PID: 90 at block/blk-settings.c:204
blk_validate_limits (block/blk-settings.c:204 (discriminator 1))
[ 2.261056] Modules linked in:
[ 2.264082] CPU: 3 PID: 90 Comm: kworker/3:1 Not tainted
6.8.0-rc6-next-20240229 #1
[ 2.271724] Hardware name: WinLink E850-96 board (DT)
[ 2.276756] Workqueue: events_freezable mmc_rescan
[ 2.281534] pstate: 80400005 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 2.288479] pc : blk_validate_limits (block/blk-settings.c:204
(discriminator 1))
[ 2.293076] lr : blk_set_default_limits (block/blk-settings.c:241)
<trim>
[ 2.337595] PM: genpd: Disabling unused power domains
[ 2.338361] xhci-hcd xhci-hcd.0.auto: remove, state 84
[ 2.338391] usb usb1: USB disconnect, device number 1
[ 2.343322] x1 : 0000000000001000 x0 : ffff8000a504f888
[ 2.343335] Call trace:
[ 2.343339] blk_validate_limits (block/blk-settings.c:204 (discriminator 1))
[ 2.343350] blk_alloc_queue (block/blk-core.c:421)
[ 2.343362] blk_mq_alloc_queue (block/blk-mq.c:4083 (discriminator 4))
[ 2.350571] ALSA device list:
[ 2.352700] xhci-hcd xhci-hcd.0.auto: USB bus 1 deregistered
[ 2.355651] __blk_mq_alloc_disk (block/blk-mq.c:4130)
[ 2.355664] mmc_alloc_disk (drivers/mmc/core/queue.c:379)
[ 2.355678] mmc_init_queue (drivers/mmc/core/queue.c:463)
[ 2.359577] No soundcards found.
[ 2.362942] mmc_blk_alloc_req (drivers/mmc/core/block.c:2500
(discriminator 1))
[ 2.362951] mmc_blk_probe (drivers/mmc/core/block.c:2603
drivers/mmc/core/block.c:3025)
[ 2.446877] mmc_bus_probe (drivers/mmc/core/bus.c:132)
[ 2.446892] really_probe (drivers/base/dd.c:579 drivers/base/dd.c:658)
[ 2.454016] __driver_probe_device (drivers/base/dd.c:800)
[ 2.458346] driver_probe_device (drivers/base/dd.c:830)
[ 2.462513] __device_attach_driver (drivers/base/dd.c:959)
[ 2.466939] bus_for_each_drv (drivers/base/bus.c:457)
[ 2.470761] __device_attach (drivers/base/dd.c:1032)
[ 2.474579] device_initial_probe (drivers/base/dd.c:1080)
[ 2.478746] bus_probe_device (drivers/base/bus.c:532)
[ 2.482563] device_add (drivers/base/core.c:3646)
[ 2.486037] mmc_add_card (drivers/mmc/core/bus.c:371)
[ 2.489683] mmc_attach_mmc (drivers/mmc/core/mmc.c:2353)
[ 2.493500] mmc_rescan (drivers/mmc/core/core.c:2097 (discriminator
1) drivers/mmc/core/core.c:2259 (discriminator 1))
[ 2.496974] process_one_work (kernel/workqueue.c:3253)
[ 2.500966] worker_thread (kernel/workqueue.c:3322 (discriminator
2) kernel/workqueue.c:3409 (discriminator 2))
[ 2.504700] kthread (kernel/kthread.c:388)
[ 2.507914] ret_from_fork (arch/arm64/kernel/entry.S:861)
[ 2.511473] ---[ end trace 0000000000000000 ]---
[ 2.516435] mmcblk: probe of mmc0:0001 failed with error -22
[ 2.522114] Waiting for root device /dev/mmcblk0p12...
Kconfig:
- https://storage.tuxsuite.com/public/linaro/lkft/builds/2ccVUNsQjaWBcREYI0wFRjzG3oa/config
Steps to reproduce:
- https://tuxapi.tuxsuite.com/v1/groups/linaro/projects/lkft/tests/2ccVWjLtBJ3IuDSKLkyaFholQ67/reproducer
Links:
- https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20240229/testrun/22884460/suite/boot/test/gcc-13-lkftconfig-16k_page_size/log
- https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20240229/testrun/22884435/suite/boot/test/gcc-13-lkftconfig-64k_page_size/details/
- https://tuxapi.tuxsuite.com/v1/groups/linaro/projects/lkft/tests/2d2JxLaSCZ6cTNGMeKdCbAjk9Sh
- https://storage.tuxsuite.com/public/linaro/lkft/builds/2d2JuyeiEHCtWGJYV80n1tF77lh/
--
Linaro LKFT
https://lkft.linaro.org
On Thu, Feb 29, 2024 at 07:44:01PM +0530, Naresh Kamboju wrote:
> The arm64 WinLink E850-96 Board boot failed with 16K and 64K page size builds
> Please find the below warning log on Linux next-20240229.
> First noticed on the next-20240220 tag.
>
> This issue arises only when one of these Kconfig options is enabled.
> CONFIG_ARM64_16K_PAGES=y
> CONFIG_ARM64_64K_PAGES=y
That means this device doesn't set a max_segment_size, or one smaller
than the page size. This configurtio has never been supported by
Linux (Bart has some patches to try to make it work), but with the
new block limits API we now actively catchthis and warn.
On Thu, Feb 29, 2024, at 15:14, Naresh Kamboju wrote:
> The arm64 WinLink E850-96 Board boot failed with 16K and 64K page size builds
> Please find the below warning log on Linux next-20240229.
> First noticed on the next-20240220 tag.
>
> This issue arises only when one of these Kconfig options is enabled.
> CONFIG_ARM64_16K_PAGES=y
> CONFIG_ARM64_64K_PAGES=y
>
> Reported-by: Linux Kernel Functional Testing <[email protected]>
>
> Warning log:
> -------
> [ 2.231008] mmc_host mmc0: Bus speed (slot 0) = 49968750Hz (slot
> req 52000000Hz, actual 49968750HZ div = 0)
> [ 2.231714] mmc_host mmc0: Bus speed (slot 0) = 399750000Hz (slot
> req 200000000Hz, actual 199875000HZ div = 1)
> [ 2.241961] mmc0: new HS400 Enhanced strobe MMC card at address 0001
> [ 2.249182] ------------[ cut here ]------------
> [ 2.252371] WARNING: CPU: 3 PID: 90 at block/blk-settings.c:204
> blk_validate_limits (block/blk-settings.c:204 (discriminator 1))
The warning was added with commit d690cb8ae14b ("block: add
an API to atomically update queue limits")
+ if (!lim->max_segment_size)
+ lim->max_segment_size = BLK_MAX_SEGMENT_SIZE;
+ if (WARN_ON_ONCE(lim->max_segment_size < PAGE_SIZE))
+ return -EINVAL;
Whereas mmc_alloc_disk sets the limit as
/*
* Setting a virt_boundary implicity sets a max_segment_size, so try
* to set the hardware one here.
*/
if (host->can_dma_map_merge) {
lim.virt_boundary_mask = dma_get_merge_boundary(mmc_dev(host));
lim.max_segments = MMC_DMA_MAP_MERGE_SEGMENTS;
} else {
lim.max_segment_size =
round_down(host->max_seg_size, lim.logical_block_size);
lim.max_segments = host->max_segs;
}
and max_seg_size gets initialized to either one less
than 64k, or to exactly 4k in the dwmmc driver here:
/* Useful defaults if platform data is unset. */
if (host->use_dma == TRANS_MODE_IDMAC) {
mmc->max_segs = host->ring_size;
mmc->max_blk_size = 65535;
mmc->max_seg_size = 0x1000;
mmc->max_req_size = mmc->max_seg_size * host->ring_size;
mmc->max_blk_count = mmc->max_req_size / 512;
} else if (host->use_dma == TRANS_MODE_EDMAC) {
mmc->max_segs = 64;
mmc->max_blk_size = 65535;
mmc->max_blk_count = 65535;
mmc->max_req_size =
mmc->max_blk_size * mmc->max_blk_count;
mmc->max_seg_size = mmc->max_req_size;
} else {
/* TRANS_MODE_PIO */
mmc->max_segs = 64;
mmc->max_blk_size = 65535; /* BLKSIZ is 16 bits */
mmc->max_blk_count = 512;
mmc->max_req_size = mmc->max_blk_size *
mmc->max_blk_count;
mmc->max_seg_size = mmc->max_req_size;
}
Adding the mmc and dw-mmc maintainers to cc for clarification.
Arnd
> [ 2.261056] Modules linked in:
> [ 2.264082] CPU: 3 PID: 90 Comm: kworker/3:1 Not tainted
> 6.8.0-rc6-next-20240229 #1
> [ 2.271724] Hardware name: WinLink E850-96 board (DT)
> [ 2.276756] Workqueue: events_freezable mmc_rescan
> [ 2.281534] pstate: 80400005 (Nzcv daif +PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [ 2.288479] pc : blk_validate_limits (block/blk-settings.c:204
> (discriminator 1))
> [ 2.293076] lr : blk_set_default_limits (block/blk-settings.c:241)
> <trim>
> [ 2.337595] PM: genpd: Disabling unused power domains
> [ 2.338361] xhci-hcd xhci-hcd.0.auto: remove, state 84
> [ 2.338391] usb usb1: USB disconnect, device number 1
> [ 2.343322] x1 : 0000000000001000 x0 : ffff8000a504f888
> [ 2.343335] Call trace:
> [ 2.343339] blk_validate_limits (block/blk-settings.c:204 (discriminator 1))
> [ 2.343350] blk_alloc_queue (block/blk-core.c:421)
> [ 2.343362] blk_mq_alloc_queue (block/blk-mq.c:4083 (discriminator 4))
> [ 2.350571] ALSA device list:
> [ 2.352700] xhci-hcd xhci-hcd.0.auto: USB bus 1 deregistered
> [ 2.355651] __blk_mq_alloc_disk (block/blk-mq.c:4130)
> [ 2.355664] mmc_alloc_disk (drivers/mmc/core/queue.c:379)
> [ 2.355678] mmc_init_queue (drivers/mmc/core/queue.c:463)
> [ 2.359577] No soundcards found.
> [ 2.362942] mmc_blk_alloc_req (drivers/mmc/core/block.c:2500
> (discriminator 1))
> [ 2.362951] mmc_blk_probe (drivers/mmc/core/block.c:2603
> drivers/mmc/core/block.c:3025)
> [ 2.446877] mmc_bus_probe (drivers/mmc/core/bus.c:132)
> [ 2.446892] really_probe (drivers/base/dd.c:579 drivers/base/dd.c:658)
> [ 2.454016] __driver_probe_device (drivers/base/dd.c:800)
> [ 2.458346] driver_probe_device (drivers/base/dd.c:830)
> [ 2.462513] __device_attach_driver (drivers/base/dd.c:959)
> [ 2.466939] bus_for_each_drv (drivers/base/bus.c:457)
> [ 2.470761] __device_attach (drivers/base/dd.c:1032)
> [ 2.474579] device_initial_probe (drivers/base/dd.c:1080)
> [ 2.478746] bus_probe_device (drivers/base/bus.c:532)
> [ 2.482563] device_add (drivers/base/core.c:3646)
> [ 2.486037] mmc_add_card (drivers/mmc/core/bus.c:371)
> [ 2.489683] mmc_attach_mmc (drivers/mmc/core/mmc.c:2353)
> [ 2.493500] mmc_rescan (drivers/mmc/core/core.c:2097 (discriminator
> 1) drivers/mmc/core/core.c:2259 (discriminator 1))
> [ 2.496974] process_one_work (kernel/workqueue.c:3253)
> [ 2.500966] worker_thread (kernel/workqueue.c:3322 (discriminator
> 2) kernel/workqueue.c:3409 (discriminator 2))
> [ 2.504700] kthread (kernel/kthread.c:388)
> [ 2.507914] ret_from_fork (arch/arm64/kernel/entry.S:861)
> [ 2.511473] ---[ end trace 0000000000000000 ]---
> [ 2.516435] mmcblk: probe of mmc0:0001 failed with error -22
> [ 2.522114] Waiting for root device /dev/mmcblk0p12...
>
> Kconfig:
> -
> https://storage.tuxsuite.com/public/linaro/lkft/builds/2ccVUNsQjaWBcREYI0wFRjzG3oa/config
>
> Steps to reproduce:
> -
> https://tuxapi.tuxsuite.com/v1/groups/linaro/projects/lkft/tests/2ccVWjLtBJ3IuDSKLkyaFholQ67/reproducer
>
> Links:
> -
> https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20240229/testrun/22884460/suite/boot/test/gcc-13-lkftconfig-16k_page_size/log
> -
> https://qa-reports.linaro.org/lkft/linux-next-master/build/next-20240229/testrun/22884435/suite/boot/test/gcc-13-lkftconfig-64k_page_size/details/
> -
> https://tuxapi.tuxsuite.com/v1/groups/linaro/projects/lkft/tests/2d2JxLaSCZ6cTNGMeKdCbAjk9Sh
> -
> https://storage.tuxsuite.com/public/linaro/lkft/builds/2d2JuyeiEHCtWGJYV80n1tF77lh/
>
>
> --
> Linaro LKFT
> https://lkft.linaro.org
On Thu, Feb 29, 2024 at 8:56 AM Arnd Bergmann <[email protected]> wrote:
>
> On Thu, Feb 29, 2024, at 15:14, Naresh Kamboju wrote:
> > The arm64 WinLink E850-96 Board boot failed with 16K and 64K page size builds
> > Please find the below warning log on Linux next-20240229.
> > First noticed on the next-20240220 tag.
> >
> > This issue arises only when one of these Kconfig options is enabled.
> > CONFIG_ARM64_16K_PAGES=y
> > CONFIG_ARM64_64K_PAGES=y
> >
> > Reported-by: Linux Kernel Functional Testing <[email protected]>
> >
> > Warning log:
> > -------
> > [ 2.231008] mmc_host mmc0: Bus speed (slot 0) = 49968750Hz (slot
> > req 52000000Hz, actual 49968750HZ div = 0)
> > [ 2.231714] mmc_host mmc0: Bus speed (slot 0) = 399750000Hz (slot
> > req 200000000Hz, actual 199875000HZ div = 1)
> > [ 2.241961] mmc0: new HS400 Enhanced strobe MMC card at address 0001
> > [ 2.249182] ------------[ cut here ]------------
> > [ 2.252371] WARNING: CPU: 3 PID: 90 at block/blk-settings.c:204
> > blk_validate_limits (block/blk-settings.c:204 (discriminator 1))
>
>
> The warning was added with commit d690cb8ae14b ("block: add
> an API to atomically update queue limits")
>
> + if (!lim->max_segment_size)
> + lim->max_segment_size = BLK_MAX_SEGMENT_SIZE;
> + if (WARN_ON_ONCE(lim->max_segment_size < PAGE_SIZE))
> + return -EINVAL;
>
> Whereas mmc_alloc_disk sets the limit as
>
> /*
> * Setting a virt_boundary implicity sets a max_segment_size, so try
> * to set the hardware one here.
> */
> if (host->can_dma_map_merge) {
> lim.virt_boundary_mask = dma_get_merge_boundary(mmc_dev(host));
> lim.max_segments = MMC_DMA_MAP_MERGE_SEGMENTS;
> } else {
> lim.max_segment_size =
> round_down(host->max_seg_size, lim.logical_block_size);
> lim.max_segments = host->max_segs;
> }
>
> and max_seg_size gets initialized to either one less
> than 64k, or to exactly 4k in the dwmmc driver here:
>
> /* Useful defaults if platform data is unset. */
> if (host->use_dma == TRANS_MODE_IDMAC) {
> mmc->max_segs = host->ring_size;
> mmc->max_blk_size = 65535;
Changing this value here to PAGE_SIZE (instead of 65535) makes E850-96
functional again (tested with CONFIG_ARM64_16K_PAGES=y). I'm sure this
is just a quick hack, but maybe it can be helpful in further analysis.
If anybody wants me to run some tests on E850-96 -- please let me
know.
[snip]
On Fri, Mar 1, 2024 at 2:51 PM Sam Protsenko <[email protected]> wrote:
>
> On Thu, Feb 29, 2024 at 8:56 AM Arnd Bergmann <[email protected]> wrote:
> >
> > On Thu, Feb 29, 2024, at 15:14, Naresh Kamboju wrote:
> > > The arm64 WinLink E850-96 Board boot failed with 16K and 64K page size builds
> > > Please find the below warning log on Linux next-20240229.
> > > First noticed on the next-20240220 tag.
> > >
> > > This issue arises only when one of these Kconfig options is enabled.
> > > CONFIG_ARM64_16K_PAGES=y
> > > CONFIG_ARM64_64K_PAGES=y
> > >
> > > Reported-by: Linux Kernel Functional Testing <[email protected]>
> > >
> > > Warning log:
> > > -------
> > > [ 2.231008] mmc_host mmc0: Bus speed (slot 0) = 49968750Hz (slot
> > > req 52000000Hz, actual 49968750HZ div = 0)
> > > [ 2.231714] mmc_host mmc0: Bus speed (slot 0) = 399750000Hz (slot
> > > req 200000000Hz, actual 199875000HZ div = 1)
> > > [ 2.241961] mmc0: new HS400 Enhanced strobe MMC card at address 0001
> > > [ 2.249182] ------------[ cut here ]------------
> > > [ 2.252371] WARNING: CPU: 3 PID: 90 at block/blk-settings.c:204
> > > blk_validate_limits (block/blk-settings.c:204 (discriminator 1))
> >
> >
> > The warning was added with commit d690cb8ae14b ("block: add
> > an API to atomically update queue limits")
> >
> > + if (!lim->max_segment_size)
> > + lim->max_segment_size = BLK_MAX_SEGMENT_SIZE;
> > + if (WARN_ON_ONCE(lim->max_segment_size < PAGE_SIZE))
> > + return -EINVAL;
> >
> > Whereas mmc_alloc_disk sets the limit as
> >
> > /*
> > * Setting a virt_boundary implicity sets a max_segment_size, so try
> > * to set the hardware one here.
> > */
> > if (host->can_dma_map_merge) {
> > lim.virt_boundary_mask = dma_get_merge_boundary(mmc_dev(host));
> > lim.max_segments = MMC_DMA_MAP_MERGE_SEGMENTS;
> > } else {
> > lim.max_segment_size =
> > round_down(host->max_seg_size, lim.logical_block_size);
> > lim.max_segments = host->max_segs;
> > }
> >
> > and max_seg_size gets initialized to either one less
> > than 64k, or to exactly 4k in the dwmmc driver here:
> >
> > /* Useful defaults if platform data is unset. */
> > if (host->use_dma == TRANS_MODE_IDMAC) {
> > mmc->max_segs = host->ring_size;
> > mmc->max_blk_size = 65535;
>
> Changing this value here to PAGE_SIZE (instead of 65535) makes E850-96
> functional again (tested with CONFIG_ARM64_16K_PAGES=y). I'm sure this
> is just a quick hack, but maybe it can be helpful in further analysis.
> If anybody wants me to run some tests on E850-96 -- please let me
> know.
>
Sorry, just noticed I commented on the wrong line. Here is the change I made:
- mmc->max_seg_size = 0x1000;
+ mmc->max_seg_size = PAGE_SIZE;
for (host->use_dma == TRANS_MODE_IDMAC) case.
> [snip]
On Fri, Mar 1, 2024 at 3:18 PM Sam Protsenko <[email protected]> wrote:
>
[snip]
>
> Sorry, just noticed I commented on the wrong line. Here is the change I made:
>
> - mmc->max_seg_size = 0x1000;
> + mmc->max_seg_size = PAGE_SIZE;
>
> for (host->use_dma == TRANS_MODE_IDMAC) case.
>
Just submitted the fix [1]. Please review.
[1] https://lore.kernel.org/all/[email protected]/T/#u
> > [snip]
On Fri, Mar 1, 2024, at 22:18, Sam Protsenko wrote:
> On Fri, Mar 1, 2024 at 2:51 PM Sam Protsenko <[email protected]> wrote:
>> On Thu, Feb 29, 2024 at 8:56 AM Arnd Bergmann <[email protected]> wrote:
>
> Sorry, just noticed I commented on the wrong line. Here is the change I made:
>
> - mmc->max_seg_size = 0x1000;
> + mmc->max_seg_size = PAGE_SIZE;
I went over all MMC drivers to see what else sets a max_seg_size
smaller than a page and found these:
drivers/mmc/host/alcor.c: mmc->max_seg_size = AU6601_MAX_DMA_BLOCK_SIZE; // 0x1000
drivers/mmc/host/au1xmmc.c: mmc->max_seg_size = AU1100_MMC_DESCRIPTOR_SIZE; // 64K-1
drivers/mmc/host/dw_mmc.c: mmc->max_seg_size = 0x1000;
drivers/mmc/host/meson-gx-mmc.c: mmc->max_seg_size = mmc->max_req_size; // 1536 bytes
drivers/mmc/host/mmci_stm32_sdmmc.c: host->mmc->max_seg_size = host->variant->stm32_idmabsize_mask; // GENMASK(12, 5),
drivers/mmc/host/sunxi-mmc.c: mmc->max_seg_size = (1 << host->cfg->idma_des_size_bits); // 1 << 13, only on arm32
drivers/mmc/host/wmt-sdmmc.c: .max_seg_size = 65024,
I've tried to add the maintainers to Cc here, these likely all
need attention to work with large page sizes, in case of
meson-gx-mmc it even seems like the limit is less than a 4KB
page, so it will stop working entirely.
There are also a couple of drivers that look like they have
an off-by-one error and pass a segment size of one less than
a power-off-two number, e.g.:
drivers/mmc/host/davinci_mmc.c: mmc->max_seg_size = MAX_CCNT * rw_threshold; // (64k-1) * 32
drivers/mmc/host/atmel-mci.c: mmc->max_seg_size = mmc->max_blk_size * mmc->max_segs; // 4095*256
I think some of these are intentional, while others are
probably bugs.
Arnd
On 2/29/24 06:21, Christoph Hellwig wrote:
> On Thu, Feb 29, 2024 at 07:44:01PM +0530, Naresh Kamboju wrote:
>> The arm64 WinLink E850-96 Board boot failed with 16K and 64K page size builds
>> Please find the below warning log on Linux next-20240229.
>> First noticed on the next-20240220 tag.
>>
>> This issue arises only when one of these Kconfig options is enabled.
>> CONFIG_ARM64_16K_PAGES=y
>> CONFIG_ARM64_64K_PAGES=y
>
> That means this device doesn't set a max_segment_size, or one smaller
> than the page size. This configurtio has never been supported by
> Linux (Bart has some patches to try to make it work), but with the
> new block limits API we now actively catch this and warn.
Hi Christoph,
Please let me know if you would like me to repost the patch series that
adds support to the block layer for DMA segments that are smaller than
the page size. Despite my requests to not support such hardware in the
Android kernel, that patch series ended up as an out-of-tree series
in the Android kernel and will continue to be maintained by the Android
core team for considerable time.
Thanks,
Bart.