Add uImage.FIT partition parser and wire it up to allow mounting
filesystem sub-images from uImage.FIT in GPT partitions as well as
mtdblock and ubiblock devices within Linux (e.g. as root filesystem).
Using uImage.FIT to store the root filesystem besides kernel and dtb has
several obvious advantages which are hard to obtain in any other way:
* single image accross different storage types
* dynamically sized partitions for kernel and rootfs
* hash also for rootfs checked by U-Boot before launching kernel
* images may include additional filesystems e.g. for localization or
branding
For this to work, the image has to be created with external data and
sub-images aligned to the system's memory page boundaries, ie.
mkimage -E -B 0x1000 -p 0x1000 ...
Booting such images has been supported by U-Boot since v2018.01.
A previous version of this partition parser is in production use on some
OpenWrt devices, eg. the BananaPi R64 where using the FIT parser allows
booting the very same image from eMMC, SD Card or SPI-NAND/UBI and also
using it as a firmware-upgrade image at the same time.
The Ubiquiti UniFi 6 LR access point served as a reference board with
SPI-NOR flash and use of the partition parser on top of a mtdblock
device.
As U-Boot by now also passes down the selected configuration node name
via device tree this allows the partition parser (or userspace process
via sysfs) to identify the selected image configuration.
Device Tree schema for that:
https://github.com/devicetree-org/dt-schema/commit/a24d97d43491e55d4def006213213a6c4045b646
In most cases this partition parser can be used without relying on the
bootloader to pass-down the configuration node name. The default
configuration node is used then.
Changes since v4:
* use folio instead of page (requested by Matthew Wilcox)
* remove unneeded checks (requested by Matthew Wilcox)
* use strscpy instead of strlcpy (strlcpy is deprecated)
* fix off-by-one string size limit label_min (discovered during tests)
* use only a single config symbol MTD_BLOCK_PARTITION affecting both
ubiblock and mtdblock (requested by Richard Weinberger)
Changes since v3:
* use min_t(size_t, ...)
Changes since v2:
* use returned length to limit all strings read from fit/dt
* use __be32 type for 32-bit values read from fit/dt
Reported-by: kernel test robot <[email protected]>
* Kconfig: select LIBFDT for FIT_PARTITION
Reported-by: kernel test robot <[email protected]>
Changes since v1:
* Use again #ifdef's in partitions/efi to only build against FIT
parser symbols if it is actually selected. Otherwise the efi/gpt
would unconditionally depend on the FTT parser to be present.
Changes since RFC:
* fixed wrong variable used in error path
* introduced dedicated Kconfig options to enable partition
parsers on mtdblock and ubiblock
* drop #ifdef'ery, use IS_ENABLED(...) where needed
Daniel Golle (4):
block: add new flag to add partitions read-only
block: add partition parser for U-Boot uImage.FIT
partitions/efi: add support for uImage.FIT sub-partitions
mtd: add option to enable scanning for partitions
MAINTAINERS | 6 +
block/blk.h | 1 +
block/partitions/Kconfig | 15 ++
block/partitions/Makefile | 1 +
block/partitions/check.h | 4 +
block/partitions/core.c | 6 +
block/partitions/efi.c | 9 +
block/partitions/efi.h | 3 +
block/partitions/fit.c | 346 ++++++++++++++++++++++++++++++++++++++
drivers/mtd/Kconfig | 11 ++
drivers/mtd/mtd_blkdevs.c | 4 +-
drivers/mtd/ubi/block.c | 5 +-
12 files changed, 409 insertions(+), 2 deletions(-)
create mode 100644 block/partitions/fit.c
--
2.38.1
Add new GUID allowing to parse uImage.FIT stored in a GPT partition
and map filesystem sub-image as sub-partitions.
Signed-off-by: Daniel Golle <[email protected]>
---
block/partitions/efi.c | 9 +++++++++
block/partitions/efi.h | 3 +++
2 files changed, 12 insertions(+)
diff --git a/block/partitions/efi.c b/block/partitions/efi.c
index 5e9be13a56a8..f4406b443f04 100644
--- a/block/partitions/efi.c
+++ b/block/partitions/efi.c
@@ -716,6 +716,9 @@ int efi_partition(struct parsed_partitions *state)
gpt_entry *ptes = NULL;
u32 i;
unsigned ssz = queue_logical_block_size(state->disk->queue) / 512;
+#ifdef CONFIG_FIT_PARTITION
+ u32 extra_slot = 65;
+#endif
if (!find_valid_gpt(state, &gpt, &ptes) || !gpt || !ptes) {
kfree(gpt);
@@ -749,6 +752,12 @@ int efi_partition(struct parsed_partitions *state)
ARRAY_SIZE(ptes[i].partition_name));
utf16_le_to_7bit(ptes[i].partition_name, label_max, info->volname);
state->parts[i + 1].has_info = true;
+ /* If this is a U-Boot FIT volume it may have subpartitions */
+#ifdef CONFIG_FIT_PARTITION
+ if (!efi_guidcmp(ptes[i].partition_type_guid, PARTITION_LINUX_FIT_GUID))
+ (void) parse_fit_partitions(state, start * ssz, size * ssz,
+ &extra_slot, 127, 1);
+#endif
}
kfree(ptes);
kfree(gpt);
diff --git a/block/partitions/efi.h b/block/partitions/efi.h
index 84b9f36b9e47..06c11f6ae398 100644
--- a/block/partitions/efi.h
+++ b/block/partitions/efi.h
@@ -51,6 +51,9 @@
#define PARTITION_LINUX_LVM_GUID \
EFI_GUID( 0xe6d6d379, 0xf507, 0x44c2, \
0xa2, 0x3c, 0x23, 0x8f, 0x2a, 0x3d, 0xf9, 0x28)
+#define PARTITION_LINUX_FIT_GUID \
+ EFI_GUID( 0xcae9be83, 0xb15f, 0x49cc, \
+ 0x86, 0x3f, 0x08, 0x1b, 0x74, 0x4a, 0x2d, 0x93)
typedef struct _gpt_header {
__le64 signature;
--
2.38.1
On Tue, Nov 15, 2022 at 09:47:06PM +0000, Daniel Golle wrote:
> Add new GUID allowing to parse uImage.FIT stored in a GPT partition
> and map filesystem sub-image as sub-partitions.
NAK, we should not go out from the partition code to parse random
weird image formats. If you want to support uImage.FIT just write
a tinty stackable block driver or dm table for it.
Hi Christoph,
On Tue, Nov 15, 2022 at 10:01:05PM -0800, Christoph Hellwig wrote:
> On Tue, Nov 15, 2022 at 09:47:06PM +0000, Daniel Golle wrote:
> > Add new GUID allowing to parse uImage.FIT stored in a GPT partition
> > and map filesystem sub-image as sub-partitions.
>
> NAK, we should not go out from the partition code to parse random
> weird image formats.
While weirdness is certainly subjective, uImage.FIT is not just a
random image format but used by a great majority of headless embedded
Linux devices out there. It's the default image format of many of the
SDKs distributed by chip vendors such as Allwinner, Marvell, MediaTek,
NXP, Qualcomm/Atheros, ...
Having better support for it in Linux hence doesn't seem too far-fetched
to me, especially given that we got partition parsers for all sorts of
historic (Acorn, Amiga, Atari, ...) or actually exotic (Karma?) formats.
Even Microsoft Windows' Logical Disk Manager is supported natively by
the kernel...
> If you want to support uImage.FIT just write a tiny stackable block
> driver or dm table for it.
As this is used on rather tiny embedded devices my hope was to keep
things simple and not having to enable device mapper on systems which
have anyway only very small amounts of storage and won't ever need
most of the device mapper features.
Using a tiny block driver instead is an option, I've implemented this
approach in the past couple of hours and it works just as fine.
In this case I would introduce a new kernel cmdline option allowing
to specify which block device (ie. a partition on eMMC, or mtdblock
or ubiblock device) to launch the uImage.FIT parser on.
Allowing this new driver to add block partitions by exporting a new
helper functions for that in block/partition/core.c would greatly
simplify things, as then the existing partitioning code could still
be used (instead of basically having to re-implement loopdev and
introduce a whole new type of block devices).
I will post an RFC series illustrating this approach.
Please let me know if this sounds acceptable, so I won't put effort
into implementing something which will then be rejected again after 5
iterations on the mailing list for reasons which could have been
expressed from the beginning. An RFC for this series was posted on
2022-04-25 [1], I wouldn't have worked months to fix all requests of
other maintainers and tested it on a variety of different hardware
knowing that the whole approach will be NACK'ed...
And, of course, thank you anyway for reviewing!
Cheers
Daniel
[1]: https://patchwork.kernel.org/project/linux-block/list/?series=635369&state=*
On Thu, Nov 17, 2022 at 12:19:10AM +0000, Daniel Golle wrote:
> While weirdness is certainly subjective, uImage.FIT is not just a
> random image format but used by a great majority of headless embedded
> Linux devices out there. It's the default image format of many of the
> SDKs distributed by chip vendors such as Allwinner, Marvell, MediaTek,
> NXP, Qualcomm/Atheros, ...
"Look see, my weird format is used by all these companies building
crappy SOCs, it is not weird.."
> Please let me know if this sounds acceptable, so I won't put effort
> into implementing something which will then be rejected again after 5
> iterations on the mailing list for reasons which could have been
> expressed from the beginning. An RFC for this series was posted on
> 2022-04-25 [1], I wouldn't have worked months to fix all requests of
> other maintainers and tested it on a variety of different hardware
> knowing that the whole approach will be NACK'ed...
If people ignore something that is obviously broken they might just hope
for it to go away, becaue often it does.
----- Ursprüngliche Mail -----
> Von: "Christoph Hellwig" <[email protected]>
> On Thu, Nov 17, 2022 at 12:19:10AM +0000, Daniel Golle wrote:
>> While weirdness is certainly subjective, uImage.FIT is not just a
>> random image format but used by a great majority of headless embedded
>> Linux devices out there. It's the default image format of many of the
>> SDKs distributed by chip vendors such as Allwinner, Marvell, MediaTek,
>> NXP, Qualcomm/Atheros, ...
>
> "Look see, my weird format is used by all these companies building
> crappy SOCs, it is not weird.."
Well, FIT is not something strange invented by SoC companies, it comes from u-boot
and is more or less a de-facto standard.
While I agree that using the block layer for partition parsing is questionable
I think supporting these images in Linux is a worthwhile goal.
Thanks,
//richard
On Thu, Nov 17, 2022 at 07:50:08AM +0100, Richard Weinberger wrote:
> I think supporting these images in Linux is a worthwhile goal.
I never argued against that. But it is not a fit for partitions.
So write a proper stacked block driver or dm driver for it if you
care enough. The format is a complete mess and should be isolated
to not affect the rest of the kernel.
On Wed, Nov 16, 2022 at 10:00:25PM -0800, Christoph Hellwig wrote:
> On Thu, Nov 17, 2022 at 12:19:10AM +0000, Daniel Golle wrote:
> > While weirdness is certainly subjective, uImage.FIT is not just a
> > random image format but used by a great majority of headless embedded
> > Linux devices out there. It's the default image format of many of the
> > SDKs distributed by chip vendors such as Allwinner, Marvell, MediaTek,
> > NXP, Qualcomm/Atheros, ...
>
> "Look see, my weird format is used by all these companies building
> crappy SOCs, it is not weird.."
I didn't invent this, and it's just as broken and yet perdominant as,
let's say, MS LDM on x86.
>
> > Please let me know if this sounds acceptable, so I won't put effort
> > into implementing something which will then be rejected again after 5
> > iterations on the mailing list for reasons which could have been
> > expressed from the beginning. An RFC for this series was posted on
> > 2022-04-25 [1], I wouldn't have worked months to fix all requests of
> > other maintainers and tested it on a variety of different hardware
> > knowing that the whole approach will be NACK'ed...
>
> If people ignore something that is obviously broken they might just hope
> for it to go away, becaue often it does.
While I'm sure that strategy works seen from your perspective, it does
waste resources on the other end. In this case it might not have been
obvious to everybody, I did receive feedback from other maintainers,
as I said. It's not that everybody ignored this contribution. Hence,
looking at it from my end, the picture is a bit different. Anyway.
I would have appreciated an earlier explicite NACK, that's all I
wanted to say.