LinuxLists.cc - [PATCH v5 0/2] ubi: add possibility to skip CRC check for static UBI volumes

2018-07-02 09:57:05

Subject: [PATCH v5 0/2] ubi: add possibility to skip CRC check for static UBI volumes

2018-07-02 09:56:40

Subject: [PATCH v5 1/2] ubi: provide a way to skip CRC checks

Some users of static UBI volumes implement their own integrity check,
thus making the volume CRC check done at open time useless. For
instance, this is the case when one use the ubiblock + dm-verity +
squashfs combination, where dm-verity already checks integrity of the
block device but this time at the block granularity instead of verifying
the whole volume.

Skipping this test drastically improves the boot-time.

Suggested-by: Boris Brezillon <[email protected]>
Signed-off-by: Quentin Schulz <[email protected]>
Reviewed-by: Boris Brezillon <[email protected]>
Reviewed-by: Richard Weinberger <[email protected]>
---
drivers/mtd/ubi/kapi.c | 2 +-
drivers/mtd/ubi/ubi-media.h | 6 ++++++
drivers/mtd/ubi/ubi.h | 4 ++++
drivers/mtd/ubi/vmt.c | 9 +++++++++
drivers/mtd/ubi/vtbl.c | 3 +++
5 files changed, 23 insertions(+), 1 deletion(-)

diff --git a/drivers/mtd/ubi/kapi.c b/drivers/mtd/ubi/kapi.c
index d4b2e87..e9e9ecb 100644
--- a/drivers/mtd/ubi/kapi.c
+++ b/drivers/mtd/ubi/kapi.c
@@ -202,7 +202,7 @@ struct ubi_volume_desc *ubi_open_volume(int ubi_num, int vol_id, int mode)
desc->mode = mode;

mutex_lock(&ubi->ckvol_mutex);
- if (!vol->checked) {
+ if (!vol->checked && !vol->skip_check) {
/* This is the first open - check the volume */
err = ubi_check_volume(ubi, vol_id);
if (err < 0) {
diff --git a/drivers/mtd/ubi/ubi-media.h b/drivers/mtd/ubi/ubi-media.h
index 195ff8c..b5fe8f8 100644
--- a/drivers/mtd/ubi/ubi-media.h
+++ b/drivers/mtd/ubi/ubi-media.h
@@ -45,6 +45,11 @@ enum {
* Volume flags used in the volume table record.
*
* @UBI_VTBL_AUTORESIZE_FLG: auto-resize this volume
+ * @UBI_VTBL_SKIP_CRC_CHECK_FLG: skip the CRC check done on a static volume at
+ * open time. Should only be set on volumes that
+ * are used by upper layers doing this kind of
+ * check. Main use-case for this flag is
+ * boot-time reduction
*
* %UBI_VTBL_AUTORESIZE_FLG flag can be set only for one volume in the volume
* table. UBI automatically re-sizes the volume which has this flag and makes
@@ -76,6 +81,7 @@ enum {
*/
enum {
UBI_VTBL_AUTORESIZE_FLG = 0x01,
+ UBI_VTBL_SKIP_CRC_CHECK_FLG = 0x02,
};

/*
diff --git a/drivers/mtd/ubi/ubi.h b/drivers/mtd/ubi/ubi.h
index f5ba97c..d47b9e4 100644
--- a/drivers/mtd/ubi/ubi.h
+++ b/drivers/mtd/ubi/ubi.h
@@ -327,6 +327,9 @@ struct ubi_eba_leb_desc {
* atomic LEB change
*
* @eba_tbl: EBA table of this volume (LEB->PEB mapping)
+ * @skip_check: %1 if CRC check of this static volume should be skipped.
+ * Directly reflects the presence of the
+ * %UBI_VTBL_SKIP_CRC_CHECK_FLG flag in the vtbl entry
* @checked: %1 if this static volume was checked
* @corrupted: %1 if the volume is corrupted (static volumes only)
* @upd_marker: %1 if the update marker is set for this volume
@@ -374,6 +377,7 @@ struct ubi_volume {
void *upd_buf;

struct ubi_eba_table *eba_tbl;
+ unsigned int skip_check:1;
unsigned int checked:1;
unsigned int corrupted:1;
unsigned int upd_marker:1;
diff --git a/drivers/mtd/ubi/vmt.c b/drivers/mtd/ubi/vmt.c
index 0be5167..e2606a4 100644
--- a/drivers/mtd/ubi/vmt.c
+++ b/drivers/mtd/ubi/vmt.c
@@ -299,6 +299,10 @@ int ubi_create_volume(struct ubi_device *ubi, struct ubi_mkvol_req *req)
vtbl_rec.vol_type = UBI_VID_DYNAMIC;
else
vtbl_rec.vol_type = UBI_VID_STATIC;
+
+ if (vol->skip_check)
+ vtbl_rec.flags |= UBI_VTBL_SKIP_CRC_CHECK_FLG;
+
memcpy(vtbl_rec.name, vol->name, vol->name_len);

err = ubi_change_vtbl_record(ubi, vol_id, &vtbl_rec);
@@ -733,6 +737,11 @@ static int self_check_volume(struct ubi_device *ubi, int vol_id)
ubi_err(ubi, "bad used_bytes");
goto fail;
}
+
+ if (vol->skip_check) {
+ ubi_err(ubi, "bad skip_check");
+ goto fail;
+ }
} else {
if (vol->used_ebs < 0 || vol->used_ebs > vol->reserved_pebs) {
ubi_err(ubi, "bad used_ebs");
diff --git a/drivers/mtd/ubi/vtbl.c b/drivers/mtd/ubi/vtbl.c
index 94d7a86..2c133cd 100644
--- a/drivers/mtd/ubi/vtbl.c
+++ b/drivers/mtd/ubi/vtbl.c
@@ -560,6 +560,9 @@ static int init_volumes(struct ubi_device *ubi,
vol->name[vol->name_len] = '\0';
vol->vol_id = i;

+ if (vtbl[i].flags & UBI_VTBL_SKIP_CRC_CHECK_FLG)
+ vol->skip_check = 1;
+
if (vtbl[i].flags & UBI_VTBL_AUTORESIZE_FLG) {
/* Auto re-size flag may be set only for one volume */
if (ubi->autoresize_vol_id != -1) {
--
git-series 0.9.1

2018-07-02 09:57:55

by Quentin Schulz

[permalink] [raw]

Subject: [PATCH v5 2/2] ubi: expose the volume CRC check skip flag

Now that we have the logic for skipping CRC check for static UBI volumes
in the core, let's expose it to users.

This makes use of a padding byte in the volume description data
structure as a flag. This flag only tell for now whether we should skip
the CRC check of a volume.

This checks the UBI volume for which we are trying to skip the CRC check
is static.

Let's also make sure that the flags passed to verify_mkvol_req are
valid.

We voluntarily do not take into account the skip_check flag in
vol_cdev_write() as we want to make sure what we wrote was correctly
written.

Suggested-by: Boris Brezillon <[email protected]>
Signed-off-by: Quentin Schulz <[email protected]>
Reviewed-by: Boris Brezillon <[email protected]>
---
drivers/mtd/ubi/cdev.c | 11 +++++++++++
drivers/mtd/ubi/vmt.c | 3 +++
include/uapi/mtd/ubi-user.h | 18 ++++++++++++++++--
3 files changed, 30 insertions(+), 2 deletions(-)

diff --git a/drivers/mtd/ubi/cdev.c b/drivers/mtd/ubi/cdev.c
index 45c3296..22547d7 100644
--- a/drivers/mtd/ubi/cdev.c
+++ b/drivers/mtd/ubi/cdev.c
@@ -367,6 +367,10 @@ static ssize_t vol_cdev_write(struct file *file, const char __user *buf,
return count;
}

+ /*
+ * We voluntarily do not take into account the skip_check flag
+ * as we want to make sure what we wrote was correctly written.
+ */
err = ubi_check_volume(ubi, vol->vol_id);
if (err < 0)
return err;
@@ -622,6 +626,13 @@ static int verify_mkvol_req(const struct ubi_device *ubi,
req->vol_type != UBI_STATIC_VOLUME)
goto bad;

+ if (req->flags & ~UBI_VOL_VALID_FLGS)
+ goto bad;
+
+ if (req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG &&
+ req->vol_type != UBI_STATIC_VOLUME)
+ goto bad;
+
if (req->alignment > ubi->leb_size)
goto bad;

diff --git a/drivers/mtd/ubi/vmt.c b/drivers/mtd/ubi/vmt.c
index e2606a4..729588b 100644
--- a/drivers/mtd/ubi/vmt.c
+++ b/drivers/mtd/ubi/vmt.c
@@ -174,6 +174,9 @@ int ubi_create_volume(struct ubi_device *ubi, struct ubi_mkvol_req *req)
vol->dev.class = &ubi_class;
vol->dev.groups = volume_dev_groups;

+ if (req->flags & UBI_VOL_SKIP_CRC_CHECK_FLG)
+ vol->skip_check = 1;
+
spin_lock(&ubi->volumes_lock);
if (vol_id == UBI_VOL_NUM_AUTO) {
/* Find unused volume ID */
diff --git a/include/uapi/mtd/ubi-user.h b/include/uapi/mtd/ubi-user.h
index 5b04a49..aad3b62 100644
--- a/include/uapi/mtd/ubi-user.h
+++ b/include/uapi/mtd/ubi-user.h
@@ -285,6 +285,20 @@ struct ubi_attach_req {
__s8 padding[10];
};

+/*
+ * UBI volume flags.
+ *
+ * @UBI_VOL_SKIP_CRC_CHECK_FLG: skip the CRC check done on a static volume at
+ * open time. Only valid for static volumes and
+ * should only be used if the volume user has a
+ * way to verify data integrity
+ */
+enum {
+ UBI_VOL_SKIP_CRC_CHECK_FLG = 0x1,
+};
+
+#define UBI_VOL_VALID_FLGS (UBI_VOL_SKIP_CRC_CHECK_FLG)
+
/**
* struct ubi_mkvol_req - volume description data structure used in
* volume creation requests.
@@ -292,7 +306,7 @@ struct ubi_attach_req {
* @alignment: volume alignment
* @bytes: volume size in bytes
* @vol_type: volume type (%UBI_DYNAMIC_VOLUME or %UBI_STATIC_VOLUME)
- * @padding1: reserved for future, not used, has to be zeroed
+ * @flags: volume flags (%UBI_VOL_SKIP_CRC_CHECK_FLG)
* @name_len: volume name length
* @padding2: reserved for future, not used, has to be zeroed
* @name: volume name
@@ -321,7 +335,7 @@ struct ubi_mkvol_req {
__s32 alignment;
__s64 bytes;
__s8 vol_type;
- __s8 padding1;
+ __u8 flags;
__s16 name_len;
__s8 padding2[4];
char name[UBI_MAX_VOLUME_NAME + 1];
--
git-series 0.9.1

2018-07-02 13:51:14

by Artem Bityutskiy

[permalink] [raw]

Subject: Re: [PATCH v5 0/2] ubi: add possibility to skip CRC check for static UBI volumes

On Mon, 2018-07-02 at 11:43 +0200, Quentin Schulz wrote:
> Some users of static UBI volumes implement their own integrity check, thus
> making the volume CRC check done at open time useless. For instance, this
> is the case when one use the ubiblock + dm-verity + squashfs combination,
> where dm-verity already checks integrity of the block device but this time
> at the block granularity instead of verifying the whole volume.
>
> Skipping this test drastically improves the boot-time.
>
> mtd-utils patches are already merged. See:
> http://git.infradead.org/mtd-utils.git/commit/9095d213f536aec0f3c37f6666177f3b907afde7
> http://git.infradead.org/mtd-utils.git/commit/7b4a65a27d2621b58c634d02c6a068ed9562383c
> http://git.infradead.org/mtd-utils.git/commit/5e9bc0daa41d84ce5de81c4a1665d65f51893c10
> http://git.infradead.org/mtd-utils.git/commit/8ba21ab75b41a1f9a6e27eed3ea80c9829669c5a

For the whole patch-set,

Reviewed-by: Artem Bityutskiy <[email protected]>