Received-SPF: pass (google.com: domain of linux-ext4+bounces-2297-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="===============8318157665090275533=="
Precedence: bulk
MIME-Version: 1.0
Content-Disposition: inline
In-Reply-To: <171483317081.2626447.5951155062757257572@localhost>
References: <171483317081.2626447.5951155062757257572@localhost>
Subject: Re: created ext4 disk image differs depending on the underlying filesystem
From: Johannes Schauer Marin Rodrigues <josch@mister-muffin.de>
To: linux-ext4@vger.kernel.org
Date: Sat, 04 May 2024 19:53:29 +0200
Message-ID: <171484520952.2626447.2160419274451668597@localhost>
User-Agent: alot/0.10

--===============8318157665090275533==
Content-Type: text/plain; charset="utf-8"
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable

Quoting Johannes Schauer Marin Rodrigues (2024-05-04 16:32:50)
> I originally observed this issue when creating ext4 disk images on a 9p
> filesystem which differed from the images I created on a tmpfs. I observed
> that the difference also exists when the underlying file system is fat32,=
 so
> I'm using this as an example here. For what it's worth, the ext4 filesyst=
em
> images created on a tmpfs are identical to those created on an ext4 fs. To
> demonstrate the issue, please see the script at the end of this mail (it
> requires sudo to mount and unmount the fat32 disk image). As you can see =
from
> the printed hashes, the disk images produced outside the fat32 disk are
> always identical as expected. The diff between the reproducible images and
> those stored on fat32 is also very short but I don't know what data is st=
ored
> at those points:
>=20
> @@ -85,7 +85,7 @@
>  00000540: 0000 0000 0000 0000 0000 0000 0000 1000  ................
>  00000550: 0000 0000 0000 0000 0000 0000 2000 2000  ............ . .
>  00000560: 0200 0000 0000 0000 0000 0000 0000 0000  ................
> -00000570: 0000 0000 0401 0000 8c04 0000 0000 0000  ................
> +00000570: 0000 0000 0401 0000 4900 0000 0000 0000  ........I.......
>  00000580: 0000 0000 0000 0000 0000 0000 0000 0000  ................
>  00000590: 0000 0000 0000 0000 0000 0000 0000 0000  ................
>  000005a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
> @@ -125,9 +125,9 @@
>  000007c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
>  000007d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
>  000007e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
> -000007f0: 0000 0000 0000 0000 0000 0000 264c 0251  ............&L.Q
> +000007f0: 0000 0000 0000 0000 0000 0000 64ca bba5  ............d...
>  00000800: 1200 0000 2200 0000 3200 0000 9d03 7300  ...."...2.....s.
> -00000810: 0200 0000 0000 0000 babb 8a41 7300 2004  ...........As. .
> +00000810: 0200 0400 0000 0000 babb 8a41 7300 69f5  ...........As.i.
>  00000820: 0000 0000 0000 0000 0000 0000 0000 0000  ................
>  00000830: 0000 0000 0000 0000 bc7a 6e31 0000 0000  .........zn1....
>  00000840: 0000 0000 0000 0000 0000 0000 0000 0000  ................
>=20
> Any idea what is going on? Is there a better way to diff two ext4 disk im=
ages
> than diffing the xxd output? If I try diffing the dumpe2fs output I get t=
hese
> differences:
>=20
> @@ -32,7 +32,7 @@
>  Maximum mount count:      -1
>  Last checked:             Fri May  3 16:14:49 2024
>  Check interval:           0 (<none>)
> -Lifetime writes:          1164 kB
> +Lifetime writes:          73 kB
>  Reserved blocks uid:      0 (user root)
>  Reserved blocks gid:      0 (group root)
>  First inode:              11
> @@ -44,7 +44,7 @@
>  Directory Hash Seed:      0b7f9cfd-0113-486c-a453-4f5483bd486b
>  Journal backup:           inode blocks
>  Checksum type:            crc32c
> -Checksum:                 0x51024c26
> +Checksum:                 0xa5bbca64
>  Checksum seed:            0xf81d767d
>  Orphan file inode:        12
>  Journal features:         (none)
> @@ -56,7 +56,7 @@
>  Journal start:            0
> =20
> =20
> -Group 0: (Blocks 1-2047) csum 0x0420
> +Group 0: (Blocks 1-2047) csum 0xf569 [ITABLE_ZEROED]
>    Primary superblock at 1, Group descriptors at 2-2
>    Reserved GDT blocks at 3-17
>    Block bitmap at 18 (+17), csum 0x7abcbbba
>=20
> Why would these bits differ depending on the filesystem on which the disk=
 image
> is stored? Is there a way to equalize this information so that the disk i=
mage
> looks the same independent on the underlying filesystem?

The diff becomes a bit smaller when using
-E lazy_itable_init=3D0,assume_storage_prezeroed=3D0,nodiscard

@@ -85,7 +85,7 @@
 00000540: 0000 0000 0000 0000 0000 0000 0000 1000  ................
 00000550: 0000 0000 0000 0000 0000 0000 2000 2000  ............ . .
 00000560: 0200 0000 0000 0000 0000 0000 0000 0000  ................
-00000570: 0000 0000 0401 0000 ac04 0000 0000 0000  ................
+00000570: 0000 0000 0401 0000 4900 0000 0000 0000  ........I.......
 00000580: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000590: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000005a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
@@ -125,7 +125,7 @@
 000007c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000007d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000007e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
-000007f0: 0000 0000 0000 0000 0000 0000 fb8d a90f  ................
+000007f0: 0000 0000 0000 0000 0000 0000 64ca bba5  ............d...
 00000800: 1200 0000 2200 0000 3200 0000 9d03 7300  ...."...2.....s.
 00000810: 0200 0400 0000 0000 babb 8a41 7300 69f5  ...........As.i.
 00000820: 0000 0000 0000 0000 0000 0000 0000 0000  ................

@@ -32,7 +32,7 @@
 Maximum mount count:      -1
 Last checked:             Fri May  3 16:14:49 2024
 Check interval:           0 (<none>)
-Lifetime writes:          1196 kB
+Lifetime writes:          73 kB
 Reserved blocks uid:      0 (user root)
 Reserved blocks gid:      0 (group root)
 First inode:              11
@@ -44,7 +44,7 @@
 Directory Hash Seed:      0b7f9cfd-0113-486c-a453-4f5483bd486b
 Journal backup:           inode blocks
 Checksum type:            crc32c
-Checksum:                 0x0fa98dfb
+Checksum:                 0xa5bbca64
 Checksum seed:            0xf81d767d
 Orphan file inode:        12
 Journal features:         (none)


The "Lifetime writes" being much higher on fat32 suggests that despite
"nodiscard", less zeroes were written out when ext4 or tmpfs are the underl=
ying
FS?

Thanks!

cheers, josch
--===============8318157665090275533==
MIME-Version: 1.0
Content-Transfer-Encoding: 7bit
Content-Description: signature
Content-Type: application/pgp-signature; name="signature.asc"; charset="us-ascii"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEElFhU6KL81LF4wVq58sulx4+9g+EFAmY2dhYACgkQ8sulx4+9
g+HkOBAAjgHV4v0L4BXR03VadrNQJw8welBnJfWNrpASPF9eXMA8zCGbD2lIC3tk
aVHWyk/AoYOSvZp7ED74pr8W8SSTmBNrqdEwsJLg18b2Ix09X+07j8nW5pjw56B5
6ERMQjl07U21PHkV7SJE9R6ZGXHhI6grxf1wy5vDRjRP4l7R6wVyxfFuN4VLk4FD
Ur3bEUQaqYj620g/jeZwJTKPjtCYEYeEMtAZuKq8ZHVyZcjZpwbjkihVwBkr5Mjj
SFp1H7/RsvbL08u3LWUD54N2qrt21TMuY1NMtdPUVCmbv29xRnKu5jyWjBNRPfHn
Cs+aaHjEC6RUA6AfwPRC07DGQCnlv1PVZeiHFP21OTpv7d0I746pS3qSeS6OlQpa
7EBwUNboZZECJhiRhxumaZ1BfLE63zNUQruAgL6R/HFN1Xy0CYRbOu3zOhizGxJ6
GmiyNpB/19dfr9L+M7gSACchOONXtFtH9JnM3w0Ef8MuFg1Y0Io+vuwBNswGUsoU
YodRqeUqHKUxE0iyd90k3oXoKK9BrRVWLO8WrbkpCDQT/LH486f0d5C0Aus+biwE
5gOZwydl11M4lyZsAdcoORlPK/rvxaGoC/uKyD3Sl2y6wgpvbXNyoC6w+M1szulf
iFbg2RXhHJdRJReC/sFg4vaSSazGCZxHcU3X7gvlLccjBtpEFmY=
=rVim
-----END PGP SIGNATURE-----

--===============8318157665090275533==--