Received-SPF: pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <CAKbZUD01uR5kfP4=SSfQ111jKsfKi8ojfDZs5CStLD_h5qb5GQ@mail.gmail.com>
 <20230628045206.GA1908@sol.localdomain> <20230628185832.GK11467@frogsfrogsfrogs>
 <20230703194850.GF1194@sol.localdomain>
In-Reply-To: <20230703194850.GF1194@sol.localdomain>
From:   Pedro Falcato <pedro.falcato@gmail.com>
Date:   Fri, 4 Aug 2023 22:08:13 +0100
Message-ID: <CAKbZUD0ModY66yJ+ieAs=XYecMHc-X2=Sanxcv8wmSf0T=BSzQ@mail.gmail.com>
Subject: Re: Question regarding the use of CRC32c for checksumming
To:     Eric Biggers <ebiggers@kernel.org>
Cc:     "Darrick J. Wong" <djwong@kernel.org>, linux-ext4@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Mon, Jul 3, 2023 at 8:48=E2=80=AFPM Eric Biggers <ebiggers@kernel.org> w=
rote:
>

Hi folks, really sorry for the big delay, this thread really slipped my min=
d :)

> IMO the best API for CRC's is like zlib's where you pass in 0 to start th=
e CRC
> and it does both the pre and post inversions for you.  Note, "updates" st=
ill
> work as expected, since two inversions cancel each other out.

I agree, I did that when adding CRC32c to EFI. u32
calculate_crc32c(const void *buf, size_t len, u32 initial) with
inversions on initial and the result is pretty simple and effective.

> Unfortunately, many but not all of the CRC APIs in Linux decided to go wi=
th the
> other convention, which is to leave the inversions entirely to the caller=
.
>
> I think the kernel should also make the architecture-specific CRC
> implementations accessible directly via a library API, similar to what's =
done
> for Blake2s and ChaCha20.  There should be no need to go through shash at=
 all...
>
> >
> > This misuse could be fixed, but you'd have to burn an incompat flag to
> > do it.  I'm less smart about crc32* than I was back in 2008, so I also
> > don't have the skills to figure out if the correction is worth the cost=
.
> >
> > --D
>
> No, it's not worth changing the ext4 on-disk format for this.

I don't think we'd need to change the on-disk format for this? Or for
any other hash algorithm change (as long as the resulting digest is
32-bit), right? Given we have s_checksum_type.
Or do existing tools dangerously assume CRC32c at the moment?

In any case, thank you both for the background on this, I'll try to
submit a patch to the docs to clarify this point.

--=20
Pedro