Received: by 2002:a05:6358:3188:b0:123:57c1:9b43 with SMTP id q8csp16666271rwd; Mon, 26 Jun 2023 13:19:59 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ5WZ8x+T28jWDCp6ejaUsMYj+nWCuDEXFUwLF8BD1y79k6OOMv/d92yDV6sIQ11c5InugPX X-Received: by 2002:aa7:d7d1:0:b0:51d:8955:9c8f with SMTP id e17-20020aa7d7d1000000b0051d89559c8fmr5117677eds.29.1687810799040; Mon, 26 Jun 2023 13:19:59 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1687810799; cv=none; d=google.com; s=arc-20160816; b=FaejiO/7AzuiDrcwp2Bt0WD61CWbfSQwIDF5zrnw8m88I8r9jk3DFt+4rHBCHeDMvi ZygJsNI0EdfGQG4NzhujsLO8aRxs23o/QF9evJd0JL9XC5jSFVtZf94adw0k3v92dXwI Sa3iaL8g6gdK7c5vwAqnIz/T80dI/kuJVwtHMO2x1X4MxsTtxOCG0NxhDmsvRXdzbtxT iMixjCbwtyhZLk2tlaZ9NB4/9pm/QK+tdUOSSq+t76zanvC8AKCNSmbfilMyBGl2AwMp YseplCRuFBBhm30UJuln0eF74VxPS6gIJM0nQfK+se1CCJ50qea4ElsI1b5nZ7f8Gblm 22qA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:to:subject:message-id:date:from:mime-version :dkim-signature; bh=SKcrdBSkM8MYRQqsDhtbvkGn1p7TbA8H2miaLprsg8c=; fh=p7ag6FEVq6O6SO6wnQipX2sl53SHiwDhr5emJ7pPi9U=; b=acZ62GphuI2Mn1e/YiYw8XdxN8NQvteYBgg8c/CYfC6eVBp2bGKKP7CVMigUesScH6 ORief8iSc0lVKj2NuW7mbwyNiqnv26UJimIf6bwEU0acA3wZqFHbgU7qo81xH5I/xMD5 u8RPwczGAaTHi+8vxnKPQPyotWki0if200E4yp5v8q3dBXQx82am3fO6sNqlrgIrQcNz krrmHIpotKwELimWMe5DwvQErMkZbodOeahICedrbzMQ6sJumBkd2Ct4Gt8Cby4DDbTI I34VcuXsqxtWcNghmqNMaCtONDKIe5BdtVx3YozwqXOqrA4GvtRiRZrDAFzclLudi3ad fNhA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=NTUKf8AG; spf=pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-ext4-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bo8-20020a0564020b2800b00514be34853csi3128963edb.214.2023.06.26.13.19.31; Mon, 26 Jun 2023 13:19:59 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20221208 header.b=NTUKf8AG; spf=pass (google.com: domain of linux-ext4-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-ext4-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229621AbjFZUR4 (ORCPT + 99 others); Mon, 26 Jun 2023 16:17:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:60524 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229628AbjFZURZ (ORCPT ); Mon, 26 Jun 2023 16:17:25 -0400 Received: from mail-vk1-xa32.google.com (mail-vk1-xa32.google.com [IPv6:2607:f8b0:4864:20::a32]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0AA66E4 for ; Mon, 26 Jun 2023 13:17:24 -0700 (PDT) Received: by mail-vk1-xa32.google.com with SMTP id 71dfb90a1353d-471b3ad20e1so1041811e0c.1 for ; Mon, 26 Jun 2023 13:17:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20221208; t=1687810642; x=1690402642; h=to:subject:message-id:date:from:mime-version:from:to:cc:subject :date:message-id:reply-to; bh=SKcrdBSkM8MYRQqsDhtbvkGn1p7TbA8H2miaLprsg8c=; b=NTUKf8AGs7KNMj9Id2Xy93Jh1RSlOOw7pRqIt9U5IwDZlnKv3pDykAhPlShCbqonLO 9Bjv5WnfwxREZa0Nh5paXYALanu4HCP8vU7E8k/Acjc/8p8djqg6+WnlvNvOsjdqntj1 vik0l2kySHmbzgWgeSAuje9iEqImTNQAYR6DWQaMfMpQTrxz3PW82iaF8IokcBWPj8UL ZyflohKdu7bE+jzPM5XmZPuiKuMwEC6Fo6L2SqhbWp68E0zq7Jmn08FuE+GVscbcEYQl SN86RmQc5y8+FVRRi0rs0F9YtyAnt23cHmJybCO7B7Rdk4JvMYqfl913nf4GZ04ekNKr Q8tQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20221208; t=1687810642; x=1690402642; h=to:subject:message-id:date:from:mime-version:x-gm-message-state :from:to:cc:subject:date:message-id:reply-to; bh=SKcrdBSkM8MYRQqsDhtbvkGn1p7TbA8H2miaLprsg8c=; b=dydzGGBSNxK6BhX771fEhtBqGS2TcGfBTe42eeW8fa8LocWCUHyImqQT1Xvz2NhUyP tfe4FtPav3+hr1tk1KOK/IiMbGpVrMKRiP5j3FZ8BPJt/ke0WhEwjb5D1pjeTH2dps2K aYkW49DF7mldp+EhuIOzYNlJ7hEJWlASTAXDsRHzPrt241+NiOqSs38a1Xr1THDq3i9E 8ZUTyj50VgtO+PkdnIbWqcaWJa2wm5b0AXahimikwwK49ZOOl65eNB8Kf+qCob7NVwHO ZO4VzfoV01IWbpLtQ7fG4LNTe+uYBc9HefFptDStLAguidT1CpVuP2icS2//yISAEMb9 w5bw== X-Gm-Message-State: AC+VfDwU1v+c5zsUwb62QBD6syZd5Xmb78P+Tl++Sj0DsMCdj3k7qfxW nfn8rU6MvT1NOTXciHTW0UPBNlDBsQaiKOAkoPJbA/y97bQ= X-Received: by 2002:a1f:d286:0:b0:471:be92:daa9 with SMTP id j128-20020a1fd286000000b00471be92daa9mr9226074vkg.13.1687810642068; Mon, 26 Jun 2023 13:17:22 -0700 (PDT) MIME-Version: 1.0 From: Pedro Falcato Date: Mon, 26 Jun 2023 21:17:10 +0100 Message-ID: Subject: Question regarding the use of CRC32c for checksumming To: linux-ext4@vger.kernel.org, "Darrick J. Wong" Content-Type: text/plain; charset="UTF-8" X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM, RCVD_IN_DNSWL_NONE,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-ext4@vger.kernel.org Hi, (+CC the original author, Darrick) I've been investigating (in the context of my EFI ext4 driver) why all ext4 checksums appear inverted. After making sure my CRC32c implementation was correct and up-to-par with other ones, I looked at the fs/ext4 checksumming code, which took me to the implementation of ext4_chksum in ext4.h (excuse the gmail whitespace damage): >static inline u32 ext4_chksum(struct ext4_sb_info *sbi, u32 crc, > const void *address, unsigned int length) >{ > struct { > struct shash_desc shash; > char ctx[4]; > } desc; Open coding the crc32c crypto driver's internal state, seemingly to save a call? > > BUG_ON(crypto_shash_descsize(sbi->s_chksum_driver)!=sizeof(desc.ctx)); > > desc.shash.tfm = sbi->s_chksum_driver; > *(u32 *)desc.ctx = crc; ...we set the starting CRC > > BUG_ON(crypto_shash_update(&desc.shash, address, length)); then call update, which keeps the current internal state in ctx[4] > > return *(u32 *)desc.ctx; and then we never call ->final() (nor ->finup()), which for crc32c would do: > put_unaligned_le32(~ctx->crc, out); and as such get me the properly "inverted" crc32c I would expect. FreeBSD never found this issue as their calculate_crc32c seems borked too, and never inverts the result. Is my assessment correct? Was ->final() never called on purpose, or is it an accident? Or is this merely a CRC32c variation I'm unaware of? I'd like to make sure I get all the context on this, before sending any kind of documentation patch :) Thanks, Pedro