Received: by 2002:a05:6358:3188:b0:123:57c1:9b43 with SMTP id q8csp22469709rwd; Fri, 30 Jun 2023 08:22:30 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4vAgcyCThbWI9cXr1mADGo4gLMNX6JCFsRg+YykGpbAIujNNzHWkgfY1q5/C5sZCeJHxv5 X-Received: by 2002:a05:6a21:6d82:b0:11f:985e:ae2c with SMTP id wl2-20020a056a216d8200b0011f985eae2cmr4191344pzb.3.1688138550598; Fri, 30 Jun 2023 08:22:30 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1688138550; cv=none; d=google.com; s=arc-20160816; b=C9aG9Y1inaF/4YyfoaI+LbMM4yeVg1ZIJ0nLreTUSUb/HFT1Zbd742S5TbdpISKWvw AovipXBpeJt7YOw6JBNX5P1vdA0Rvm1y9wJasOBp+8PkChalTtgQvWtOPre8MF7LffmJ ilZ0xeYi+PkKcE+RhwuP8tYw89zBxZ7o98IHvBdEhJPHg3oz2atCTuQJl1qESujrDLM5 ECKyTdmu8xyfRTbuLtoQ97qdL5uOmJezcQqYq1dUHAPrl60kgSqV+gTpf+diVYVN23WG PBziHeeRfZAEvPGFx43ZxFhpS9si5j6HFRagskVTS/sJsw73u+/oSjYfzTYgssAHOyHw JMXw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:cc:to:subject :message-id:date:from:in-reply-to:references:mime-version :dkim-signature; bh=z7AGSXDIWEX6D07QNcIzEX6hF/OEc51nsc+gf2PwPQk=; fh=ixFOam3Mo/TNk8BvTQlncxuGVa82wQNH3Bd63twaP9k=; b=hX7VvD57px6NceDETRwNE251LE9CD9eMGjLVIknH/nlnycGbJLo/UyNy/NTaqhC8o2 XZ22LJYX6meYJ1VS0nVkVJjZkJrD1ZgFcpb38kTuty6+yiND2/Y1efxA9FeNu9K4ajtn SSSSgZX9mrbQ00RHDRH9+UA7EegEIAaDIHkijYTK6etTcntJtBjm3VhpIDxpVkn8+RiX bEXddH0HlrV3vGwWwxtRHel1++HL8AKNYQzVeFegSUDDkZzdabdiaRJwiQ3bYxhwbWHU +0lfpHRYD4hC6gcZTChqxEWRaxhzSaHszWjaEn28cF/KJFJsLYYq7dA8s5UFkpwjRdDU o3Rg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="F/kLkGR/"; spf=pass (google.com: domain of linux-crypto-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id i64-20020a638743000000b0055b53a863f6si2411636pge.833.2023.06.30.08.22.10; Fri, 30 Jun 2023 08:22:30 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-crypto-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b="F/kLkGR/"; spf=pass (google.com: domain of linux-crypto-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-crypto-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232605AbjF3PR7 (ORCPT + 99 others); Fri, 30 Jun 2023 11:17:59 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35862 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232628AbjF3PRl (ORCPT ); Fri, 30 Jun 2023 11:17:41 -0400 Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0F56F49FA for ; Fri, 30 Jun 2023 08:17:00 -0700 (PDT) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id 20C1561773 for ; Fri, 30 Jun 2023 15:16:21 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 86C02C433C8; Fri, 30 Jun 2023 15:16:20 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1688138180; bh=AfDywCb2+AWTCrpWQEU0+M02rOsm/xUaTET5Eh90bBc=; h=References:In-Reply-To:From:Date:Subject:To:Cc:From; b=F/kLkGR/taZXcopd/ScaWwpB1BhQ3KKxPL3b5Ud9RuF3PbVq6oj0esTVWglVuflEX XOuSuC3dVm1i2iO9rXHr0hVIqXZ11276AV7Fe0FieNS5IIXEEXMpo4PkTcK5BGdhth Jxe9nKkD3WQD/7xbilj8PHE4xWUbPCy9YwUx/P5oWSFqzko65J07/2dxleEWdNuHzK +UwRhfuQd4inzCeFcdAL2qVP4LZNfbUYS0N0dDnBXl1gJkpDwOy5GBi/IL0Z7Dy2Pz 4raHECfMX9HuXHoXe5iVvJ62OoZ8gGr9NuSRFjdHTaS6TALmhFg+xCOfPAZupGhTW2 rQv9ktYlsIYdg== Received: by mail-lj1-f175.google.com with SMTP id 38308e7fff4ca-2b6a5fd1f46so30834951fa.1; Fri, 30 Jun 2023 08:16:20 -0700 (PDT) X-Gm-Message-State: ABy/qLYOBdkjXimnrIx34b3O2KhdY8a1zrnHBuUwS2wXROH3lVPuJg3B ALq2tUiggxdnlRvtOhEbN6TZ0xSipezQ2SDE01w= X-Received: by 2002:ac2:5bd0:0:b0:4f8:6600:4074 with SMTP id u16-20020ac25bd0000000b004f866004074mr2674183lfn.17.1688138178555; Fri, 30 Jun 2023 08:16:18 -0700 (PDT) MIME-Version: 1.0 References: <0000000000008a7ae505aef61db1@google.com> <20200911170150.GA889@sol.localdomain> <59e1d5c0-aedb-7b5b-f37f-0c20185d7e9b@I-love.SAKURA.ne.jp> In-Reply-To: From: Ard Biesheuvel Date: Fri, 30 Jun 2023 17:16:06 +0200 X-Gmail-Original-Message-ID: Message-ID: Subject: Re: [PATCH] net: tls: enable __GFP_ZERO upon tls_init() To: Alexander Potapenko Cc: Tetsuo Handa , Boris Pismenny , John Fastabend , Jakub Kicinski , herbert@gondor.apana.org.au, linux-crypto@vger.kernel.org, syzkaller-bugs@googlegroups.com, syzbot , Eric Biggers , Aviad Yehezkel , Daniel Borkmann , netdev@vger.kernel.org, "David S. Miller" , Eric Dumazet , Paolo Abeni Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI, SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-crypto@vger.kernel.org On Fri, 30 Jun 2023 at 13:55, Alexander Potapenko wrote= : > > On Fri, Jun 30, 2023 at 1:49=E2=80=AFPM Ard Biesheuvel = wrote: > > > > On Fri, 30 Jun 2023 at 13:38, Alexander Potapenko w= rote: > > > > > > On Fri, Jun 30, 2023 at 12:18=E2=80=AFPM Ard Biesheuvel wrote: > > > > > > > > On Fri, 30 Jun 2023 at 12:11, Alexander Potapenko wrote: > > > > > > > > > > On Fri, Jun 30, 2023 at 12:02=E2=80=AFPM Ard Biesheuvel wrote: > > > > > > > > > > > > On Fri, 30 Jun 2023 at 11:53, Tetsuo Handa > > > > > > wrote: > > > > > > > > > > > > > > On 2023/06/30 18:36, Ard Biesheuvel wrote: > > > > > > > > Why are you sending this now? > > > > > > > > > > > > > > Just because this is currently top crasher and I can reproduc= e locally. > > > > > > > > > > > > > > > Do you have a reproducer for this issue? > > > > > > > > > > > > > > Yes. https://syzkaller.appspot.com/text?tag=3DReproC&x=3D1293= 1621900000 works. > > > > > > > > > > > > > > > > > > > Could you please share your kernel config and the resulting ker= nel log > > > > > > when running the reproducer? I'll try to reproduce locally as w= ell, > > > > > > and see if I can figure out what is going on in the crypto laye= r > > > > > > > > > > The config together with the repro is available at > > > > > https://syzkaller.appspot.com/bug?extid=3D828dfc12440b4f6f305d, s= ee the > > > > > latest row of the "Crashes" table that contains a C repro. > > > > > > > > Could you explain why that bug contains ~50 reports that seem entir= ely > > > > unrelated? > > > > > > These are some unfortunate effects of syzbot trying to deduplicate > > > bugs. There's a tradeoff between reporting every single crash > > > separately and grouping together those that have e.g. the same origin= . > > > Applying this algorithm transitively results in bigger clusters > > > containing unwanted reports. > > > We'll look closer. > > > > > > > AIUI, this actual issue has not been reproduced since > > > > 2020?? > > > > > > Oh, sorry, I misread the table and misinformed you. The topmost row o= f > > > the table is indeed the _oldest_ one. > > > Another manifestation of the bug was on 2023/05/23 > > > (https://syzkaller.appspot.com/text?tag=3DCrashReport&x=3D146f66b1280= 000) > > > > > > > That one has nothing to do with networking, so I don't see how this > > patch would affect it. > > I definitely have to be more attentive. > You are right that this bug report is also unrelated. Yet it is still > fine to use the build artifacts corresponding to it (which is what I > did). > I'll investigate why so many reports got clustered into this one. > > > > > OK, thanks for the instructions. > > > > Out of curiosity - does the stack trace you cut off here include the > > BPF routine mentioned in the report? > > It does: > > [ 151.522472][ T5865] =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D > [ 151.523843][ T5865] BUG: KMSAN: uninit-value in aes_encrypt+0x15cc/0x1= db0 > [ 151.525120][ T5865] aes_encrypt+0x15cc/0x1db0 > [ 151.526113][ T5865] aesti_encrypt+0x7d/0xf0 > [ 151.527057][ T5865] crypto_cipher_encrypt_one+0x112/0x200 > [ 151.528224][ T5865] crypto_cbcmac_digest_update+0x301/0x4b0 > [ 151.529459][ T5865] shash_ahash_finup+0x66e/0xc00 > [ 151.530541][ T5865] shash_async_finup+0x7f/0xc0 > [ 151.531542][ T5865] crypto_ahash_finup+0x1b8/0x3e0 > [ 151.532583][ T5865] crypto_ccm_auth+0x1269/0x1350 > [ 151.533606][ T5865] crypto_ccm_encrypt+0x1c9/0x7a0 > [ 151.534650][ T5865] crypto_aead_encrypt+0xe0/0x150 > [ 151.535695][ T5865] tls_push_record+0x3bf3/0x4ec0 > [ 151.539491][ T5865] bpf_exec_tx_verdict+0x46e/0x21d0 > ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ > [ 151.540597][ T5865] tls_sw_do_sendpage+0x1150/0x1ad0 OK, so after poking around a little bit, I have managed to confirm that the problem is in the TLS layer, and I am a bit out of my depth debugging that. With the debugging code below applied, KMSAN triggers on an uninit-memory in the input scatterlist provided by the TLS layer into the CCM code. [ 148.375852][ T2424] =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D [ 148.377269][ T2424] BUG: KMSAN: uninit-value in tls_push_record+0x2d9f/0= x3eb0 [ 148.378623][ T2424] tls_push_record+0x2d9f/0x3eb0 [ 148.379559][ T2424] bpf_exec_tx_verdict+0x5ba/0x2530 [ 148.380534][ T2424] tls_sw_do_sendpage+0x169c/0x1f80 [ 148.381519][ T2424] tls_sw_sendpage+0x247/0x2b0 ... [ 148.411559][ T2424] [ 148.412108][ T2424] Bytes 0-15 of 16 are uninitialized [ 148.413379][ T2424] Memory access of size 16 starts at ffff8880157889c7 Note that this is the *input* scatterlist containing the AAD (additional authenticated data) and the crypto input, and so there is definitely a bug here that shouldn't be papered over by zero'ing the allocation. --- a/net/tls/tls_sw.c +++ b/net/tls/tls_sw.c @@ -543,6 +543,21 @@ static int tls_do_encryption(struct sock *sk, list_add_tail((struct list_head *)&rec->list, &ctx->tx_list); atomic_inc(&ctx->encrypt_pending); + { + int len =3D aead_req->assoclen + aead_req->cryptlen; + struct sg_mapping_iter miter; + + sg_miter_start(&miter, rec->sg_aead_in, + sg_nents(rec->sg_aead_in), + SG_MITER_TO_SG | SG_MITER_ATOMIC); + + while (len > 0 && sg_miter_next(&miter)) { + kmsan_check_memory(miter.addr, min(len, (int)miter.length)); + len -=3D miter.length; + } + sg_miter_stop(&miter); + } + The reason that this cascades all the way down to the AES cipher code appears to be that sbox substitution involves array indexing, which is one of the actions KMSAN qualifies as 'use' of uninit data.