Received: by 2002:a05:7412:37c9:b0:e2:908c:2ebd with SMTP id jz9csp2520250rdb; Fri, 22 Sep 2023 00:30:29 -0700 (PDT) X-Google-Smtp-Source: AGHT+IES+XqBaBVLbiuNB5yIaAnpDezGysR6c5IlCVe1NeWdsgxWKuejql2/VtQv9EtFGFgXycO7 X-Received: by 2002:a05:6870:c69f:b0:1d6:790e:dacc with SMTP id cv31-20020a056870c69f00b001d6790edaccmr8429658oab.6.1695367829509; Fri, 22 Sep 2023 00:30:29 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1695367829; cv=none; d=google.com; s=arc-20160816; b=a6+WBKLLb86icM2d0qKSuqBrc1pYei3fgGNDJm5WwvwB8e9zr38gRBE/hiiZZhYmkZ xHSEcGcJL2u79RvUWfrApszsei7u+L93447pBGDRs9ApR6kiwEAqmV7+JQcYLjIE2hL6 Q4cpFDzBBzPINw5eEQuheug8Sr+F7fnJYG2rRQjnLuM1dWGJYTCxvAHusc+hc44vX3GG SogCh4K9HKxQ4uEII4KrtZYtOOovSi0vpxRcUwswxtAOLWzDGSqu+C4MvxPuy3aor40f sGhNVyx1D6whr50mXLxrA/SJ/undOfaJ0w1SzM2xCpoeJacMVff6WEbvao0d2XDk96oT T3dw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:in-reply-to :content-language:references:cc:to:from:subject:user-agent :mime-version:date:message-id:dkim-signature; bh=Q0jmxeUHMyRvSul47ykDzG/kcgtqwpKI1qC1sVJFmMs=; fh=3DcPGeyQ1uAXNtvQh9A9vivh+R0C8JAd8qT4XQIPRTs=; b=Cxj8Ga4ezVU2Ylxk4Vv01mvG6B8eI5WwjKLc+TM/m32dKpUAILvJyWL3hRNM6A1Ryo Vovu1u1I3/Am9EC3jklz7Z7VqsQ88vleUCw2qItMCdorLUA6UoPjOnbcNDr/UzinnedY mbYPQzInM2c7sRm+Ls0zrrf01KAxxqw6NJV6pM3QmbrDrm+sBcsVBRRodohLfSq8YDbA JNNXb2Fhf6vgogZUSV4+beVljtzVzzqU5QsrW0s1cCpSlttR/DAlYgCbgvaH2JHJr4n9 oC3BQnzpCgqVeMNHgicktNxR/9WZUVFtcbisyRqE5Yayd8sGwRBAyxEjc7Xb3UD5GyZe 3SJA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@arista.com header.s=google header.b=Uu67o6wH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=arista.com Return-Path: Received: from lipwig.vger.email (lipwig.vger.email. [23.128.96.33]) by mx.google.com with ESMTPS id s17-20020a656911000000b0056c2f508898si3240113pgq.725.2023.09.22.00.30.28 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 22 Sep 2023 00:30:29 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) client-ip=23.128.96.33; Authentication-Results: mx.google.com; dkim=pass header.i=@arista.com header.s=google header.b=Uu67o6wH; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.33 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=arista.com Received: from out1.vger.email (depot.vger.email [IPv6:2620:137:e000::3:0]) by lipwig.vger.email (Postfix) with ESMTP id EE88780926B0; Thu, 21 Sep 2023 16:15:57 -0700 (PDT) X-Virus-Status: Clean X-Virus-Scanned: clamav-milter 0.103.10 at lipwig.vger.email Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229609AbjIUXPv (ORCPT + 99 others); Thu, 21 Sep 2023 19:15:51 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59056 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229490AbjIUXPu (ORCPT ); Thu, 21 Sep 2023 19:15:50 -0400 Received: from mail-wm1-x342.google.com (mail-wm1-x342.google.com [IPv6:2a00:1450:4864:20::342]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 077FB110 for ; Thu, 21 Sep 2023 16:15:43 -0700 (PDT) Received: by mail-wm1-x342.google.com with SMTP id 5b1f17b1804b1-4051fea48a8so17443485e9.2 for ; Thu, 21 Sep 2023 16:15:42 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=arista.com; s=google; t=1695338141; x=1695942941; darn=vger.kernel.org; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id:from:to :cc:subject:date:message-id:reply-to; bh=Q0jmxeUHMyRvSul47ykDzG/kcgtqwpKI1qC1sVJFmMs=; b=Uu67o6wHSCJOmW99y3/jtHtDUDAtf/NQmtb7dPJHoFi2nsTzA4MNbQoRMvDuygl3n+ eA6WHhNZ264slaSeFY8ulm8mNBDmt0GbEzrOMf1VvTmqGgBKnXIvTEMz8iAxge9XKNq/ 86SmNvBauir3eZPjMQdzX2KiZ3RUTkRoJPgNXal7PkQ2dQs/j8tXWTAlaFQxuRT8/Nti JrAwTA5DmI6P8UNLwVZQ4SiFGVtx9+V2jbEwcmqmW8JH1i/YIo93kP0GnA8XwEu4a94O X6Y36LkX3DqpWEDoFrMrbijZMFFwnhdKwP1NzlOxSN5t1ksFxPCwGhOzl4R29uYHkTAd NftQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1695338141; x=1695942941; h=content-transfer-encoding:in-reply-to:content-language:references :cc:to:from:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=Q0jmxeUHMyRvSul47ykDzG/kcgtqwpKI1qC1sVJFmMs=; b=CwUjf9PS8CkRED5r9srkdj6U5uQo7zZDZo9fprCy18gXjIv3MWTOapNoi4VVLC7SBQ vtfu12G3/xMg+ORXdwSZxR9DQ4iZfAjfMpIOJ/aeoEdl5mNSDf61V2/361JaweuNYg3z 0cFk+HVvHFy9iUptSF6gpJsDKcRLeRJFXO/MuTkTbyhUcNhGz3AGEyXzdDD0+koqITWV uma1p7eiY3J/9yQVN9An4N9T2P5UWzB13mgtVsxfkmazrCnX2mQHq6nN6xljErsbqpxK gKCnISImH/TezyKSf21/i+t+RNRDi6L0nQ9BXwaN+RbiCkgYphIueQDyHnHXHXyU+1Nj O2SA== X-Gm-Message-State: AOJu0Yz85318ir8wbwATkqrUpFGIFDQvLvb82i9ovrb//DBQWNUw50RJ ohPKAfD/P+5cNmCRD4x6V+jd+g== X-Received: by 2002:a5d:440b:0:b0:319:6d03:13ae with SMTP id z11-20020a5d440b000000b003196d0313aemr6267180wrq.55.1695338141414; Thu, 21 Sep 2023 16:15:41 -0700 (PDT) Received: from [10.83.37.178] ([217.173.96.166]) by smtp.gmail.com with ESMTPSA id a3-20020a5d5083000000b003198a9d758dsm2911162wrt.78.2023.09.21.16.15.40 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Thu, 21 Sep 2023 16:15:40 -0700 (PDT) Message-ID: <0d9983af-1483-d43e-810e-64ce6068a381@arista.com> Date: Fri, 22 Sep 2023 00:15:27 +0100 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.13.0 Subject: Re: [syzbot] [net?] memory leak in tcp_md5_do_add From: Dmitry Safonov To: Eric Dumazet Cc: bpf@vger.kernel.org, davem@davemloft.net, dsahern@kernel.org, kuba@kernel.org, linux-kernel@vger.kernel.org, netdev@vger.kernel.org, pabeni@redhat.com, syzkaller-bugs@googlegroups.com, syzbot , Catalin Marinas References: <0000000000004d83170605e16003@google.com> <18267b34-1dcf-08d5-5ba1-4f5162e6c43a@arista.com> Content-Language: en-US In-Reply-To: <18267b34-1dcf-08d5-5ba1-4f5162e6c43a@arista.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.3 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,NICE_REPLY_A,SPF_HELO_NONE,SPF_PASS,URIBL_BLOCKED autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lipwig.vger.email Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (lipwig.vger.email [0.0.0.0]); Thu, 21 Sep 2023 16:15:58 -0700 (PDT) Hi Eric, On 9/21/23 18:01, Dmitry Safonov wrote: > On 9/21/23 17:59, Eric Dumazet wrote: >> On Thu, Sep 21, 2023 at 6:56 PM syzbot >> wrote: >>> >>> Hello, >>> >>> syzbot found the following issue on: >>> >>> HEAD commit: ee3f96b16468 Merge tag 'nfsd-6.3-1' of git://git.kernel.or.. >>> git tree: upstream >>> console output: https://syzkaller.appspot.com/x/log.txt?x=1312bba8c80000 >>> kernel config: https://syzkaller.appspot.com/x/.config?x=f5733ca1757172ad >>> dashboard link: https://syzkaller.appspot.com/bug?extid=68662811b3d5f6695bcb >>> compiler: gcc (Debian 10.2.1-6) 10.2.1 20210110, GNU ld (GNU Binutils for Debian) 2.35.2 >>> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=105393a8c80000 >>> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=1113917f480000 >>> >>> Downloadable assets: >>> disk image: https://storage.googleapis.com/syzbot-assets/29e7966ab711/disk-ee3f96b1.raw.xz >>> vmlinux: https://storage.googleapis.com/syzbot-assets/ae21b8e855de/vmlinux-ee3f96b1.xz >>> kernel image: https://storage.googleapis.com/syzbot-assets/803ee0425ad6/bzImage-ee3f96b1.xz >>> >>> IMPORTANT: if you fix the issue, please add the following tag to the commit: >>> Reported-by: syzbot+68662811b3d5f6695bcb@syzkaller.appspotmail.com >>> >>> executing program >>> BUG: memory leak >>> unreferenced object 0xffff88810a86f7a0 (size 32): >>> comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s) >>> hex dump (first 32 bytes): >>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ >>> 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ >>> backtrace: >>> [] kmalloc_trace+0x24/0x90 mm/slab_common.c:1061 >>> [] kmalloc include/linux/slab.h:580 [inline] >>> [] tcp_md5sig_info_add net/ipv4/tcp_ipv4.c:1169 [inline] >>> [] tcp_md5_do_add+0xa0/0x150 net/ipv4/tcp_ipv4.c:1240 >>> [] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671 >>> [] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720 >>> [] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806 >>> [] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274 >>> [] __do_sys_setsockopt net/socket.c:2285 [inline] >>> [] __se_sys_setsockopt net/socket.c:2282 [inline] >>> [] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282 >>> [] do_syscall_x64 arch/x86/entry/common.c:50 [inline] >>> [] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80 >>> [] entry_SYSCALL_64_after_hwframe+0x63/0xcd >>> >>> BUG: memory leak >>> unreferenced object 0xffff88811225ccc0 (size 192): >>> comm "syz-executor325", pid 5099, jiffies 4294978342 (age 119.240s) >>> hex dump (first 32 bytes): >>> 00 00 00 00 00 00 00 00 22 01 00 00 00 00 ad de ........"....... >>> 22 0a 80 00 fe 80 00 00 00 00 00 00 00 00 00 00 "............... >>> backtrace: >>> [] __do_kmalloc_node mm/slab_common.c:966 [inline] >>> [] __kmalloc+0x4a/0x120 mm/slab_common.c:980 >>> [] kmalloc include/linux/slab.h:584 [inline] >>> [] sock_kmalloc net/core/sock.c:2635 [inline] >>> [] sock_kmalloc+0x65/0xa0 net/core/sock.c:2624 >>> [] __tcp_md5_do_add+0xcb/0x300 net/ipv4/tcp_ipv4.c:1212 >>> [] tcp_md5_do_add+0x67/0x150 net/ipv4/tcp_ipv4.c:1253 >>> [] tcp_v6_parse_md5_keys+0x253/0x4a0 net/ipv6/tcp_ipv6.c:671 >>> [] do_tcp_setsockopt+0x40e/0x1360 net/ipv4/tcp.c:3720 >>> [] tcp_setsockopt+0x9b/0xa0 net/ipv4/tcp.c:3806 >>> [] __sys_setsockopt+0x1ab/0x330 net/socket.c:2274 >>> [] __do_sys_setsockopt net/socket.c:2285 [inline] >>> [] __se_sys_setsockopt net/socket.c:2282 [inline] >>> [] __x64_sys_setsockopt+0x26/0x30 net/socket.c:2282 >>> [] do_syscall_x64 arch/x86/entry/common.c:50 [inline] >>> [] do_syscall_64+0x39/0xb0 arch/x86/entry/common.c:80 >>> [] entry_SYSCALL_64_after_hwframe+0x63/0xcd >>> >>> >>> >>> --- >>> This report is generated by a bot. It may contain errors. >>> See https://goo.gl/tpsmEJ for more information about syzbot. >>> syzbot engineers can be reached at syzkaller@googlegroups.com. >>> >>> syzbot will keep track of this issue. See: >>> https://goo.gl/tpsmEJ#status for how to communicate with syzbot. >>> >>> If the bug is already fixed, let syzbot know by replying with: >>> #syz fix: exact-commit-title >>> >>> If you want syzbot to run the reproducer, reply with: >>> #syz test: git://repo/address.git branch-or-commit-hash >>> If you attach or paste a git patch, syzbot will apply it before testing. >>> >>> If you want to overwrite bug's subsystems, reply with: >>> #syz set subsystems: new-subsystem >>> (See the list of subsystem names on the web dashboard) >>> >>> If the bug is a duplicate of another bug, reply with: >>> #syz dup: exact-subject-of-another-report >>> >>> If you want to undo deduplication, reply with: >>> #syz undup >> >> Dmitry, please take a look at this bug, we need to fix it before your >> patch series. > > Sure, seems reasonable to me to fix before merging something on top. It seems to me that it's related to a race between RCU grace period and kmemleak scan period. There seems to be a patch [1] that likely fixes that, albeit I couldn't verify it as all my attempts to reproduce syzbot issue produced only unrelated to TCP-MD5 log: > [ 263.201211] kmemleak: unreferenced object 0xffff9ceb047d9948 (size 192): > [ 263.201781] kmemleak: comm "ip", pid 730, jiffies 4294937874 (age 257.270s) > [ 263.202460] kmemleak: hex dump (first 32 bytes): > [ 263.202921] kmemleak: 00 c8 e9 01 eb 9c ff ff e0 00 00 01 00 00 00 00 ................ > [ 263.203700] kmemleak: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ > [ 263.204484] kmemleak: backtrace: > [ 263.204814] kmemleak: [] kmalloc_trace+0x26/0x90 > [ 263.205440] kmemleak: [] ____ip_mc_inc_group+0xa0/0x240 > [ 263.206134] kmemleak: [] ip_mc_up+0x4b/0xb0 > [ 263.206725] kmemleak: [] inetdev_event+0xbb/0x5c0 > [ 263.207358] kmemleak: [] notifier_call_chain+0x56/0xc0 > [ 263.208070] kmemleak: [] __dev_notify_flags+0x58/0xf0 > [ 263.208784] kmemleak: [] dev_change_flags+0x50/0x60 > [ 263.209471] kmemleak: [] devinet_ioctl+0x378/0x770 > [ 263.210152] kmemleak: [] inet_ioctl+0x187/0x1d0 > [ 263.210805] kmemleak: [] sock_do_ioctl+0x3d/0x100 > [ 263.211482] kmemleak: [] sock_ioctl+0xe3/0x2b0 > [ 263.212131] kmemleak: [] __x64_sys_ioctl+0x8c/0xc0 > [ 263.212789] kmemleak: [] do_syscall_64+0x35/0x80 > [ 263.213438] kmemleak: [] entry_SYSCALL_64_after_hwframe+0x46/0xb0 > [ 263.214283] kmemleak: unreferenced object 0xffff9ceb03ad5400 (size 512): > [ 263.214982] kmemleak: comm "ip", pid 730, jiffies 4294937874 (age 257.290s) > [ 263.215728] kmemleak: hex dump (first 32 bytes): > [ 263.216231] kmemleak: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 ................ > [ 263.217106] kmemleak: 80 00 00 00 00 00 00 00 ff ff ff ff ff ff ff ff ................ > [ 263.218041] kmemleak: backtrace: > [ 263.218438] kmemleak: [] kmalloc_trace+0x26/0x90 > [ 263.219181] kmemleak: [] ipv6_add_addr+0x13b/0x6c0 > [ 263.219931] kmemleak: [] add_addr+0x75/0x150 > [ 263.220627] kmemleak: [] addrconf_notify+0x53d/0x730 > [ 263.221377] kmemleak: [] notifier_call_chain+0x56/0xc0 > [ 263.222104] kmemleak: [] __dev_notify_flags+0x58/0xf0 > [ 263.222844] kmemleak: [] dev_change_flags+0x50/0x60 > [ 263.223581] kmemleak: [] devinet_ioctl+0x378/0x770 > [ 263.224293] kmemleak: [] inet_ioctl+0x187/0x1d0 > [ 263.224961] kmemleak: [] sock_do_ioctl+0x3d/0x100 > [ 263.225660] kmemleak: [] sock_ioctl+0xe3/0x2b0 > [ 263.226331] kmemleak: [] __x64_sys_ioctl+0x8c/0xc0 > [ 263.227039] kmemleak: [] do_syscall_64+0x35/0x80 > [ 263.227747] kmemleak: [] entry_SYSCALL_64_after_hwframe+0x46/0xb0 > [ 263.228708] kmemleak: 2 new suspected memory leaks (see /sys/kernel/debug/kmemleak) This seems to be quite the same issue: inet6_ifa_finish_destroy() destroys inet6_ifaddr with kfree_rcu(). [1] https://lore.kernel.org/linux-mm/ZQA064908T5nngcc@arm.com/T/#ma4a68fdc44793e2594c9e7cadefa8ea40da5807d Thanks, Dmitry