Received: by 2002:a25:1985:0:0:0:0:0 with SMTP id 127csp2203468ybz; Sun, 26 Apr 2020 13:29:38 -0700 (PDT) X-Google-Smtp-Source: APiQypJNKSqJuTbUJVuWWsrPEWbzyBZR5g4vBK/DLNZ6EVy/kGGitKXrsNfG6j/AIDiMuDlqkOj7 X-Received: by 2002:a05:6402:391:: with SMTP id o17mr16231963edv.71.1587932978751; Sun, 26 Apr 2020 13:29:38 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1587932978; cv=none; d=google.com; s=arc-20160816; b=RJfvsf3UUsG3Yr/Aa94YrqbxvaVG9uBBxu9H1sJekEsz445d2zHs3M9VJL5fYys4HU nIH3VGnFtsm5fwqc94JCrI9CiE0JGyKQOSGdy3bBCKmU7HFxcC8mBHS8CXtlOEceVRsN QuhPwOZIq3SaQw6bU0bnu/S8dpZeR3XsvorvWAqBBozSjii3TzUCY0wFScw/Ekd+2JFU Fit05WhCqJBNLez2DiJqqDGtJ4ovm/wjhilwDzMqcj6ZgPlIHIKlmQ35jkoF0tAE+hGZ MpOHsdWamCE23o4VWsyoOKTSRy+rgzzmqegNzCuC2hYvLgSodS0SSEAJblbqNilkx06W GpTQ== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding :content-language:in-reply-to:mime-version:user-agent:date :message-id:from:references:cc:to:subject:dkim-signature; bh=vTJAHAMO4w1usv9r2nAHSyTTMt/8kF833Hejpn2JxlM=; b=fdOxgUzFGH2ONCsBzOKMzyWQq+CdKU0Ok4Hf/l8+FaeojScLXEj2U3d1nZ0n81edjW LQ/vpFTRVD+ju6I9B9DZsRsRfg0UjxO29NbSYyZLPtNtDiiOVD26WqZYI0snPMuRkSL+ wfZTMUddaOn1MyjfBKs8X2VYHuVDiDUI70OfaB53cN1ceSQ7HYPS+QzsHZH5rjJvKTXw vS0Kh4mlB38eWPL2znmE1uSkW5q5NvCnp/m1rOtDjaQuQvPB4ZsQ7NfsNGU0zhDOIj4z m8LM3FflaLf6w5wJI4ZOOCk9UBq4M7fGayW+XXIjRDBkUPRM86MQiumB8LFTnF6HYnaA z6ow== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=T82Gpc8p; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id u21si7253066ejz.518.2020.04.26.13.29.15; Sun, 26 Apr 2020 13:29:38 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=T82Gpc8p; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726332AbgDZU1B (ORCPT + 99 others); Sun, 26 Apr 2020 16:27:01 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:35712 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-FAIL-OK-FAIL) by vger.kernel.org with ESMTP id S1726176AbgDZU1B (ORCPT ); Sun, 26 Apr 2020 16:27:01 -0400 Received: from mail-pf1-x444.google.com (mail-pf1-x444.google.com [IPv6:2607:f8b0:4864:20::444]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 68B68C061A0F; Sun, 26 Apr 2020 13:27:00 -0700 (PDT) Received: by mail-pf1-x444.google.com with SMTP id x2so4096400pfx.7; Sun, 26 Apr 2020 13:27:00 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:cc:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=vTJAHAMO4w1usv9r2nAHSyTTMt/8kF833Hejpn2JxlM=; b=T82Gpc8ptWG7LSO05/xKP5AayQbZ/sxluy3IIF6ePxiLktxwR1NcI7S4/Fh5Kf8uMJ 2IAtNszONmRCHxApd07MJ1/u6umxPD7SEA5NA9qWxaSodQyr6i6rfAMCeNOu9ardVS1y pFW96vxRkeGqoH//3ZNNaIoiRLFguw3pEoVHMhktijYKLPrxoYmg162WKhc9qmzNvKXG htMHhLhtaIFDRyup/rCHRYEZAfbWl+cR5UcZKN9YAho23nfpmWYkKAWUvgfrQifQpT0O STiiFB0mSPD/Y2lDNI7LHksfdneo2QKiy+rBXbJCzXzGoQGMI3FI7XBfudOHSiz00YwK 88bQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:cc:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=vTJAHAMO4w1usv9r2nAHSyTTMt/8kF833Hejpn2JxlM=; b=IC7QQB9l9w2EqSDU2NuiLAqNki0okTIVRf4b0fwN4cytBfye309GblIdkAMKZPDohG kVTaFXqEJ65bfVujTELta5YyIH//GcgwfrFqokep8JszDTZJ5f+TPrA+Al9McbGC0i0e b2cJLhEZpM4ct1G1zTsoQ54ywyyIxA1MeAB2S8nsGqbmjw4aEeXO1iajlzvh8gVNKjaf 9m2YSOevjdtT46zv57vCzHpxAzxtFpm/Po6F9YNFG1oySafz5vdwWt6F6L3WsiODqsmO Zbtfmznx/QnRTwlsv6C48meCGzBfjg/DeLlFO4NT0+HvtlkW3pje3MScmgsRD34taCqN UF+g== X-Gm-Message-State: AGi0PuZZ0dvm8zkqL3meH640e/d7XDTSisTvdCSb7ciDrgk3DMKYn0VR ZfIonVGCU+vE0D3Nx9f2enU= X-Received: by 2002:a63:5d5c:: with SMTP id o28mr20467058pgm.322.1587932819918; Sun, 26 Apr 2020 13:26:59 -0700 (PDT) Received: from [192.168.86.235] (c-73-241-150-58.hsd1.ca.comcast.net. [73.241.150.58]) by smtp.gmail.com with ESMTPSA id q11sm3748378pfl.97.2020.04.26.13.26.57 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sun, 26 Apr 2020 13:26:59 -0700 (PDT) Subject: Re: INFO: rcu detected stall in wg_packet_tx_worker To: "Jason A. Donenfeld" , Eric Dumazet Cc: syzbot , David Miller , Florian Fainelli , Greg Kroah-Hartman , jhs@mojatatu.com, =?UTF-8?B?SmnFmcOtIFDDrXJrbw==?= , Krzysztof Kozlowski , kuba@kernel.org, kvalo@codeaurora.org, leon@kernel.org, LKML , linux-kselftest@vger.kernel.org, Netdev , Shuah Khan , syzkaller-bugs@googlegroups.com, Thomas Gleixner , vivien.didelot@gmail.com, Cong Wang References: <0000000000005fd19505a4355311@google.com> From: Eric Dumazet Message-ID: Date: Sun, 26 Apr 2020 13:26:56 -0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.6.0 MIME-Version: 1.0 In-Reply-To: Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 4/26/20 12:42 PM, Jason A. Donenfeld wrote: > On Sun, Apr 26, 2020 at 1:40 PM Eric Dumazet wrote: >> >> >> >> On 4/26/20 10:57 AM, syzbot wrote: >>> syzbot has bisected this bug to: >>> >>> commit e7096c131e5161fa3b8e52a650d7719d2857adfd >>> Author: Jason A. Donenfeld >>> Date: Sun Dec 8 23:27:34 2019 +0000 >>> >>> net: WireGuard secure network tunnel >>> >>> bisection log: https://syzkaller.appspot.com/x/bisect.txt?x=15258fcfe00000 >>> start commit: b2768df2 Merge branch 'for-linus' of git://git.kernel.org/.. >>> git tree: upstream >>> final crash: https://syzkaller.appspot.com/x/report.txt?x=17258fcfe00000 >>> console output: https://syzkaller.appspot.com/x/log.txt?x=13258fcfe00000 >>> kernel config: https://syzkaller.appspot.com/x/.config?x=b7a70e992f2f9b68 >>> dashboard link: https://syzkaller.appspot.com/bug?extid=0251e883fe39e7a0cb0a >>> userspace arch: i386 >>> syz repro: https://syzkaller.appspot.com/x/repro.syz?x=15f5f47fe00000 >>> C reproducer: https://syzkaller.appspot.com/x/repro.c?x=11e8efb4100000 >>> >>> Reported-by: syzbot+0251e883fe39e7a0cb0a@syzkaller.appspotmail.com >>> Fixes: e7096c131e51 ("net: WireGuard secure network tunnel") >>> >>> For information about bisection process see: https://goo.gl/tpsmEJ#bisection >>> >> >> I have not looked at the repro closely, but WireGuard has some workers >> that might loop forever, cond_resched() might help a bit. > > I'm working on this right now. Having a bit difficult of a time > getting it to reproduce locally... > > The reports show the stall happening always at: > > static struct sk_buff * > sfq_dequeue(struct Qdisc *sch) > { > struct sfq_sched_data *q = qdisc_priv(sch); > struct sk_buff *skb; > sfq_index a, next_a; > struct sfq_slot *slot; > > /* No active slots */ > if (q->tail == NULL) > return NULL; > > next_slot: > a = q->tail->next; > slot = &q->slots[a]; > > Which is kind of interesting, because it's not like that should block > or anything, unless there's some kasan faulting happening. > I am not really sure WireGuard is involved, the repro does not rely on it anyway.