Received: by 2002:a05:6358:4e97:b0:b3:742d:4702 with SMTP id ce23csp1722507rwb; Fri, 19 Aug 2022 08:19:53 -0700 (PDT) X-Google-Smtp-Source: AA6agR4vsozkzye5mZXefdiy/dZYDoAjpBuUdOeHrXbe+awioIDLrTVjdOMhg4e7vl1FZU6fX8ix X-Received: by 2002:a05:6402:f0e:b0:43d:61d6:bff9 with SMTP id i14-20020a0564020f0e00b0043d61d6bff9mr6298853eda.78.1660922392780; Fri, 19 Aug 2022 08:19:52 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1660922392; cv=none; d=google.com; s=arc-20160816; b=j/FoaMFfMPcU/MHtX6Twgb4cl6HAm87gijQyzvBV1y5CCaVfv4orti5kfbjS8msGzK vKB2px32GQlPxG5Fwzv5fkoaVCLAEtTylBxk97BImMB5fFgWTprWGaIRVRC/drPrC0rG YGa661xJvSSoWdde2zxkHiFNeen2iOFvXB1pVXLAmUEMndC7ZFjpPmS7M0CwDqWM3vi5 qUXmgXE41qgTB3JdTm5xHYMo135V5tW3EDwe+tyvnB4WzrLU1+CtWjDpKBHMAB/LT4Nx Ng4g4k5HSZRHdhBsamLcs/p722Yfso729T3S6klupWbJAoP/OWcz/X8Dm9u2QsXTZuAf iXEA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:in-reply-to:from :references:cc:to:content-language:subject:user-agent:mime-version :date:message-id; bh=AEaEgab2mhaMiSkvYoYiW7yx1z+hEFyxx9wlIiPpIa8=; b=Tes9OFP9X+d3zWFsTfbXhXCaDG3Ei4jyjjkXfog/22VkdPFdpXuCDSqht1flwQq7Cv zfH+fTtYNxwPqt0MhCruzzatMTejmOanhGnlaUoz1Cb1+Xvbt7DCpUI46cU4xX6h7Q15 xfghxUKPpqU9+p7UEt/MlFk0kidqGJKLUzwRDyYxt1XVEAXZhRWkeYGVRqTqIYwqR1AC UdiWSAIXoEUID+GLv2auLniOCNTeDZMTdV1/vgs1zjzxdiHGsNcrrrsJDzr3FjjGb/ZT nQVH6hrLX5Hrey3m++VFuYfkrPIlLIQC4MDKHBS97F6KVsb2DgeZoFB/oBZUEfd4xmtc /3Og== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=acm.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id x20-20020a05640226d400b0043c32b20a26si3554840edd.38.2022.08.19.08.19.26; Fri, 19 Aug 2022 08:19:52 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=acm.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1349430AbiHSOtb (ORCPT + 99 others); Fri, 19 Aug 2022 10:49:31 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:59476 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1348837AbiHSOta (ORCPT ); Fri, 19 Aug 2022 10:49:30 -0400 Received: from mail-pg1-f172.google.com (mail-pg1-f172.google.com [209.85.215.172]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 05583D87F1; Fri, 19 Aug 2022 07:49:29 -0700 (PDT) Received: by mail-pg1-f172.google.com with SMTP id c24so3864011pgg.11; Fri, 19 Aug 2022 07:49:29 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=content-transfer-encoding:in-reply-to:from:references:cc:to :content-language:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc; bh=AEaEgab2mhaMiSkvYoYiW7yx1z+hEFyxx9wlIiPpIa8=; b=PZ5svkXPbBwfSmVwVlHXWBmxq3AZJqp1RSsZNV5Oq5Dh1sKLcsDXsaDJP43L1FWZQq za3PikWje+XoLxbmk7iVdND4a6hbwkG8/J/It6sJQpAhMZKI11CSTYr/oBsdm+qrKIqR j7oIuOlmekZNPKeb+R8M+whb+ocqQKLGEURKHwm5j2FnoBHwJkTFRApU6tSZ8Tg/rbLA afLL5UfJymiV9jUWKcWyEconwlHGBeqrHSmqVDjv/FOAXbWHC/rhhfF6BvFX/D8my400 bBu4iCw0YNHY6pRewxyRO+8i6c/NU59CpaQn/5ycxNs4Vj6PN3TG82q/QUyDhURpochP E/kw== X-Gm-Message-State: ACgBeo1Wv8jjSIQEvvAySHP4iZmvJ/G6BdjNUqkvisRjf1CVEAJWjENQ bL2452RhqJ+s76ySQek3n7EwFwZkyiw= X-Received: by 2002:a63:1925:0:b0:429:f4f4:7d48 with SMTP id z37-20020a631925000000b00429f4f47d48mr6689792pgl.474.1660920568337; Fri, 19 Aug 2022 07:49:28 -0700 (PDT) Received: from [192.168.3.217] ([98.51.102.78]) by smtp.gmail.com with ESMTPSA id e14-20020aa798ce000000b00534a1315a63sm3580495pfm.136.2022.08.19.07.49.26 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 19 Aug 2022 07:49:27 -0700 (PDT) Message-ID: <17ccd5ae-0268-1bee-7822-1352f4c676ba@acm.org> Date: Fri, 19 Aug 2022 07:49:25 -0700 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.1.2 Subject: Re: 6.0-rc1 regression block (blk_mq) / RCU task stuck errors + block-io hang Content-Language: en-US To: Hans de Goede , Linux Kernel Mailing List , "regressions@lists.linux.dev" , Jens Axboe Cc: linux-block@vger.kernel.org, rcu@vger.kernel.org References: From: Bart Van Assche In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-1.4 required=5.0 tests=BAYES_00, FREEMAIL_FORGED_FROMDOMAIN,FREEMAIL_FROM,HEADER_FROM_DIFFERENT_DOMAINS, NICE_REPLY_A,RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 8/19/22 05:01, Hans de Goede wrote: > I've been dogfooding 6.0-rc1 on my main workstation and I have hit > this pretty serious bug, serious enough for me to go back to 5.19 > > My dmesg is showing various blk_mq (RCU?) related lockdep splats > followed by some tasks getting stuck on disk-IO. E.g. "sync" > is guaranteed to hang, but other tasks too. > > This seems to be mainly the case on "sd" disks (both sata > and USB) where as my main nvme drive seems fine, which has > probably saved me from worse issues... > > Here are 4 task stuck reports from my last boot, where > I had to turn off the machine by keeping the power button > pressed for 4 seconds. > > [ ... ] > > Sorry for not being able to write a better bug-report but I don't have > the time to dive into this deeper. I hope this report is enough for > someone to have a clue what is going on. Thank you for the detailed report. I think this report is detailed enough to root-cause this issue, something that was not possible before this report. Please help with verifying whether this patch fixes this issue: "[PATCH] scsi: sd: Revert "Rework asynchronous resume support"" (https://lore.kernel.org/linux-scsi/20220816172638.538734-1-bvanassche@acm.org/). Thanks, Bart.