Received: by 2002:a05:6359:322:b0:b3:69d0:12d8 with SMTP id ef34csp572699rwb; Wed, 10 Aug 2022 13:04:54 -0700 (PDT) X-Google-Smtp-Source: AA6agR5yFRtQBxyqURofh17TccL0C+onY96Al4vSrKIgY5kLZ/7JA8BCX6sDhuRDXQigJ2hTjL4A X-Received: by 2002:a05:6402:b66:b0:441:58ec:11 with SMTP id cb6-20020a0564020b6600b0044158ec0011mr8564630edb.267.1660161893801; Wed, 10 Aug 2022 13:04:53 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1660161893; cv=none; d=google.com; s=arc-20160816; b=NdQJE0AuZdza5FrJcsyeio8p3cRibroeMM7wQ+4FwZf8Oia/eJ2bqdeSBZKyHz1u5Z tTvBzyDBtjes6uvVbTXwPbzkyGt5v3zvQom86eB4+UKaWIHmjIoFsEu4CfLU///rNRn0 Zp3s6uk2PZQwVeK83e7vgtJ13RjnZ/7JJqfApS+yaERSNfi6f8A5rPNEoija1jyiIguc +Ly065Vj3qP8w7SEaVWC/48+2CJTluT2BkmEvDodBQFTPoo/sjWFonOV1S8UG73zCw1j uepP2ybkRI7GjgBdVKFoA4voC4I22SWTJwP8I7o7Hf0cvn+y/Zyps4I5NArVVYJbPnFH 65cw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:subject:cc:to:from:date:references:in-reply-to :message-id:mime-version:user-agent:feedback-id:dkim-signature :dkim-signature; bh=jLKPYg8r1NWKdk49fFDpdK5qae/XsFnguQ1wMSjTmTo=; b=WL1OGFFgXJRINoqwcNVMllxZALrSfs3O0sB0bMsf2Zw7DwdG5v5wh38GQUSh2tlUfo OiFeyBGdYtwN95JjzljmBCehMU5RdxRgUvK8jR2fkIPeZxtntcI9rLA2/eT8I6SUCIss Lxxb5FqsBWf3MgTvGkbWpG3pvFOv4yfXVyMltDGNQJzTtB66QDZrr9KlL+h8KBhsnsGl n+jEo5t2ONmbGOYTn23QEAiQzuzaFo1sx3Q/dNijSrgIB9LrlEtGuSmr2JilPWz+JutW LfqmI70xMW+Xc6fUJVswcBAYGhAlZEMGb+jzMRlSEADOR0PkpbHvylt4HcrbUEWCjUYz MCZw== ARC-Authentication-Results: i=1; mx.google.com; dkim=neutral (expired) header.i=@colorremedies.com header.s=fm1 header.b=jvoINbCT; dkim=neutral (expired) header.i=@messagingengine.com header.s=fm1 header.b=EzwBop3R; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id qk14-20020a1709077f8e00b007324aa3cc71si4974744ejc.386.2022.08.10.13.04.24; Wed, 10 Aug 2022 13:04:53 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=neutral (expired) header.i=@colorremedies.com header.s=fm1 header.b=jvoINbCT; dkim=neutral (expired) header.i=@messagingengine.com header.s=fm1 header.b=EzwBop3R; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S232479AbiHJSnF (ORCPT + 99 others); Wed, 10 Aug 2022 14:43:05 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:47298 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229924AbiHJSnD (ORCPT ); Wed, 10 Aug 2022 14:43:03 -0400 Received: from new3-smtp.messagingengine.com (new3-smtp.messagingengine.com [66.111.4.229]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 3AE90321; Wed, 10 Aug 2022 11:43:02 -0700 (PDT) Received: from compute3.internal (compute3.nyi.internal [10.202.2.43]) by mailnew.nyi.internal (Postfix) with ESMTP id 8D04E580902; Wed, 10 Aug 2022 14:43:01 -0400 (EDT) Received: from imap50 ([10.202.2.100]) by compute3.internal (MEProxy); Wed, 10 Aug 2022 14:43:01 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= colorremedies.com; h=cc:cc:content-type:date:date:from:from :in-reply-to:in-reply-to:message-id:mime-version:references :reply-to:sender:subject:subject:to:to; s=fm1; t=1660156981; x= 1660160581; bh=jLKPYg8r1NWKdk49fFDpdK5qae/XsFnguQ1wMSjTmTo=; b=j voINbCT7I34DAt8YlnXHdXbhohDUDl1XY9UryT5X3UbyLKV8Cm2dk2Ise8WlBqd7 1nEH8aoK6oHWzw+vUUksa6p+ixc7ISyR9nhdQ5H7adMVrKTUz1wtwu4z+gDJgFK8 HcIW4sglyTKAheCqk+nXraquVuV7m15vrXRZ/yyxStuGA5OXlZie/u/FWKGgoOrg kxqrrt2+sO6t9F0CBxxJsBhDDrgbkzuKVCQbwtbiKk9T+Liawo6I8mJ4YE30iwUt frQoFplUs0ZYoAlAU/vAo3KsC0k7wd1xLvoZgVJOWbQX7NptR91REaK6sOO5JLKH ySQIVaZl2aSPb5LpolyUg== DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d= messagingengine.com; h=cc:cc:content-type:date:date:feedback-id :feedback-id:from:from:in-reply-to:in-reply-to:message-id :mime-version:references:reply-to:sender:subject:subject:to:to :x-me-proxy:x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s= fm1; t=1660156981; x=1660160581; bh=jLKPYg8r1NWKdk49fFDpdK5qae/X sFnguQ1wMSjTmTo=; b=EzwBop3RAVvNTG/DtwC3n8JGE2I78/II65Xbj98SF7l7 lT4zzF0fs0NJge3SjchB21PrstCTQuc3MCfu+d578kAMSuVk516g2RK3kmVkWfxW s291/CCqRFRK7MFOh/l+Bp5ZE9gG/IhweejXJx4yC5iPyZporl97Tm0egwm3QKT9 41Q/qa2TKiv58UuXZVZ9XcDU5jbwNtS5MW5KT4NGXdWvHWCOQiZkFFDYeoaDSARO ng4GRsNZb4HfBx2L6/ncyQMbTXil2xKqrJxoQ5XeL3WiOYS394H/rQ6hMRtSqpwD UhsYS0ACemJZILDR+ewe6rpM6Y3NrL7M4yQQdaDdxw== X-ME-Sender: X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvfedrvdegvddguddviecutefuodetggdotefrod ftvfcurfhrohhfihhlvgemucfhrghsthforghilhdpqfgfvfdpuffrtefokffrpgfnqfgh necuuegrihhlohhuthemuceftddtnecusecvtfgvtghiphhivghnthhsucdlqddutddtmd enucfjughrpefofgggkfgjfhffhffvvefutgesthdtredtreertdenucfhrhhomhepfdev hhhrihhsucfouhhrphhhhidfuceolhhishhtshestgholhhorhhrvghmvgguihgvshdrtg homheqnecuggftrfgrthhtvghrnhepgfdvueektdefgfefgfdtleffvdeileetgfefuddt ffelueeiveeiveekhedtheeunecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpe hmrghilhhfrhhomheplhhishhtshestgholhhorhhrvghmvgguihgvshdrtghomh X-ME-Proxy: Feedback-ID: i06494636:Fastmail Received: by mailuser.nyi.internal (Postfix, from userid 501) id 54B791700082; Wed, 10 Aug 2022 14:43:01 -0400 (EDT) X-Mailer: MessagingEngine.com Webmail Interface User-Agent: Cyrus-JMAP/3.7.0-alpha0-811-gb808317eab-fm-20220801.001-gb808317e Mime-Version: 1.0 Message-Id: In-Reply-To: <2220d403-e443-4e60-b7c3-d149e402c13e@www.fastmail.com> References: <2220d403-e443-4e60-b7c3-d149e402c13e@www.fastmail.com> Date: Wed, 10 Aug 2022 14:42:40 -0400 From: "Chris Murphy" To: "Josef Bacik" Cc: "Btrfs BTRFS" , Linux-RAID , linux-block@vger.kernel.org, linux-kernel Subject: Re: stalling IO regression in linux 5.12 Content-Type: text/plain X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,SPF_HELO_PASS, SPF_PASS,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, Aug 10, 2022, at 2:33 PM, Chris Murphy wrote: > On Wed, Aug 10, 2022, at 1:48 PM, Josef Bacik wrote: > >> To help narrow this down can you disable any IO controller you've got enabled >> and see if you can reproduce? If you can sysrq+w is super helpful as it'll >> point us in the next direction to look. Thanks, > > I'm not following, sorry. I can boot with > systemd.unified_cgroup_hierarchy=0 to make sure it's all off, but we're > not using an IO cgroup controllers specifically as far as I'm aware. OK yeah that won't work because the workload requires cgroup2 or it won't run. -- Chris Murphy