Received: by 2002:a05:6a10:5bc5:0:0:0:0 with SMTP id os5csp5119789pxb; Thu, 14 Oct 2021 20:02:44 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwM1z0wvmjKyVo5SPsT+LEMn9UZjQVOy1U9BrqleaxMHnjDHigoOdfleTUvfcpZ3m0YECLA X-Received: by 2002:a17:90a:ec17:: with SMTP id l23mr10503755pjy.184.1634266964739; Thu, 14 Oct 2021 20:02:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1634266964; cv=none; d=google.com; s=arc-20160816; b=YM9swx76MN3XBhEdOSkx2LSsMn4JH1yrr4rV0imlAKK04Tg19iDBs7PpEo4kQog3Fn sLieDIxar7QwwFUQTPnpnuwrlrj0+2OhD3EANGplW013ur39VxZgIn4oOS4xbE3GiR9u m7k+a+dzMFsPJkPjGnHRJPhPPLnhGy811CGBkkJ0VSTTlpqtEu6VHml6zne18jM/qpPn p2FPC8hO2YBVw0ywORz24Hq4CGKX0mx+IOqvNhWP2erWQYsVicjeooVFiiQ8aJ8wctzk Ih779QD2yWinqJZKTXN6Tk9QfMvmuDQnqXC/VcKP9+pGagSzRbQVkx/J0VPV0D5bgSZa iacg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:message-id:date:references:in-reply-to:subject :cc:to:from:mime-version:content-transfer-encoding:dkim-signature :dkim-signature; bh=ZUFL3ZLGPm0rWVZLWnRAj8hRPQ4ZV05W4Un+rW6T6io=; b=APWjIhd98hyjI3uOYP7F0B10O8Sm7UyrX2TlAkuIL7iRmTdQVtYnW6tQtf8ek+IrCd VWny7bWYgnY/GnhDJMoB6iCgoD0rlY6yBN+/Rfb90VKu5UIHVttmhLo6IjByqEnrm0nb hVmszd45Nz3cf5CsxlksuD0qZZPQmWDmUjCn5nJAE4wGJW3+yiaRr+MGj7lAqrrCGkjL btRpTEjZMdsEhOgJa63nlGC9VIOkIXkyNoLkPzUsQyirjq4fnahz8HFpOlT6Fkw5Xz2p oWKWvXwhruA2ocUg6yMJ0beWyRbC+WXEWfv3FrjEqWxNvjXr/TCkIvwuG56nHyP007GS a4AQ== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=PchXP96A; dkim=neutral (no key) header.i=@suse.de; spf=pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-nfs-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id k5si6327214pgq.57.2021.10.14.20.02.25; Thu, 14 Oct 2021 20:02:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@suse.de header.s=susede2_rsa header.b=PchXP96A; dkim=neutral (no key) header.i=@suse.de; spf=pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-nfs-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=suse.de Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233893AbhJNWaV (ORCPT + 99 others); Thu, 14 Oct 2021 18:30:21 -0400 Received: from smtp-out1.suse.de ([195.135.220.28]:42856 "EHLO smtp-out1.suse.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229768AbhJNWaU (ORCPT ); Thu, 14 Oct 2021 18:30:20 -0400 Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by smtp-out1.suse.de (Postfix) with ESMTPS id 8EEC921A72; Thu, 14 Oct 2021 22:28:14 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_rsa; t=1634250494; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZUFL3ZLGPm0rWVZLWnRAj8hRPQ4ZV05W4Un+rW6T6io=; b=PchXP96AsDXWfThRfYutw0VLLgHywxPa5D33oJCwcd7Z2ClFl3BETX3g8tcEIxQWj3GBUi GtSIU29aa0NiCfBIqpwv2t/JxEZjAab+jC0bah6vE9YvZJZFOaMW63Dd00kNrsATcXLnw1 zvTKn1azyjrPSIp3GKL/YdTQUn7uHEk= DKIM-Signature: v=1; a=ed25519-sha256; c=relaxed/relaxed; d=suse.de; s=susede2_ed25519; t=1634250494; h=from:from:reply-to:date:date:message-id:message-id:to:to:cc:cc: mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=ZUFL3ZLGPm0rWVZLWnRAj8hRPQ4ZV05W4Un+rW6T6io=; b=skP2MspvCNtFDeI9MSDhoQrHyAFp2kiXrfBWNSNLmQCe92zlGBxtqSlwM3qw1e6u2Mdton m8fjV3Hqpzn8f5Dg== Received: from imap2.suse-dmz.suse.de (imap2.suse-dmz.suse.de [192.168.254.74]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature ECDSA (P-521) server-digest SHA512) (No client certificate requested) by imap2.suse-dmz.suse.de (Postfix) with ESMTPS id 3D4F513B3A; Thu, 14 Oct 2021 22:28:11 +0000 (UTC) Received: from dovecot-director2.suse.de ([192.168.254.65]) by imap2.suse-dmz.suse.de with ESMTPSA id M2i2NvuuaGHiPAAAMHmgww (envelope-from ); Thu, 14 Oct 2021 22:28:11 +0000 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit MIME-Version: 1.0 From: "NeilBrown" To: "Chuck Lever III" Cc: "Bruce Fields" , "Scott Mayhew" , "Timothy Pearson" , "Linux NFS Mailing List" , "Trond Myklebust" Subject: Re: CPU stall, eventual host hang with BTRFS + NFS under heavy load In-reply-to: <34A4C7EB-2798-4482-A786-90161F5F9E34@oracle.com> References: <281642234.3818.1625478269194.JavaMail.zimbra@raptorengineeringinc.com>, <162855621114.22632.14151019687856585770@noble.neil.brown.name>, <20210812144428.GA9536@fieldses.org>, <162880418532.15074.7140645794203395299@noble.neil.brown.name>, , <589AFA4F-DF8E-45A3-8299-54E820E33169@oracle.com>, <20211011143028.GB22387@fieldses.org>, <34A4C7EB-2798-4482-A786-90161F5F9E34@oracle.com> Date: Tue, 12 Oct 2021 08:57:47 +1100 Message-id: <163398946770.17149.14605560717849885454@noble.neil.brown.name> Precedence: bulk List-ID: X-Mailing-List: linux-nfs@vger.kernel.org On Tue, 12 Oct 2021, Chuck Lever III wrote: > > Scott seems well positioned to identify a reproducer. Maybe we > can give him some likely candidates for possible bugs to explore > first. Has this patch been tried? NeilBrown diff --git a/net/sunrpc/sched.c b/net/sunrpc/sched.c index c045f63d11fa..308f5961cb78 100644 --- a/net/sunrpc/sched.c +++ b/net/sunrpc/sched.c @@ -814,6 +814,7 @@ rpc_reset_task_statistics(struct rpc_task *task) { task->tk_timeouts = 0; task->tk_flags &= ~(RPC_CALL_MAJORSEEN|RPC_TASK_SENT); + clear_bit(RPC_TASK_SIGNALLED, &task->tk_runstate); rpc_init_task_statistics(task); }