Received: by 2002:a05:7412:e794:b0:fa:551:50a7 with SMTP id o20csp1170286rdd; Wed, 10 Jan 2024 10:40:38 -0800 (PST) X-Google-Smtp-Source: AGHT+IFqy5d1l1A4wUu+RanXCJNgAL4mCODwhS+epAonEnkvJcTZhLsm1RnIJLpEtlgl4WBNivFz X-Received: by 2002:a17:90a:cf01:b0:28c:815e:e0f0 with SMTP id h1-20020a17090acf0100b0028c815ee0f0mr782469pju.37.1704912038461; Wed, 10 Jan 2024 10:40:38 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1704912038; cv=none; d=google.com; s=arc-20160816; b=PaRozlMcSHoMaa/83nV3s8qAOXqNMKUldR/nlt3zEkckyJ1IykCdmgeaSCIZqJTXJK 0DUua4O7wbtbl+e3LBoc+15HTpWNC7355obuLTf7CPxntKFRBYlfqeTOMIHZt90CDAMk SqzFwEo3fYB1VsmDILdaqyCiHl/H17nqYoHi9C2XGOqrS9nop71Y+5IirA16pyvpefm/ OezJERgBXHt4HmNR8CvMVQQZRb3GaYCSH3z0I+M0zgSiu0/ghZ1ShRsXPzsfg3M9h651 EMNSqc8RUdqzEHzZHdkI308VS5duf5cO9mgGzKxJn3RLlOVsV4Nib+bTRWcgLWaUrKvR qKrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=in-reply-to:content-disposition:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:message-id:subject:cc :to:from:date:dkim-signature; bh=1Bisy52+4v+u+GwuKg5Tzrfw8JADYsFRUtOE99jsZPQ=; fh=dRNoo9m/2IU3l+z4Uslpt8YKKkvPsAG/3e6C/IkHSpU=; b=kI6ao/4+AVOMBvPuibSU6f6UXU9sVJzSFtaqYtrPp7XFSwcWfE7RblBpiI/jvG2WEr lcPAdaVvs9BQ35V1V5L3OusEk44f7Ee9QQzZHkURdCtjyh/EdyYJ/EmdD6I7Ql/d8jmN c3rfYiwQGHvneIOUnNbb5RazJMv8yRMHlJJnxi5OLZRQMGzR4S66xa9pTl0rHTqWgmaC Jc7PmwV4EiZCVxwOPH/WcacqQ8V0AfxFbZMlYogppb9/QzizcpocqdV9x6cj6gtEVaOm GDhV11rE78jeRHCMUWIFklSjkZW6I7T/VguQ3zUu6usujF1tC6LJ6vqaDQ9VzjtgiRQQ Tvog== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=L6QA385J; spf=pass (google.com: domain of linux-kernel+bounces-22607-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-22607-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Return-Path: Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id qa4-20020a17090b4fc400b0028c28cff71dsi2162479pjb.154.2024.01.10.10.40.38 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 10 Jan 2024 10:40:38 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel+bounces-22607-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=k20201202 header.b=L6QA385J; spf=pass (google.com: domain of linux-kernel+bounces-22607-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-kernel+bounces-22607-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=kernel.org Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id 2AA6C289405 for ; Wed, 10 Jan 2024 18:40:38 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 01E5E4D588; Wed, 10 Jan 2024 18:40:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=kernel.org header.i=@kernel.org header.b="L6QA385J" Received: from smtp.kernel.org (aws-us-west-2-korg-mail-1.web.codeaurora.org [10.30.226.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 244754CDEB for ; Wed, 10 Jan 2024 18:40:31 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0CCB3C433F1; Wed, 10 Jan 2024 18:40:30 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1704912031; bh=1Bisy52+4v+u+GwuKg5Tzrfw8JADYsFRUtOE99jsZPQ=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=L6QA385JFf7m8iRGMu5ALTiAL/XVPYKPMviprK2QJZHYbktlYrV+nwLKQ/Cs0Hc7A rBDwO2tSiUPsVmFysnIoNeW6CfAvy/1ZcqgvJD4oVJwGrP/4i9YB1VbLXRofhJPnmW Sc/i2T65Cmmo4zidwHPfbpJQfuF3JQwn5q2v8JAoR/EqWAQoF+grcNsmQbIx/Fenz8 nOHBVBRLx++duafZnUBwortRsjkArIqJ9VbfCpO9/FNLmdyBOQZdNgLaDGMXeuODiZ dsjCPHMdEARdX4uUWqi2PNKtPSe4aK+4Rzgpp5seXeZ2ytpovcxeIDpK+Mg9KXM8AY 0TUGgv/2L8Geg== Date: Wed, 10 Jan 2024 11:40:28 -0700 From: Keith Busch To: William Butler Cc: Pavel Emelyanov , Christoph Hellwig , Sagi Grimberg , Jens Axboe , linux-kernel@vger.kernel.org, linux-nvme@lists.infradead.org Subject: Re: [PATCH] nvme: Inform controller of doorbell config before unquiescing IO queues Message-ID: References: <20240110182855.1833512-1-wab@google.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20240110182855.1833512-1-wab@google.com> On Wed, Jan 10, 2024 at 06:28:55PM +0000, William Butler wrote: > During resets, if queues are unquiesced first, then the host can submit > IOs to the controller using shadow doorbell logic but the controller > won't be aware. This can lead to necessary MMIO doorbells from being > not issued, causing requests to be delayed and timed-out. Good catch. I guess we wouldn't normally expect any requests to be in queue at this point, but plenty of reasons it could happen. Applied to nvme-6.8 with a slightly updated title to match the local conventions.