Received: by 2002:a25:d7c1:0:0:0:0:0 with SMTP id o184csp2257994ybg; Sun, 27 Oct 2019 14:34:01 -0700 (PDT) X-Google-Smtp-Source: APXvYqyoP1jxAsiOlfHMjbjyrvD8maB0ZqfnHVbsYlWAPgNDwNvC6yJ9zPra9jgFZqUDf3Ts7DtI X-Received: by 2002:a50:cc43:: with SMTP id n3mr16495106edi.287.1572212041872; Sun, 27 Oct 2019 14:34:01 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1572212041; cv=none; d=google.com; s=arc-20160816; b=eO51FE6ByRVvDwGFM3+pFgCp/rVaPB9X3o1f7gzhmI+txjFdr5hP8g2gtByYZm3cKv fC9SwwfIVLN0MXaWXu4YLIujrfK/BgrROxOdmFiD9oSYvHM54Upw/ZcL++6LK5WB1270 nWAGjaw3wpxsOI+dR5/H+ps6yS2XU6KkyR4uZvJiU+WZ5nCJmSf2VzlHhw8Mu9ZpbAT3 NM+JwXuLp9GjI8KoQSs6U2p6Vh45l+RPHc+etkMPz85/X51uqt9REwLaGNJIF0EaQuZP IbwRj6vS9eAmIe8TyRZVOXwhOQ/PnuF+DYdanZraZlNp+yKLUyuH/QBO7sAIDabK72vh BsCg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=zjjBqTY4ECQv0ZxXURkY2Mbp0Y5w0aDLeU+cLjHwxzk=; b=fRc8Tw4OS3cKz+KyxyLhGwfBuSllWVWnd7A8DJkF2/BwnsRFkuRHFLxzRGYtB0OqPm kyAEke4hsF8t0c7rdf1y7Bavy1O6GA71QYKAjYfHYi8ZhmhQarQ6K1zo51KAKJoDhUfk sgh2YbVZ/o/gTHimeIRx95qkB/thubiyERcyCGH8U5Vd63fTrT+I0Ehpu4+PRt6oySfS AFJXrPKJF2OiVIaWA/Me6rpXBmFm7jqdjzQ8G+x2wTUtaeLKmh7/8u9qv0RpE3xlEnwR bnesFPdkDgjf92bRFXfzMpX3oCdqZiBmabFsq2KtefQH4xOF6D/To4/7+e9ggphnRY62 BgDw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b="jG/b8fSj"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id q40si6016936eda.218.2019.10.27.14.33.37; Sun, 27 Oct 2019 14:34:01 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b="jG/b8fSj"; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1730031AbfJ0VRu (ORCPT + 99 others); Sun, 27 Oct 2019 17:17:50 -0400 Received: from mail.kernel.org ([198.145.29.99]:37520 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731091AbfJ0VRi (ORCPT ); Sun, 27 Oct 2019 17:17:38 -0400 Received: from localhost (100.50.158.77.rev.sfr.net [77.158.50.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id 57E4C2070B; Sun, 27 Oct 2019 21:17:37 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1572211057; bh=R3+/hJX/2rD8flEe2XZwxCvvYNQNQxgQCV+Xkg2vBdw=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=jG/b8fSjQNXNzbVgQggGu2pt3V0hZQ0zUCOy0ZJpzhxeX/Lzz0VIQpLmaFdyPcaHO jFT+JajxhS8NJiNqCRUgTf9+oiVSG9BA8T3Rne2AlxXVQixTr7/TxbCCFw4vhfWHRL rG3RmyddbZ3EwYONSh+L9lHbIgcWwoTHonq2CeS4= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Mario Limonciello , Keith Busch , Sagi Grimberg , Sasha Levin Subject: [PATCH 5.3 018/197] nvme-pci: Save PCI state before putting drive into deepest state Date: Sun, 27 Oct 2019 21:58:56 +0100 Message-Id: <20191027203352.659130294@linuxfoundation.org> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191027203351.684916567@linuxfoundation.org> References: <20191027203351.684916567@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Mario Limonciello [ Upstream commit 7cbb5c6f9aa7cfda7175d82a9cf77a92965b0c5e ] The action of saving the PCI state will cause numerous PCI configuration space reads which depending upon the vendor implementation may cause the drive to exit the deepest NVMe state. In these cases ASPM will typically resolve the PCIe link state and APST may resolve the NVMe power state. However it has also been observed that this register access after quiesced will cause PC10 failure on some device combinations. To resolve this, move the PCI state saving to before SetFeatures has been called. This has been proven to resolve the issue across a 5000 sample test on previously failing disk/system combinations. Signed-off-by: Mario Limonciello Reviewed-by: Keith Busch Signed-off-by: Sagi Grimberg Signed-off-by: Sasha Levin --- drivers/nvme/host/pci.c | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 732d5b63ec054..19458e85dab34 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -2894,11 +2894,21 @@ static int nvme_suspend(struct device *dev) if (ret < 0) goto unfreeze; + /* + * A saved state prevents pci pm from generically controlling the + * device's power. If we're using protocol specific settings, we don't + * want pci interfering. + */ + pci_save_state(pdev); + ret = nvme_set_power_state(ctrl, ctrl->npss); if (ret < 0) goto unfreeze; if (ret) { + /* discard the saved state */ + pci_load_saved_state(pdev, NULL); + /* * Clearing npss forces a controller reset on resume. The * correct value will be resdicovered then. @@ -2906,14 +2916,7 @@ static int nvme_suspend(struct device *dev) nvme_dev_disable(ndev, true); ctrl->npss = 0; ret = 0; - goto unfreeze; } - /* - * A saved state prevents pci pm from generically controlling the - * device's power. If we're using protocol specific settings, we don't - * want pci interfering. - */ - pci_save_state(pdev); unfreeze: nvme_unfreeze(ctrl); return ret; -- 2.20.1