Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20191003191354.GA4481@Serenity> <CAKcoMVC2LdcmUx6j5JzuT-TsFGz=mwQ0MsprrKR2qeXoTmQ-TQ@mail.gmail.com>
 <e0edf48eb84fe038c2912328b28e931900684de2.camel@amazon.com>
In-Reply-To: <e0edf48eb84fe038c2912328b28e931900684de2.camel@amazon.com>
From:   Tyler Ramer <tyaramer@gmail.com>
Date:   Sat, 5 Oct 2019 17:58:52 -0400
Message-ID: <CAKcoMVBjzM6dq33DaTYBK6U32U3NQaR5mf6ppGr6iTKRuZ2KNg@mail.gmail.com>
Subject: Re: [PATCH] nvme-pci: Shutdown when removing dead controller
To:     "Singh, Balbir" <sblbir@amazon.com>
Cc:     "kbusch@kernel.org" <kbusch@kernel.org>, "hch@lst.de" <hch@lst.de>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-nvme@lists.infradead.org" <linux-nvme@lists.infradead.org>,
        "axboe@fb.com" <axboe@fb.com>,
        "sagi@grimberg.me" <sagi@grimberg.me>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

> What is the bad CSTS bit? CSTS.RDY?

The reset will be triggered by the result of nvme_should_reset():

1196 static bool nvme_should_reset(struct nvme_dev *dev, u32 csts)
1197 {
1198
1199 =E2=87=A5       /* If true, indicates loss of adapter communication, p=
ossibly by a
1200 =E2=87=A5        * NVMe Subsystem reset.
1201 =E2=87=A5        */
1202 =E2=87=A5       bool nssro =3D dev->subsystem && (csts & NVME_CSTS_NSS=
RO);

This csts value is set in nvme_timeout:

1240 static enum blk_eh_timer_return nvme_timeout(struct request *req,
bool reserved)
1241 {
...
1247 =E2=87=A5       u32 csts =3D readl(dev->bar + NVME_REG_CSTS);
...
1256 =E2=87=A5       /*
1257 =E2=87=A5        * Reset immediately if the controller is failed
1258 =E2=87=A5        */
1259 =E2=87=A5       if (nvme_should_reset(dev, csts)) {
1260 =E2=87=A5       =E2=87=A5       nvme_warn_reset(dev, csts);
1261 =E2=87=A5       =E2=87=A5       nvme_dev_disable(dev, false);
1262 =E2=87=A5       =E2=87=A5       nvme_reset_ctrl(&dev->ctrl);


Again, here's the message printed by nvme_warn_reset:

Aug 26 15:01:27 testhost kernel: nvme nvme4: controller is down; will
reset: CSTS=3D0x3, PCI_STATUS=3D0x10

From  include/linux/nvme.h:
 105 =E2=87=A5       NVME_REG_CSTS=E2=87=A5  =3D 0x001c,=E2=87=A5      /* C=
ontroller Status */

- Tyler