Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 16 Aug 2023 15:04:17 -0600
From:   Keith Busch <kbusch@kernel.org>
To:     Genes Lists <lists@sapience.com>
Cc:     linux-kernel@vger.kernel.org, axboe@kernel.dk, sagi@grimberg.me,
        linux-nvme@lists.infradead.org, hch@lst.de
Subject: Re: Possible nvme regression in 6.4.11
Message-ID: <ZN050TFnKZ54LJ5v@kbusch-mbp.dhcp.thefacebook.com>
References: <5f968b95-6b1c-4d6f-aac7-5d54f66834a8@sapience.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <5f968b95-6b1c-4d6f-aac7-5d54f66834a8@sapience.com>
Precedence: bulk

On Wed, Aug 16, 2023 at 04:39:34PM -0400, Genes Lists wrote:
> Also reported to bugzilla [1]
> 
> Failure happens on 1 laptop with samsung ssd.
> 
> Boot log manually transcribed:
> 
> kernel: nvme nvme0: controller is down; will reset: CSTS:0xffffffff,
> PCI_STATUS=0xffff
> kernel: nvme nvme0: Does your device have a faulty power saving mode
> enabled?
> kernel: nvme nvme0: try "nvme_core.default_ps_max_latency_us=0
> pcie_aspm=off" and report a bug
> kernel: nvme 0000:04:00.0: Unable to change power state from D3cold to D0,
> device inaccessible
> kernel: nvme nvme0: Disabling device after reset failure: -19
> mount[353]: mount /sysroot: can't read suprtblock on /dev/nvme0n1p5.
> mount[353]:       dmesg(1) may have more information after failed moutn
> system call.
> kernel: nvme0m1: detected capacity change from 2000409264 to 0
> kernel: EXT4-fs (nvme0n1p5): unable to read superblock
> systemd([1]: sysroot.mount: Mount process exited, code=exited, status=32/n/a
> ...
> 
> All kernels are upstream, untainted and compiled on Arch using:
> 
>  gcc version 13.2.1
> 
> Kernels Tested:
>  - 6.4.10 - works fine
>  - 6.4.11 - fails
>  - 6.5-rc6 - fails
>  - 6.4.11 + nvme_core.default_ps_max_latency_us=0 pcie_aspm=off - fails
>  - 6.4.11 with 1 revert below - fails
> 
>     Revert "nvme-pci: add NVME_QUIRK_BOGUS_NID for Samsung PM9B1 256G and
> 512G"
>     This reverts commit 061fbf64825fb47367bbb6e0a528611f08119473.

It sounds like you can recreate this. Since .10 worked and .11 doesn't,
could you bisect the git commits? It looks like it will take 7 steps
between those two versions.

I don't think there are any nvme specific patches that could contribute
to what you're seeing, it's more likely some lower level platform patch
if a kernel change really did cause the regression. None of the recent
commits really stood out to me, so bisect is what I'd recommend.