Received-SPF: pass (google.com: domain of linux-kernel+bounces-153790-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Message-ID: <dfa3124d-f2e0-4a57-ae1b-618552711b8d@linux.dev>
Date: Mon, 22 Apr 2024 13:15:38 -0400
Precedence: bulk
MIME-Version: 1.0
Subject: Re: [PATCH] nvme-pci: Add quirk for broken MSIs
To: Keith Busch <kbusch@kernel.org>
Cc: Jens Axboe <axboe@kernel.dk>, Christoph Hellwig <hch@lst.de>,
 Sagi Grimberg <sagi@grimberg.me>, linux-nvme@lists.infradead.org,
 linux-kernel@vger.kernel.org, stable@vger.kernel.org
References: <20240422162822.3539156-1-sean.anderson@linux.dev>
 <ZiaVEfdUXO97eWzV@kbusch-mbp.dhcp.thefacebook.com>
Content-Language: en-US
From: Sean Anderson <sean.anderson@linux.dev>
In-Reply-To: <ZiaVEfdUXO97eWzV@kbusch-mbp.dhcp.thefacebook.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit

On 4/22/24 12:49, Keith Busch wrote:
> On Mon, Apr 22, 2024 at 12:28:23PM -0400, Sean Anderson wrote:
>> Sandisk SN530 NVMe drives have broken MSIs. On systems without MSI-X
>> support, all commands time out resulting in the following message:
>> 
>> nvme nvme0: I/O tag 12 (100c) QID 0 timeout, completion polled
>> 
>> These timeouts cause the boot to take an excessively-long time (over 20
>> minutes) while the initial command queue is flushed.
>> 
>> Address this by adding a quirk for drives with buggy MSIs. The lspci
>> output for this device (recorded on a system with MSI-X support) is:
> 
> Based on your description, the patch looks good. This will fallback to
> legacy emulated pin interrupts, and that's better than timeout polling,
> but will still appear sluggish compared to MSI's. Is there an errata
> from the vendor on this? I'm just curious if the bug is at the Device ID
> level, and not something we could constrain to a particular model or
> firmware revision. 

I wasn't able to find any errata for this drive. I wasn't able to
determine if there are any firmware updates for this drive (FWIW I have
version "21160001"). I'll contact WD and see if they know about this
issue.

[1] https://www.westerndigital.com/products/internal-drives/pc-sn530-ssd

>> 02:00.0 Non-Volatile memory controller: Sandisk Corp Device 5008 (rev 01) (prog-if 02 [NVM Express])
>> 	Subsystem: Sandisk Corp Device 5008
>> 	Flags: bus master, fast devsel, latency 0, IRQ 16, NUMA node 0
>> 	Memory at f7e00000 (64-bit, non-prefetchable) [size=16K]
>> 	Memory at f7e04000 (64-bit, non-prefetchable) [size=256]
>> 	Capabilities: [80] Power Management version 3
>> 	Capabilities: [90] MSI: Enable- Count=1/32 Maskable- 64bit+
>> 	Capabilities: [b0] MSI-X: Enable+ Count=17 Masked-
> 
> Interesting, the MSI capability does look weird here. I've never seen
> MSI-x count smaller than the MSI's. As long as both work, though, I
> think nvme would actually prefer whichever is bigger!

--Sean