2021-05-21 20:19:10

by Daniel Wagner

[permalink] [raw]
Subject: [PATCH] nvme: Use NN for max_namespaces if MNAN is zero

NVMe 1.4 states that MNAN might be zero, in which case we should
evaluate NN to get the number of supported namespaces.

Signed-off-by: Daniel Wagner <[email protected]>
---
drivers/nvme/host/core.c | 2 ++
1 file changed, 2 insertions(+)

diff --git a/drivers/nvme/host/core.c b/drivers/nvme/host/core.c
index e7441ccaa8db..32457c77c9ab 100644
--- a/drivers/nvme/host/core.c
+++ b/drivers/nvme/host/core.c
@@ -2905,6 +2905,8 @@ static int nvme_init_identify(struct nvme_ctrl *ctrl)
ctrl->sgls = le32_to_cpu(id->sgls);
ctrl->kas = le16_to_cpu(id->kas);
ctrl->max_namespaces = le32_to_cpu(id->mnan);
+ if (!ctrl->max_namespaces)
+ ctrl->max_namespaces = le32_to_cpu(id->nn);
ctrl->ctratt = le32_to_cpu(id->ctratt);

if (id->rtd3e) {
--
2.29.2


2021-05-21 20:19:23

by Daniel Wagner

[permalink] [raw]
Subject: Re: [PATCH] nvme: Use NN for max_namespaces if MNAN is zero

On Fri, May 21, 2021 at 04:47:34PM +0200, Daniel Wagner wrote:
> NVMe 1.4 states that MNAN might be zero, in which case we should
> evaluate NN to get the number of supported namespaces.
>
> Signed-off-by: Daniel Wagner <[email protected]>
> ---

Forgot to mention: During testing dynamically adding namespaces it was
possible to trigger the WARNINGs in the nvme_parse_ana_log(). Initially
the subsystem started with 8 namespaces and during runtime another 8
namespaces was added.

WARNING: CPU: 3 PID: 3990 at ../drivers/nvme/host/multipath.c:464 nvme_parse_ana_log+0x15f/0x180 [nvme_core]
Modules linked in: nls_utf8 isofs af_packet iscsi_ibft iscsi_boot_sysfs rfkill intel_rapl_msr iTCO_wdt intel_pmc_bxt iTCO_vendor_support dcdbas(XX) intel_rapl_common sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel nls_iso8859_1 nls_cp437 vfat fat xfs kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel crypto_simd cryptd glue_helper pcspkr mgag200 joydev drm_kms_helper cec rc_core syscopyarea sysfillrect sysimgblt fb_sys_fops ipmi_ssif mei_me mei igb lpc_ich i2c_algo_bit dca ipmi_si ipmi_devintf ipmi_msghandler button drm fuse btrfs libcrc32c xor raid6_pq sr_mod cdrom sd_mod hid_generic usbhid uas usb_storage lpfc(OEXX) nvmet_fc nvmet configfs ehci_pci ehci_hcd nvme_fc ahci nvme_fabrics libahci nvme_core crc32c_intel libata usbcore scsi_transport_fc megaraid_sas t10_pi wmi sg dm_multipath dm_mod scsi_dh_rdac scsi_dh_emc scsi_dh_alua scsi_mod msr efivarfs
Supported: Yes, External
CPU: 3 PID: 3990 Comm: kworker/u82:4 Kdump: loaded Tainted: G OE X 5.3.18-56-default #1 SLE15-SP3
Hardware name: Dell Inc. PowerEdge R630/0CNCJW, BIOS 2.11.0 11/02/2019
Workqueue: nvme-wq nvme_scan_work [nvme_core]
RIP: 0010:nvme_parse_ana_log+0x15f/0x180 [nvme_core]
Code: 48 83 c4 08 b8 ea ff ff ff 5b 5d 41 5c 41 5d 41 5e 41 5f c3 0f 0b 48 83 c4 08 b8 ea ff ff ff 5b 5d 41 5c 41 5d 41 5e 41 5f c3 <0f> 0b 48 83 c4 08 b8 ea ff ff ff 5b 5d 41 5c 41 5d 41 5e 41 5f c3
RSP: 0018:ffffa59f495cbca0 EFLAGS: 00010283
RAX: 0000000000000010 RBX: ffff98bacb7bd470 RCX: 0000000000000028
RDX: 0000000000000001 RSI: ffffa59f495cbce0 RDI: ffff98bacb7bd470
RBP: 0000000000000030 R08: 0000000000000001 R09: 0000000000000228
R10: ffff98c2b1caaa10 R11: 000000000001d900 R12: 0000000000000040
R13: 0000000000000000 R14: ffffffffc02a9000 R15: ffff98c2b1caaa00
FS: 0000000000000000(0000) GS:ffff98c2ff840000(0000) knlGS:0000000000000000
CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
CR2: 000055a303a6a5e0 CR3: 00000006e820a004 CR4: 00000000001706e0
Call Trace:
nvme_mpath_add_disk+0x81/0x100 [nvme_core]
nvme_validate_ns+0x3d4/0x900 [nvme_core]
nvme_scan_work+0x155/0x2d0 [nvme_core]
process_one_work+0x1f4/0x3e0
worker_thread+0x2d/0x3e0
? process_one_work+0x3e0/0x3e0
kthread+0x10d/0x130
? kthread_park+0xa0/0xa0
ret_from_fork+0x35/0x40


2021-05-21 20:20:33

by Keith Busch

[permalink] [raw]
Subject: Re: [PATCH] nvme: Use NN for max_namespaces if MNAN is zero

On Fri, May 21, 2021 at 04:53:06PM +0200, Daniel Wagner wrote:
> On Fri, May 21, 2021 at 04:47:34PM +0200, Daniel Wagner wrote:
> > NVMe 1.4 states that MNAN might be zero, in which case we should
> > evaluate NN to get the number of supported namespaces.
> >
> > Signed-off-by: Daniel Wagner <[email protected]>
> > ---
>
> Forgot to mention: During testing dynamically adding namespaces it was
> possible to trigger the WARNINGs in the nvme_parse_ana_log(). Initially
> the subsystem started with 8 namespaces and during runtime another 8
> namespaces was added.

The controller is required to have a non-zero MNAN value if it supports
ANA:

If the controller supports Asymmetric Namespace Access Reporting, then
this field shall be set to a non-zero value that is less than or equal
to the NN value.

> WARNING: CPU: 3 PID: 3990 at ../drivers/nvme/host/multipath.c:464 nvme_parse_ana_log+0x15f/0x180 [nvme_core]
> Modules linked in: nls_utf8 isofs af_packet iscsi_ibft iscsi_boot_sysfs rfkill intel_rapl_msr iTCO_wdt intel_pmc_bxt iTCO_vendor_support dcdbas(XX) intel_rapl_common sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel nls_iso8859_1 nls_cp437 vfat fat xfs kvm irqbypass crc32_pclmul ghash_clmulni_intel aesni_intel crypto_simd cryptd glue_helper pcspkr mgag200 joydev drm_kms_helper cec rc_core syscopyarea sysfillrect sysimgblt fb_sys_fops ipmi_ssif mei_me mei igb lpc_ich i2c_algo_bit dca ipmi_si ipmi_devintf ipmi_msghandler button drm fuse btrfs libcrc32c xor raid6_pq sr_mod cdrom sd_mod hid_generic usbhid uas usb_storage lpfc(OEXX) nvmet_fc nvmet configfs ehci_pci ehci_hcd nvme_fc ahci nvme_fabrics libahci nvme_core crc32c_intel libata usbcore scsi_transport_fc megaraid_sas t10_pi wmi sg dm_multipath dm_mod scsi_dh_rdac scsi_dh_emc scsi_dh_alua scsi_mod msr efivarfs
> Supported: Yes, External
> CPU: 3 PID: 3990 Comm: kworker/u82:4 Kdump: loaded Tainted: G OE X 5.3.18-56-default #1 SLE15-SP3
> Hardware name: Dell Inc. PowerEdge R630/0CNCJW, BIOS 2.11.0 11/02/2019
> Workqueue: nvme-wq nvme_scan_work [nvme_core]
> RIP: 0010:nvme_parse_ana_log+0x15f/0x180 [nvme_core]
> Code: 48 83 c4 08 b8 ea ff ff ff 5b 5d 41 5c 41 5d 41 5e 41 5f c3 0f 0b 48 83 c4 08 b8 ea ff ff ff 5b 5d 41 5c 41 5d 41 5e 41 5f c3 <0f> 0b 48 83 c4 08 b8 ea ff ff ff 5b 5d 41 5c 41 5d 41 5e 41 5f c3
> RSP: 0018:ffffa59f495cbca0 EFLAGS: 00010283
> RAX: 0000000000000010 RBX: ffff98bacb7bd470 RCX: 0000000000000028
> RDX: 0000000000000001 RSI: ffffa59f495cbce0 RDI: ffff98bacb7bd470
> RBP: 0000000000000030 R08: 0000000000000001 R09: 0000000000000228
> R10: ffff98c2b1caaa10 R11: 000000000001d900 R12: 0000000000000040
> R13: 0000000000000000 R14: ffffffffc02a9000 R15: ffff98c2b1caaa00
> FS: 0000000000000000(0000) GS:ffff98c2ff840000(0000) knlGS:0000000000000000
> CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> CR2: 000055a303a6a5e0 CR3: 00000006e820a004 CR4: 00000000001706e0
> Call Trace:
> nvme_mpath_add_disk+0x81/0x100 [nvme_core]
> nvme_validate_ns+0x3d4/0x900 [nvme_core]
> nvme_scan_work+0x155/0x2d0 [nvme_core]
> process_one_work+0x1f4/0x3e0
> worker_thread+0x2d/0x3e0
> ? process_one_work+0x3e0/0x3e0
> kthread+0x10d/0x130
> ? kthread_park+0xa0/0xa0
> ret_from_fork+0x35/0x40

2021-05-21 20:29:56

by Sagi Grimberg

[permalink] [raw]
Subject: Re: [PATCH] nvme: Use NN for max_namespaces if MNAN is zero


>> Forgot to mention: During testing dynamically adding namespaces it was
>> possible to trigger the WARNINGs in the nvme_parse_ana_log(). Initially
>> the subsystem started with 8 namespaces and during runtime another 8
>> namespaces was added.
>
> The controller is required to have a non-zero MNAN value if it supports
> ANA:
>
> If the controller supports Asymmetric Namespace Access Reporting, then
> this field shall be set to a non-zero value that is less than or equal
> to the NN value.

That was my thought exactly

2021-05-24 07:38:27

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH] nvme: Use NN for max_namespaces if MNAN is zero

On Fri, May 21, 2021 at 01:19:26PM -0700, Sagi Grimberg wrote:
>
>>> Forgot to mention: During testing dynamically adding namespaces it was
>>> possible to trigger the WARNINGs in the nvme_parse_ana_log(). Initially
>>> the subsystem started with 8 namespaces and during runtime another 8
>>> namespaces was added.
>>
>> The controller is required to have a non-zero MNAN value if it supports
>> ANA:
>>
>> If the controller supports Asymmetric Namespace Access Reporting, then
>> this field shall be set to a non-zero value that is less than or equal
>> to the NN value.
>
> That was my thought exactly

I think we should add a sanity check for that and reject the broken
controller if that is not the case rather than working around it.

2021-05-25 07:23:43

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH] nvme: Use NN for max_namespaces if MNAN is zero

On Tue, May 25, 2021 at 09:12:59AM +0200, Daniel Wagner wrote:
> On Mon, May 24, 2021 at 09:37:03AM +0200, Christoph Hellwig wrote:
> > I think we should add a sanity check for that and reject the broken
> > controller if that is not the case rather than working around it.
>
> Alright. I understood from the spec, that in the non ANA case the NN
> field should still be case though?

For non-ANA MNAN doesn't have to be set indeed. But we also don't use
the value at all either.

2021-05-25 07:53:37

by Daniel Wagner

[permalink] [raw]
Subject: Re: [PATCH] nvme: Use NN for max_namespaces if MNAN is zero

On Tue, May 25, 2021 at 09:22:34AM +0200, Christoph Hellwig wrote:
> For non-ANA MNAN doesn't have to be set indeed. But we also don't use
> the value at all either.

Ah yes, I somehow though we still allocate the log buffer. Obviously, the
buffer is only used with ANA... time to fetch a coffee.

2021-05-25 08:23:20

by Daniel Wagner

[permalink] [raw]
Subject: Re: [PATCH] nvme: Use NN for max_namespaces if MNAN is zero

On Mon, May 24, 2021 at 09:37:03AM +0200, Christoph Hellwig wrote:
> I think we should add a sanity check for that and reject the broken
> controller if that is not the case rather than working around it.

Alright. I understood from the spec, that in the non ANA case the NN
field should still be case though?