Received: by 2002:a25:ad19:0:0:0:0:0 with SMTP id y25csp1912783ybi; Thu, 18 Jul 2019 00:13:27 -0700 (PDT) X-Google-Smtp-Source: APXvYqz5NEPm5xxhNCKH1B8mAoJ2Rrpgb36L4VU5oiMPZIegwohdMRFHE31lacIUNgeF9xyNYN1f X-Received: by 2002:a63:c20e:: with SMTP id b14mr44677302pgd.96.1563434007108; Thu, 18 Jul 2019 00:13:27 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1563434007; cv=none; d=google.com; s=arc-20160816; b=bByiX1VQ9qnSbjJAGYJhDb6JLCCEX0e3pJoDdmQZCBEvUMAPKz0f4KG+AfXChx7hLj zGNnS/+od71NNcQ1seUVVkCYEtARd06ScsnGIrKNoKezF/XHa3Qa/iyebFgc2bItkknR QyiSQJhgvRoYw8d9wBt/zR44CM6O+CsOCt7bTYZJ/3YPsAdG3BksS9ulDVe8NnqxC4a1 GCHN5xGsKeWBWwTGVEGyeUJXZR155rLqzxzetl+LVfznMUYYj7z3o6ev5Mhc+XKc1Cg2 eVn4HACceux7i/7MEviglZXiLmlkXPSCQwsWcQu4PqhEvDIbM+JQQ3TtoHgGBl05pZEM gE4A== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:date:cc:to:from:subject:message-id; bh=XKQIQ3HvBvwdx2Y/9xNcGHv9ytTlMuhDX0uXtzwEdmY=; b=NP9+Db0fldS4GCFZYkN4Yvsfhp6ie7QEPRl5NWG0qOW/qt2BGx1IpCLwbnB6JgxzZ9 EI5iA+YALaHRQSfxMGDddkGFtRQyQ92cI3wcXhehvzgio1fWALRw8S0XmCSyizyRLGVW 7FT8Wu8dSo8omw5HzWdvmW3lkwLBCo+Bx2EltloTnJDEr1Jv7t/NTZ7xivcAxWkvSPA1 mgfgP99rdXPKlVoH1s4d+P52HG3hZ1XC62YZZZv2AVk7HoHdSpynVNgr/d+WYB2CN55Q Gldv8BAQchAHSf3xMPbagQ3Fg4qv/Owpm/Kud506zYM3g3R3Lvk8ycUuHdU+76o3BZ1B JCdw== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c14si761133pgc.244.2019.07.18.00.13.09; Thu, 18 Jul 2019 00:13:27 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1726972AbfGRHL1 (ORCPT + 99 others); Thu, 18 Jul 2019 03:11:27 -0400 Received: from gate.crashing.org ([63.228.1.57]:58911 "EHLO gate.crashing.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1726386AbfGRHL1 (ORCPT ); Thu, 18 Jul 2019 03:11:27 -0400 Received: from localhost (localhost.localdomain [127.0.0.1]) by gate.crashing.org (8.14.1/8.14.1) with ESMTP id x6I7B3j6009151; Thu, 18 Jul 2019 02:11:03 -0500 Message-ID: Subject: [PATCH] nvme-pci: Support shared tags across queues for Apple 2018 controllers From: Benjamin Herrenschmidt To: linux-nvme@lists.infradead.org Cc: linux-kernel@vger.kernel.org, Paul Pawlowski , Jens Axboe , Keith Busch , Christoph Hellwig , Minwoo Im Date: Thu, 18 Jul 2019 17:11:02 +1000 In-Reply-To: <2cc90b8cfa935e345ec2b185b087f1859a040176.camel@kernel.crashing.org> References: <20190717004527.30363-1-benh@kernel.crashing.org> <20190717004527.30363-2-benh@kernel.crashing.org> <20190717115145.GB10495@minwoo-desktop> <2cc90b8cfa935e345ec2b185b087f1859a040176.camel@kernel.crashing.org> Content-Type: text/plain; charset="UTF-8" X-Mailer: Evolution 3.28.5-0ubuntu0.18.04.1 Mime-Version: 1.0 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Another issue with the Apple T2 based 2018 controllers seem to be that they blow up (and shut the machine down) if there's a tag collision between the IO queue and the Admin queue. This adds a quirk that offsets all the tags in the IO queue by 32 to avoid those collisions. It also limits the number of IO queues to 1 since the code wouldn't otherwise make sense (the device supports only one queue anyway but better safe than sorry). The bug is typically triggered by tag collisions between SMART commands from smartd and IO commands, often at boot time. Signed-off-by: Benjamin Herrenschmidt --- Note: This is the smallest way I found of doing this that keeps the impact self contained to pci.c. Feel free to suggest alternatives. drivers/nvme/host/nvme.h | 5 +++++ drivers/nvme/host/pci.c | 26 ++++++++++++++++++++------ 2 files changed, 25 insertions(+), 6 deletions(-) diff --git a/drivers/nvme/host/nvme.h b/drivers/nvme/host/nvme.h index 564b967058f4..eeb99e485898 100644 --- a/drivers/nvme/host/nvme.h +++ b/drivers/nvme/host/nvme.h @@ -102,6 +102,11 @@ enum nvme_quirks { * Use non-standard 128 bytes SQEs. */ NVME_QUIRK_128_BYTES_SQES = (1 << 11), + + /* + * Prevent tag overlap between queues + */ + NVME_QUIRK_SHARED_TAGS = (1 << 12), }; /* diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index e399e59863c7..1055f19e57a4 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -194,6 +194,7 @@ struct nvme_queue { u16 cq_head; u16 last_cq_head; u16 qid; + u16 tag_offset; u8 cq_phase; u8 sqes; unsigned long flags; @@ -506,6 +507,7 @@ static void nvme_submit_cmd(struct nvme_queue *nvmeq, struct nvme_command *cmd, bool write_sq) { spin_lock(&nvmeq->sq_lock); + cmd->common.command_id += nvmeq->tag_offset; memcpy(nvmeq->sq_cmds + (nvmeq->sq_tail << nvmeq->sqes), cmd, sizeof(*cmd)); if (++nvmeq->sq_tail == nvmeq->q_depth) @@ -967,9 +969,10 @@ static inline void nvme_ring_cq_doorbell(struct nvme_queue *nvmeq) static inline void nvme_handle_cqe(struct nvme_queue *nvmeq, u16 idx) { volatile struct nvme_completion *cqe = &nvmeq->cqes[idx]; + u16 ctag = cqe->command_id - nvmeq->tag_offset; struct request *req; - if (unlikely(cqe->command_id >= nvmeq->q_depth)) { + if (unlikely(ctag >= nvmeq->q_depth)) { dev_warn(nvmeq->dev->ctrl.device, "invalid id %d completed on queue %d\n", cqe->command_id, le16_to_cpu(cqe->sq_id)); @@ -982,14 +985,13 @@ static inline void nvme_handle_cqe(struct nvme_queue *nvmeq, u16 idx) * aborts. We don't even bother to allocate a struct request * for them but rather special case them here. */ - if (unlikely(nvmeq->qid == 0 && - cqe->command_id >= NVME_AQ_BLK_MQ_DEPTH)) { + if (unlikely(nvmeq->qid == 0 && ctag >= NVME_AQ_BLK_MQ_DEPTH)) { nvme_complete_async_event(&nvmeq->dev->ctrl, cqe->status, &cqe->result); return; } - req = blk_mq_tag_to_rq(*nvmeq->tags, cqe->command_id); + req = blk_mq_tag_to_rq(*nvmeq->tags, ctag); trace_nvme_sq(req, cqe->sq_head, nvmeq->sq_tail); nvme_end_request(req, cqe->status, cqe->result); } @@ -1020,7 +1022,10 @@ static inline int nvme_process_cq(struct nvme_queue *nvmeq, u16 *start, *start = nvmeq->cq_head; while (nvme_cqe_pending(nvmeq)) { - if (tag == -1U || nvmeq->cqes[nvmeq->cq_head].command_id == tag) + u16 ctag = nvmeq->cqes[nvmeq->cq_head].command_id; + + ctag -= nvmeq->tag_offset; + if (tag == -1U || ctag == tag) found++; nvme_update_cq_head(nvmeq); } @@ -1499,6 +1504,10 @@ static int nvme_alloc_queue(struct nvme_dev *dev, int qid, int depth) nvmeq->qid = qid; dev->ctrl.queue_count++; + if (qid && (dev->ctrl.quirks & NVME_QUIRK_SHARED_TAGS)) + nvmeq->tag_offset = NVME_AQ_DEPTH; + else + nvmeq->tag_offset = 0; return 0; free_cqdma: @@ -2110,6 +2119,10 @@ static int nvme_setup_io_queues(struct nvme_dev *dev) unsigned long size; nr_io_queues = max_io_queues(); + + if (dev->ctrl.quirks & NVME_QUIRK_SHARED_TAGS) + nr_io_queues = 1; + result = nvme_set_queue_count(&dev->ctrl, &nr_io_queues); if (result < 0) return result; @@ -2957,7 +2970,8 @@ static const struct pci_device_id nvme_id_table[] = { { PCI_DEVICE(PCI_VENDOR_ID_APPLE, 0x2003) }, { PCI_DEVICE(PCI_VENDOR_ID_APPLE, 0x2005), .driver_data = NVME_QUIRK_SINGLE_VECTOR | - NVME_QUIRK_128_BYTES_SQES }, + NVME_QUIRK_128_BYTES_SQES | + NVME_QUIRK_SHARED_TAGS }, { 0, } }; MODULE_DEVICE_TABLE(pci, nvme_id_table);