Received: by 2002:ac0:946b:0:0:0:0:0 with SMTP id j40csp122638imj; Wed, 13 Feb 2019 05:46:52 -0800 (PST) X-Google-Smtp-Source: AHgI3IaZyJhzsscb9S97/Cm13k2h9Ejnxk1FIQ/hHt1flPb/axljO+uyzSuWmsHwABS018J4dEKF X-Received: by 2002:a63:4703:: with SMTP id u3mr564673pga.298.1550065612822; Wed, 13 Feb 2019 05:46:52 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1550065612; cv=none; d=google.com; s=arc-20160816; b=kmyo9wEFb6IfICPQF7nwcsnhoQfpGh0d9KjxR00madrkd5Lq9bk/jshihG3reGH4Ed S6mrYn5KM4dj7Po648uoq5RGellJS83xqa2F9RdCTTryWwvkwql6oibKPW0a82QAnTT7 BWnnY5Nx5zpKWRpw+ZDUWkgT/ao5rSFfFT+lozPlngxvTLYChhhXD6J6YCDutuhUCdZC sMa9G7SfX96ybBMetbpqt56/hpdg5DBExLkO1+2KfZv2dApwBgUK8LV5tKICzbcLejnr o/ZBeq69RrsuMwFt438g6nkfaThNVIvfaWgct+pZcJf93kiQN8hUbPsYke+sNccT59Yd ICyw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=Rsn6O9rKUiABrA8W67IbQ3jzmRzTRRzugKsgWuiuQZs=; b=T+Z76VZDurjlYBXJz4JmFzVhPI+OHGuDDsdqLjRoxxCtgcfqUQm0TpQM4ephn3eV7r xmK3Rq/Zsi+VnRrYVYddJm1i8bp2XlRFd02TQ3IPIfo/N2Z5uxFKnOT1S6GLXINmg9Ny mLeVxmAw+iyLt3YtM5/s6coJZydxTT41aSbQ/rDB9kNXBq+s0vIW8dR7y5bAkNdKwBCC 3ATt4XweQNoOXERMwf/2IRKOb4IjP7ZyB7pHh1E4UXB9xlzTpsoB4+hwn52rfXsrXfBX +DnwKsUykY9xZnFJhonyE3yzLAINtV4LLOCswlqt0nmdQ5Mresn3cKgCPwydKjqxObWA HREg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id d2si11431561pfn.180.2019.02.13.05.46.32; Wed, 13 Feb 2019 05:46:52 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=redhat.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2391635AbfBMKvT (ORCPT + 99 others); Wed, 13 Feb 2019 05:51:19 -0500 Received: from mx1.redhat.com ([209.132.183.28]:33298 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S2387957AbfBMKvS (ORCPT ); Wed, 13 Feb 2019 05:51:18 -0500 Received: from smtp.corp.redhat.com (int-mx04.intmail.prod.int.phx2.redhat.com [10.5.11.14]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by mx1.redhat.com (Postfix) with ESMTPS id 76A03325AF; Wed, 13 Feb 2019 10:51:18 +0000 (UTC) Received: from localhost (ovpn-8-32.pek2.redhat.com [10.72.8.32]) by smtp.corp.redhat.com (Postfix) with ESMTP id 435755D9C6; Wed, 13 Feb 2019 10:51:14 +0000 (UTC) From: Ming Lei To: Christoph Hellwig , Bjorn Helgaas , Thomas Gleixner Cc: Jens Axboe , linux-block@vger.kernel.org, Sagi Grimberg , linux-nvme@lists.infradead.org, linux-kernel@vger.kernel.org, linux-pci@vger.kernel.org, Keith Busch , Ming Lei Subject: [PATCH V3 4/5] nvme-pci: avoid irq allocation retrying via .calc_sets Date: Wed, 13 Feb 2019 18:50:40 +0800 Message-Id: <20190213105041.13537-5-ming.lei@redhat.com> In-Reply-To: <20190213105041.13537-1-ming.lei@redhat.com> References: <20190213105041.13537-1-ming.lei@redhat.com> X-Scanned-By: MIMEDefang 2.79 on 10.5.11.14 X-Greylist: Sender IP whitelisted, not delayed by milter-greylist-4.5.16 (mx1.redhat.com [10.5.110.28]); Wed, 13 Feb 2019 10:51:18 +0000 (UTC) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Currently pre-caculate each set vectors, and this way requires same 'max_vecs' and 'min_vecs' passed to pci_alloc_irq_vectors_affinity(), then nvme_setup_irqs() has to retry in case of allocation failure. This usage & interface is a bit awkward because the retry should have been avoided by providing one reasonable 'min_vecs'. Implement the callback of .calc_sets, so that pci_alloc_irq_vectors_affinity() can calculate each set's vector after IRQ vectors is allocated and before spread IRQ, then NVMe's retry in case of irq allocation failure can be removed. Reviewed-by: Keith Busch Signed-off-by: Ming Lei --- drivers/nvme/host/pci.c | 62 +++++++++++++------------------------------------ 1 file changed, 16 insertions(+), 46 deletions(-) diff --git a/drivers/nvme/host/pci.c b/drivers/nvme/host/pci.c index 0086bdf80ea1..8c51252a897e 100644 --- a/drivers/nvme/host/pci.c +++ b/drivers/nvme/host/pci.c @@ -2078,14 +2078,25 @@ static void nvme_calc_io_queues(struct nvme_dev *dev, unsigned int irq_queues) } } +static void nvme_calc_irq_sets(struct irq_affinity *affd, int nvecs) +{ + struct nvme_dev *dev = affd->priv; + + nvme_calc_io_queues(dev, nvecs); + + affd->set_vectors[HCTX_TYPE_DEFAULT] = dev->io_queues[HCTX_TYPE_DEFAULT]; + affd->set_vectors[HCTX_TYPE_READ] = dev->io_queues[HCTX_TYPE_READ]; + affd->nr_sets = 2; +} + static int nvme_setup_irqs(struct nvme_dev *dev, unsigned int nr_io_queues) { struct pci_dev *pdev = to_pci_dev(dev->dev); struct irq_affinity affd = { .pre_vectors = 1, - .nr_sets = 2, + .calc_sets = nvme_calc_irq_sets, + .priv = dev, }; - int *irq_sets = affd.set_vectors; int result = 0; unsigned int irq_queues, this_p_queues; @@ -2102,50 +2113,8 @@ static int nvme_setup_irqs(struct nvme_dev *dev, unsigned int nr_io_queues) } dev->io_queues[HCTX_TYPE_POLL] = this_p_queues; - /* - * For irq sets, we have to ask for minvec == maxvec. This passes - * any reduction back to us, so we can adjust our queue counts and - * IRQ vector needs. - */ - do { - nvme_calc_io_queues(dev, irq_queues); - irq_sets[0] = dev->io_queues[HCTX_TYPE_DEFAULT]; - irq_sets[1] = dev->io_queues[HCTX_TYPE_READ]; - if (!irq_sets[1]) - affd.nr_sets = 1; - - /* - * If we got a failure and we're down to asking for just - * 1 + 1 queues, just ask for a single vector. We'll share - * that between the single IO queue and the admin queue. - * Otherwise, we assign one independent vector to admin queue. - */ - if (irq_queues > 1) - irq_queues = irq_sets[0] + irq_sets[1] + 1; - - result = pci_alloc_irq_vectors_affinity(pdev, irq_queues, - irq_queues, - PCI_IRQ_ALL_TYPES | PCI_IRQ_AFFINITY, &affd); - - /* - * Need to reduce our vec counts. If we get ENOSPC, the - * platform should support mulitple vecs, we just need - * to decrease our ask. If we get EINVAL, the platform - * likely does not. Back down to ask for just one vector. - */ - if (result == -ENOSPC) { - irq_queues--; - if (!irq_queues) - return result; - continue; - } else if (result == -EINVAL) { - irq_queues = 1; - continue; - } else if (result <= 0) - return -EIO; - break; - } while (1); - + result = pci_alloc_irq_vectors_affinity(pdev, 1, irq_queues, + PCI_IRQ_ALL_TYPES | PCI_IRQ_AFFINITY, &affd); return result; } @@ -3021,6 +2990,7 @@ static struct pci_driver nvme_driver = { static int __init nvme_init(void) { + BUILD_BUG_ON(2 > IRQ_MAX_SETS); return pci_register_driver(&nvme_driver); } -- 2.9.5