Received: by 2002:a05:6a10:206:0:0:0:0 with SMTP id 6csp2032884pxj; Sat, 5 Jun 2021 10:22:46 -0700 (PDT) X-Google-Smtp-Source: ABdhPJwKJ4rSFeaxTWsos3kDqvZjVeWv9J7zRBn0VuZBeXkQ5QauYrX9e5q/MhunOFi2OCWwIq7g X-Received: by 2002:a17:907:20f7:: with SMTP id rh23mr10306120ejb.414.1622913766619; Sat, 05 Jun 2021 10:22:46 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1622913766; cv=none; d=google.com; s=arc-20160816; b=CvUSg50IoMGuEnkRzMQu2XYJiX2IMZe9g6vIwD0qd9VvcNZ+aClq2mFXSiv5mKUQaO WPqJQdR752UQuGljdbWaYdaMPLzkFdNGXwpSx3xAa5c9DNbZXaOhp/FJGLd8kYjN0Ske ZX65m2FECpcJSVGGofZnzxNJTEW+dbbTCOUXQJCHFpV8m0WTfKfzKJ4+eOG+BMpXzMB+ uXnqcvoYsS5vz90gw9lEvU4oBsbcD3CVv37QV03JkZyhXjLp1b3YRy3GiiPczW3hPXh2 GgFS/S86Ls2MsjEbJqXK991Nvn8op2OmT0CZ5EyXhwMAzuLYJW4WAbSzyp2tI3ikfAhe yyrw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=G68YSORxe8rZrlz+rz117NbYUn0YGxl6wX6qgQenpUw=; b=tdwpVDnsYElfbfRxfXYWNIv7+jYgSovQDHYMiPGgNgp0VrmEr3WBVV777VuSpTX5ec 9lsYC6o+rY62gTEEQ/9bLpbZMTPc6W09458kPPMMX+yQpTNUyCWMxoR2V/VMo/yV/oaj 8Cd9QxE8YSG35ek8xz7UWjvwxtvB3For0mDcCBAMxx63LsGimpoJIxPrK96iu112XpT8 ud+c0KwDC9ljJ1717HTF+YeqlJtELQt1EubFRAklT3vTqGGkIyT+9kiv/ktrhLtEO6vc hpz8XCNf8YHhmrbON69gTXK+jAZzO3+rzis2gKE1SULpMJn490U2jc8UPcYPMjaTIY4J d53w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b="hdD/Ydro"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id v4si7742270edc.556.2021.06.05.10.22.23; Sat, 05 Jun 2021 10:22:46 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b="hdD/Ydro"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229982AbhFERVz (ORCPT + 99 others); Sat, 5 Jun 2021 13:21:55 -0400 Received: from mail-ej1-f51.google.com ([209.85.218.51]:38529 "EHLO mail-ej1-f51.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229964AbhFERVy (ORCPT ); Sat, 5 Jun 2021 13:21:54 -0400 Received: by mail-ej1-f51.google.com with SMTP id og14so14102628ejc.5 for ; Sat, 05 Jun 2021 10:20:06 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=G68YSORxe8rZrlz+rz117NbYUn0YGxl6wX6qgQenpUw=; b=hdD/YdrotRvnc1FN/L/7lBNYZ2RqeztUrS3XH58c4O94CyDXf9su2MZauf63DjojhQ p3i0raKHvSiZPJ9tnoFlnQzZ/Efai2sW3g+Wxl9ZH7v4qouQdOoBkUdtF730EFSi42ZN PRFtHqeCMhH+Jj+A1TMjGlN4wtAFjo8b3TkvBXnB1M0cW+ml7fhHYIROwGDKrxfe1wus z1UIhSwMCgiszGGKh6kwnBEJPN0YJn8dCJ9di+KJSBlDofKk5yMlZkGqol51qTQNYzUb 0QLkds4VrCtyR3B2xvDyrKjD18e/+r+C2zhMHw4rZBQ7GDeEO2bqVENMneKeN7ULWi59 wQ0Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version :content-transfer-encoding; bh=G68YSORxe8rZrlz+rz117NbYUn0YGxl6wX6qgQenpUw=; b=UGonMV61PMXWV6Gj0SBUKhfAe9RQewp/o8KivVnJGHjbWnEuTI2iBi/v75jLXU5o1+ ouwblsjCnkXykl4nb92/VwlyVOCZkATAEu5JfB71pkp/Vgsx7K9Aklkjk9aFUtipADCz CVDSdqCANrXExdNlXHH1kf/Joq7rF8LkkuMGIXRTK1ckC9joNGJLfhpgl4ZC7Uxe5s4O uS4BnEASWNcTvtBqTVRFPPkirOkGK05cYvJcq4G/6FIe1at15DQR98jFe1hb75j6ny14 TN96wWj7XeoZC2vQ+qVYbHuT1LoNgxXm2L1GFggMTYbB0u85RxBkK77cQvzlZzCLy4eR WrEw== X-Gm-Message-State: AOAM530NtnyKv1+tBGy2H0ggMU/mbjs6CD7up2qbP9X1PGDqT+1gglqV Yfglqsx4uISmUO2U3DLvCDE= X-Received: by 2002:a17:906:b24a:: with SMTP id ce10mr10060406ejb.83.1622913545503; Sat, 05 Jun 2021 10:19:05 -0700 (PDT) Received: from snuff.lan (178-164-181-11.pool.digikabel.hu. [178.164.181.11]) by smtp.gmail.com with ESMTPSA id r21sm5207360edy.78.2021.06.05.10.19.03 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 05 Jun 2021 10:19:05 -0700 (PDT) From: Sandor Bodo-Merle To: Ray Jui , Scott Branden , bcm-kernel-feedback-list@broadcom.com, Bjorn Helgaas , Sandor Bodo-Merle Cc: =?UTF-8?q?Pali=20Roh=C3=A1r?= , Ray Jui , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org Subject: [PATCH] PCI: iproc: restrict multi-MSI to single core CPUs Date: Sat, 5 Jun 2021 19:17:36 +0200 Message-Id: <20210605171736.15755-1-sbodomerle@gmail.com> X-Mailer: git-send-email 2.31.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Commit fc54bae28818 ("PCI: iproc: Allow allocation of multiple MSIs") introduced multi-MSI support with a broken allocation mechanism (it failed to reserve the proper number of bits from the inner domain). Natural alignment of the base vector number was also not guaranteed. The interrupt affinity scheme used by this driver is incompatible with multi-MSI as implies moving the doorbell address to that of another MSI group. This isn't possible for Multi-MSI, as all the MSIs must have the same doorbell address. As such it is restricted to systems with single CPU core. Fixes: fc54bae28818 ("PCI: iproc: Allow allocation of multiple MSIs") Reported-by: Pali Rohár Signed-off-by: Sandor Bodo-Merle --- drivers/pci/controller/pcie-iproc-msi.c | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git drivers/pci/controller/pcie-iproc-msi.c drivers/pci/controller/pcie-iproc-msi.c index eede4e8f3f75..2e42c460b626 100644 --- drivers/pci/controller/pcie-iproc-msi.c +++ drivers/pci/controller/pcie-iproc-msi.c @@ -171,7 +171,7 @@ static struct irq_chip iproc_msi_irq_chip = { static struct msi_domain_info iproc_msi_domain_info = { .flags = MSI_FLAG_USE_DEF_DOM_OPS | MSI_FLAG_USE_DEF_CHIP_OPS | - MSI_FLAG_MULTI_PCI_MSI | MSI_FLAG_PCI_MSIX, + MSI_FLAG_PCI_MSIX, .chip = &iproc_msi_irq_chip, }; @@ -252,18 +252,15 @@ static int iproc_msi_irq_domain_alloc(struct irq_domain *domain, mutex_lock(&msi->bitmap_lock); - /* Allocate 'nr_cpus' number of MSI vectors each time */ - hwirq = bitmap_find_next_zero_area(msi->bitmap, msi->nr_msi_vecs, 0, - msi->nr_cpus, 0); - if (hwirq < msi->nr_msi_vecs) { - bitmap_set(msi->bitmap, hwirq, msi->nr_cpus); - } else { - mutex_unlock(&msi->bitmap_lock); - return -ENOSPC; - } + /* Allocate 'nr_irqs' multiplied by 'nr_cpus' number of MSI vectors each time */ + hwirq = bitmap_find_free_region(msi->bitmap, msi->nr_msi_vecs, + order_base_2(msi->nr_cpus * nr_irqs)); mutex_unlock(&msi->bitmap_lock); + if (hwirq < 0) + return -ENOSPC; + for (i = 0; i < nr_irqs; i++) { irq_domain_set_info(domain, virq + i, hwirq + i, &iproc_msi_bottom_irq_chip, @@ -284,7 +281,8 @@ static void iproc_msi_irq_domain_free(struct irq_domain *domain, mutex_lock(&msi->bitmap_lock); hwirq = hwirq_to_canonical_hwirq(msi, data->hwirq); - bitmap_clear(msi->bitmap, hwirq, msi->nr_cpus); + bitmap_release_region(msi->bitmap, hwirq, + order_base_2(msi->nr_cpus * nr_irqs)); mutex_unlock(&msi->bitmap_lock); @@ -539,6 +537,9 @@ int iproc_msi_init(struct iproc_pcie *pcie, struct device_node *node) mutex_init(&msi->bitmap_lock); msi->nr_cpus = num_possible_cpus(); + if (msi->nr_cpus == 1) + iproc_msi_domain_info.flags |= MSI_FLAG_MULTI_PCI_MSI; + msi->nr_irqs = of_irq_count(node); if (!msi->nr_irqs) { dev_err(pcie->dev, "found no MSI GIC interrupt\n"); -- 2.31.0