Provide support for device specific MSI implementations for devices which
have their own resource management and interrupt chip. These devices are
not related to PCI and contrary to platform MSI they do not share a
common resource and interrupt chip. They provide their own domain specific
resource management and interrupt chip.
On top of this, add support for Interrupt Message Store or IMS[1], which
is a scalable device specific interrupt mechanism to support devices which
may need more than 2048 interrupts. With IMS, there is theoretically no
upper bound on the number of interrupts a device can support. The normal
IMS use case is for guest usages but it can very well be used by host too.
Introduce a generic IMS irq chip and domain which stores the interrupt
messages as an array in device memory. Allocation and freeing of interrupts
happens via the generic msi_domain_alloc/free_irqs() interface. One only
needs to ensure the interrupt domain is stored in the underlying device struct.
These patches can be divided into following steps:
1. X86 specific preparation for device MSI
x86/irq: Add DEV_MSI allocation type
x86/msi: Rename and rework pci_msi_prepare() to cover non-PCI MSI
2. Generic device MSI infrastructure
platform-msi: Provide default irq_chip:: Ack
genirq/proc: Take buslock on affinity write
genirq/msi: Provide and use msi_domain_set_default_info_flags()
platform-msi: Add device MSI infrastructure
irqdomain/msi: Provide msi_alloc/free_store() callbacks
genirq: Set auxiliary data for an interrupt
genirq/msi: Provide helpers to return Linux IRQ/dev_msi hw IRQ number
3. Interrupt Message Store (IMS) irq domain/chip implementation for device array
irqchip: Add IMS (Interrupt Message Store) driver
iommu/vt-d: Add DEV-MSI support
4. Add platform check for subdevice irq domain
iommu: Add capability IOMMU_CAP_VIOMMU
platform-msi: Add platform check for subdevice irq domain
The device IMS (Interrupt Message Storage) should not be enabled in any
virtualization environments unless there is a HYPERCALL domain which
makes the changes in the message store monitored by the hypervisor.[2]
As the initial step, we allow the IMS to be enabled only if we are
running on the bare metal. It's easy to enable IMS in the virtualization
environments if above preconditions are met in the future.
These patches have been tested with the IDXD driver:
https://github.com/intel/idxd-driver idxd-stage2.5
Most of these patches are originally by Thomas:
https://lore.kernel.org/linux-hyperv/[email protected]/
and are rebased on latest kernel.
This patches series has undergone a lot of changes since first submitted as an RFC
in September 2019. I have divided the changes into 3 stages for better understanding:
Stage 1: Standalone RFC IMS series[3]
-------------------------------------
https://lore.kernel.org/lkml/[email protected]/
V1:
1. Extend existing platform-msi to support IMS
2. platform_msi_domain_alloc_irqs_group is introduced to allocate IMS
interrupts, tagged with a group ID.
3. To free vectors of a particular group, platform_msi_domain_free_irqs_group
API in introduced
Stage 2: dev-msi/IMS merged with Dave Jiang's IDXD series[2]
------------------------------------------------------------
V1: (changes from stage 1):
1. Introduced a new list for platform-msi descriptors
2. Introduced dynamic allocation for IMS interrupts
3. shifted creation of ims domain from arch/x86 to drivers/
4. Removed arch specific callbacks
5. Introduced enum platform_msi_type
6. Added more technical details to the cover letter
7. Merge common code between platform-msi.c and ims-msi.c
8. Introduce new structures platform_msi_ops and platform_msi_funcs
9. Addressed Andriy Shevchenko's comments on RFC V1 patch series
10. Dropped the dynamic group allocation scheme.
11. Use RCU lock instead of mutex lock to protect the device list
V2:
1. IMS made dev-msi
2. With recommendations from Jason/Thomas/Dan on making IMS more generic
3. Pass a non-pci generic device(struct device) for IMS management instead of mdev
4. Remove all references to mdev and symbol_get/put
5. Remove all references to IMS in common code and replace with dev-msi
6. Remove dynamic allocation of platform-msi interrupts: no groups,no
new msi list or list helpers
7. Create a generic dev-msi domain with and without interrupt remapping enabled.
8. Introduce dev_msi_domain_alloc_irqs and dev_msi_domain_free_irqs apis
V3:
1. No need to add support for 2 different dev-msi irq domains, a common
once can be used for both the cases(with IR enabled/disabled)
2. Add arch specific function to specify additions to msi_prepare callback
instead of making the callback a weak function
3. Call platform ops directly instead of a wrapper function
4. Make mask/unmask callbacks as void functions
5. dev->msi_domain should be updated at the device driver level before
calling dev_msi_alloc_irqs()
6. dev_msi_alloc/free_irqs() cannot be used for PCI devices
7. Followed the generic layering scheme: infrastructure bits->arch bits->enabling bits
V4:
1. Make interrupt remapping code more readable
2. Add flush writes to unmask/write and reset ims slots
3. Interrupt Message Storm-> Interrupt Message Store
4. Merge in pasid programming code.
Stage 3: Standalone dev-msi and IMS driver series
-------------------------------------------------
V1:(Changes from Stage 2 V4)[6]
1. Split dev-msi/IMS code from Dave Jiang’s IDXD patch series
2. Set the source-id of all dev-msi interrupt requests to the parent PCI device
3. Separated core irq code from IMS related code
4. Added missing set_desc ops to the IMS msi_domain_ops
5. Added more details in the commit message-test case for auxillary interrupt data
6. Updated the copyright year from 2020 to 2021
7. Updated cover letter
8. Add platform check for subdevice irq domain (Lu Baolu):
V1->V2:
- V1 patches:[4]
- Rename probably_on_bare_metal() with on_bare_metal();
- Some vendors might use the same name for both bare metal and virtual
environment. Before we add vendor specific code to distinguish
between them, let's return false in on_bare_metal(). This won't
introduce any regression. The only impact is that the coming new
platform msi feature won't be supported until the vendor specific code
is provided.
V2->V3:
- V2 patches:[5]
- Add all identified heuristics so far
V1->V2:
1. s/arch_support_pci_device_ims/arch_support_pci_device_msi/g
2. Remove CONFIG_DEVICE_MSI in arch/x86/pci/common.c
3. Added helper functions to get linux IRQ and dev-msi HW IRQ numbers
4. Change the caching mode logic from dynamic to static
Dave Jiang (1):
genirq/msi: Provide helpers to return Linux IRQ/dev_msi hw IRQ number
Lu Baolu (2):
iommu: Add capability IOMMU_CAP_VIOMMU_HINT
platform-msi: Add platform check for subdevice irq domain
Megha Dey (3):
genirq: Set auxiliary data for an interrupt
iommu/vt-d: Add DEV-MSI support
irqchip: Add IMS (Interrupt Message Store) driver
Thomas Gleixner (7):
x86/irq: Add DEV_MSI allocation type
x86/msi: Rename and rework pci_msi_prepare() to cover non-PCI MSI
platform-msi: Provide default irq_chip:: Ack
genirq/proc: Take buslock on affinity write
genirq/msi: Provide and use msi_domain_set_default_info_flags()
platform-msi: Add device MSI infrastructure
irqdomain/msi: Provide msi_alloc/free_store() callbacks
arch/x86/include/asm/hw_irq.h | 1 +
arch/x86/include/asm/msi.h | 4 +-
arch/x86/kernel/apic/msi.c | 27 +++--
arch/x86/pci/common.c | 72 ++++++++++++
drivers/base/platform-msi.c | 141 ++++++++++++++++++++++++
drivers/iommu/amd/iommu.c | 2 +
drivers/iommu/intel/iommu.c | 5 +
drivers/iommu/intel/irq_remapping.c | 6 +-
drivers/iommu/virtio-iommu.c | 9 ++
drivers/irqchip/Kconfig | 14 +++
drivers/irqchip/Makefile | 1 +
drivers/irqchip/irq-ims-msi.c | 211 ++++++++++++++++++++++++++++++++++++
drivers/pci/controller/pci-hyperv.c | 2 +-
drivers/pci/msi.c | 7 +-
include/linux/interrupt.h | 2 +
include/linux/iommu.h | 2 +
include/linux/irq.h | 4 +
include/linux/irqchip/irq-ims-msi.h | 68 ++++++++++++
include/linux/irqdomain.h | 1 +
include/linux/msi.h | 41 +++++++
kernel/irq/Kconfig | 4 +
kernel/irq/manage.c | 38 ++++++-
kernel/irq/msi.c | 79 ++++++++++++++
23 files changed, 722 insertions(+), 19 deletions(-)
create mode 100644 drivers/irqchip/irq-ims-msi.c
create mode 100644 include/linux/irqchip/irq-ims-msi.h
--
2.7.4
From: Thomas Gleixner <[email protected]>
Until now interrupt chips which support setting affinity are not locking
the associated bus lock for two reasons:
- All chips which support affinity setting do not use buslock because they
just can operated directly on the hardware.
- All chips which use buslock do not support affinity setting because
their interrupt chips are not capable. These chips are usually connected
over a bus like I2C, SPI etc. and have an interrupt output which is
conneted to CPU interrupt of some sort. So there is no way to set the
affinity on the chip itself.
Upcoming hardware which is PCIE based sports a non standard MSI(X) variant
which stores the MSI message in RAM which is associated to e.g. a device
queue. The device manages this RAM and writes have to be issued via command
queues or similar mechanisms which is obviously not possible from interrupt
disabled, raw spinlock held context.
The buslock mechanism of irq chips can be utilized to support that. The
affinity write to the chip writes to shadow state, marks it pending and the
irq chip's irq_bus_sync_unlock() callback handles the command queue and
wait for completion similar to the other chip operations on I2C or SPI
buses.
Change the locking in irq_set_affinity() to bus_lock/unlock to help with
that. There are a few other callers than the proc interface, but none of
them is affected by this change as none of them affects an irq chip with
bus lock support.
Reviewed-by: Tony Luck <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Signed-off-by: Megha Dey <[email protected]>
---
kernel/irq/manage.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/kernel/irq/manage.c b/kernel/irq/manage.c
index dec3f73..85ede4e 100644
--- a/kernel/irq/manage.c
+++ b/kernel/irq/manage.c
@@ -443,16 +443,16 @@ int irq_update_affinity_desc(unsigned int irq,
int __irq_set_affinity(unsigned int irq, const struct cpumask *mask, bool force)
{
- struct irq_desc *desc = irq_to_desc(irq);
+ struct irq_desc *desc;
unsigned long flags;
int ret;
+ desc = irq_get_desc_buslock(irq, &flags, IRQ_GET_DESC_CHECK_GLOBAL);
if (!desc)
return -EINVAL;
- raw_spin_lock_irqsave(&desc->lock, flags);
ret = irq_set_affinity_locked(irq_desc_get_irq_data(desc), mask, force);
- raw_spin_unlock_irqrestore(&desc->lock, flags);
+ irq_put_desc_busunlock(desc, flags);
return ret;
}
--
2.7.4
Generic IMS(Interrupt Message Store) irq chips and irq domain
implementations for IMS based devices which store the interrupt messages
in an array in device memory.
Allocation and freeing of interrupts happens via the generic
msi_domain_alloc/free_irqs() interface. No special purpose IMS magic
required as long as the interrupt domain is stored in the underlying
device struct. The irq_set_auxdata() is used to program the pasid into
the IMS entry.
[Megha: Fixed compile time errors
Added necessary dependencies to IMS_MSI_ARRAY config
Fixed polarity of IMS_VECTOR_CTRL
Added reads after writes to flush writes to device
Added set_desc ops to IMS msi domain ops
Tested the IMS infrastructure with the IDXD driver]
Reviewed-by: Tony Luck <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Signed-off-by: Megha Dey <[email protected]>
---
drivers/irqchip/Kconfig | 14 +++
drivers/irqchip/Makefile | 1 +
drivers/irqchip/irq-ims-msi.c | 211 ++++++++++++++++++++++++++++++++++++
include/linux/irqchip/irq-ims-msi.h | 68 ++++++++++++
4 files changed, 294 insertions(+)
create mode 100644 drivers/irqchip/irq-ims-msi.c
create mode 100644 include/linux/irqchip/irq-ims-msi.h
diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
index e74fa20..2fb0c24 100644
--- a/drivers/irqchip/Kconfig
+++ b/drivers/irqchip/Kconfig
@@ -586,4 +586,18 @@ config MST_IRQ
help
Support MStar Interrupt Controller.
+config IMS_MSI
+ depends on PCI
+ select DEVICE_MSI
+ bool
+
+config IMS_MSI_ARRAY
+ bool "IMS Interrupt Message Store MSI controller for device memory storage arrays"
+ depends on PCI
+ select IMS_MSI
+ select GENERIC_MSI_IRQ_DOMAIN
+ help
+ Support for IMS Interrupt Message Store MSI controller
+ with IMS slot storage in a slot array in device memory
+
endmenu
diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
index c59b95a..e903201 100644
--- a/drivers/irqchip/Makefile
+++ b/drivers/irqchip/Makefile
@@ -113,3 +113,4 @@ obj-$(CONFIG_LOONGSON_PCH_MSI) += irq-loongson-pch-msi.o
obj-$(CONFIG_MST_IRQ) += irq-mst-intc.o
obj-$(CONFIG_SL28CPLD_INTC) += irq-sl28cpld.o
obj-$(CONFIG_MACH_REALTEK_RTL) += irq-realtek-rtl.o
+obj-$(CONFIG_IMS_MSI) += irq-ims-msi.o
diff --git a/drivers/irqchip/irq-ims-msi.c b/drivers/irqchip/irq-ims-msi.c
new file mode 100644
index 0000000..fa23207
--- /dev/null
+++ b/drivers/irqchip/irq-ims-msi.c
@@ -0,0 +1,211 @@
+// SPDX-License-Identifier: GPL-2.0
+// (C) Copyright 2021 Thomas Gleixner <[email protected]>
+/*
+ * Shared interrupt chips and irq domains for IMS devices
+ */
+#include <linux/device.h>
+#include <linux/slab.h>
+#include <linux/msi.h>
+#include <linux/irq.h>
+#include <linux/irqdomain.h>
+
+#include <linux/irqchip/irq-ims-msi.h>
+
+#ifdef CONFIG_IMS_MSI_ARRAY
+
+struct ims_array_data {
+ struct ims_array_info info;
+ unsigned long map[0];
+};
+
+static inline void iowrite32_and_flush(u32 value, void __iomem *addr)
+{
+ iowrite32(value, addr);
+ ioread32(addr);
+}
+
+static void ims_array_mask_irq(struct irq_data *data)
+{
+ struct msi_desc *desc = irq_data_get_msi_desc(data);
+ struct ims_slot __iomem *slot = desc->device_msi.priv_iomem;
+ u32 __iomem *ctrl = &slot->ctrl;
+
+ iowrite32_and_flush(ioread32(ctrl) | IMS_CTRL_VECTOR_MASKBIT, ctrl);
+}
+
+static void ims_array_unmask_irq(struct irq_data *data)
+{
+ struct msi_desc *desc = irq_data_get_msi_desc(data);
+ struct ims_slot __iomem *slot = desc->device_msi.priv_iomem;
+ u32 __iomem *ctrl = &slot->ctrl;
+
+ iowrite32_and_flush(ioread32(ctrl) & ~IMS_CTRL_VECTOR_MASKBIT, ctrl);
+}
+
+static void ims_array_write_msi_msg(struct irq_data *data, struct msi_msg *msg)
+{
+ struct msi_desc *desc = irq_data_get_msi_desc(data);
+ struct ims_slot __iomem *slot = desc->device_msi.priv_iomem;
+
+ iowrite32(msg->address_lo, &slot->address_lo);
+ iowrite32(msg->address_hi, &slot->address_hi);
+ iowrite32_and_flush(msg->data, &slot->data);
+}
+
+static int ims_array_set_auxdata(struct irq_data *data, unsigned int which,
+ u64 auxval)
+{
+ struct msi_desc *desc = irq_data_get_msi_desc(data);
+ struct ims_slot __iomem *slot = desc->device_msi.priv_iomem;
+ u32 val, __iomem *ctrl = &slot->ctrl;
+
+ if (which != IMS_AUXDATA_CONTROL_WORD)
+ return -EINVAL;
+ if (auxval & ~(u64)IMS_CONTROL_WORD_AUXMASK)
+ return -EINVAL;
+
+ val = ioread32(ctrl) & IMS_CONTROL_WORD_IRQMASK;
+ iowrite32_and_flush(val | (u32)auxval, ctrl);
+ return 0;
+}
+
+static const struct irq_chip ims_array_msi_controller = {
+ .name = "IMS",
+ .irq_mask = ims_array_mask_irq,
+ .irq_unmask = ims_array_unmask_irq,
+ .irq_write_msi_msg = ims_array_write_msi_msg,
+ .irq_set_auxdata = ims_array_set_auxdata,
+ .irq_retrigger = irq_chip_retrigger_hierarchy,
+ .flags = IRQCHIP_SKIP_SET_WAKE,
+};
+
+static void ims_array_reset_slot(struct ims_slot __iomem *slot)
+{
+ iowrite32(0, &slot->address_lo);
+ iowrite32(0, &slot->address_hi);
+ iowrite32(0, &slot->data);
+ iowrite32_and_flush(IMS_CTRL_VECTOR_MASKBIT, &slot->ctrl);
+}
+
+static void ims_array_free_msi_store(struct irq_domain *domain,
+ struct device *dev)
+{
+ struct msi_domain_info *info = domain->host_data;
+ struct ims_array_data *ims = info->data;
+ struct msi_desc *entry;
+
+ for_each_msi_entry(entry, dev) {
+ if (entry->device_msi.priv_iomem) {
+ clear_bit(entry->device_msi.hwirq, ims->map);
+ ims_array_reset_slot(entry->device_msi.priv_iomem);
+ entry->device_msi.priv_iomem = NULL;
+ entry->device_msi.hwirq = 0;
+ }
+ }
+}
+
+static int ims_array_alloc_msi_store(struct irq_domain *domain,
+ struct device *dev, int nvec)
+{
+ struct msi_domain_info *info = domain->host_data;
+ struct ims_array_data *ims = info->data;
+ struct msi_desc *entry;
+
+ for_each_msi_entry(entry, dev) {
+ unsigned int idx;
+
+ idx = find_first_zero_bit(ims->map, ims->info.max_slots);
+ if (idx >= ims->info.max_slots)
+ goto fail;
+ set_bit(idx, ims->map);
+ entry->device_msi.priv_iomem = &ims->info.slots[idx];
+ ims_array_reset_slot(entry->device_msi.priv_iomem);
+ entry->device_msi.hwirq = idx;
+ }
+ return 0;
+
+fail:
+ ims_array_free_msi_store(domain, dev);
+ return -ENOSPC;
+}
+
+struct ims_array_domain_template {
+ struct msi_domain_ops ops;
+ struct msi_domain_info info;
+};
+
+static void ims_set_desc(msi_alloc_info_t *arg, struct msi_desc *desc)
+{
+ arg->desc = desc;
+ arg->hwirq = desc->device_msi.hwirq;
+}
+
+static const struct ims_array_domain_template ims_array_domain_template = {
+ .ops = {
+ .msi_alloc_store = ims_array_alloc_msi_store,
+ .msi_free_store = ims_array_free_msi_store,
+ .set_desc = ims_set_desc,
+ },
+ .info = {
+ .flags = MSI_FLAG_USE_DEF_DOM_OPS |
+ MSI_FLAG_USE_DEF_CHIP_OPS,
+ .handler = handle_edge_irq,
+ .handler_name = "edge",
+ },
+};
+
+struct irq_domain *
+pci_ims_array_create_msi_irq_domain(struct pci_dev *pdev,
+ struct ims_array_info *ims_info)
+{
+ struct ims_array_domain_template *info;
+ struct ims_array_data *data;
+ struct irq_domain *domain;
+ struct irq_chip *chip;
+ unsigned int size;
+
+ /* Allocate new domain storage */
+ info = kmemdup(&ims_array_domain_template,
+ sizeof(ims_array_domain_template), GFP_KERNEL);
+ if (!info)
+ return NULL;
+ /* Link the ops */
+ info->info.ops = &info->ops;
+
+ /* Allocate ims_info along with the bitmap */
+ size = sizeof(*data);
+ size += BITS_TO_LONGS(ims_info->max_slots) * sizeof(unsigned long);
+ data = kzalloc(size, GFP_KERNEL);
+ if (!data)
+ goto err_info;
+
+ data->info = *ims_info;
+ info->info.data = data;
+
+ /*
+ * Allocate an interrupt chip because the core needs to be able to
+ * update it with default callbacks.
+ */
+ chip = kmemdup(&ims_array_msi_controller,
+ sizeof(ims_array_msi_controller), GFP_KERNEL);
+ if (!chip)
+ goto err_data;
+ info->info.chip = chip;
+
+ domain = pci_subdevice_msi_create_irq_domain(pdev, &info->info);
+ if (!domain)
+ goto err_chip;
+
+ return domain;
+
+err_chip:
+ kfree(chip);
+err_data:
+ kfree(data);
+err_info:
+ kfree(info);
+ return NULL;
+}
+EXPORT_SYMBOL_GPL(pci_ims_array_create_msi_irq_domain);
+
+#endif /* CONFIG_IMS_MSI_ARRAY */
diff --git a/include/linux/irqchip/irq-ims-msi.h b/include/linux/irqchip/irq-ims-msi.h
new file mode 100644
index 0000000..9ba767f
--- /dev/null
+++ b/include/linux/irqchip/irq-ims-msi.h
@@ -0,0 +1,68 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* (C) Copyright 2021 Thomas Gleixner <[email protected]> */
+
+#ifndef _LINUX_IRQCHIP_IRQ_IMS_MSI_H
+#define _LINUX_IRQCHIP_IRQ_IMS_MSI_H
+
+#include <linux/types.h>
+#include <linux/bits.h>
+
+/**
+ * ims_hw_slot - The hardware layout of an IMS based MSI message
+ * @address_lo: Lower 32bit address
+ * @address_hi: Upper 32bit address
+ * @data: Message data
+ * @ctrl: Control word
+ *
+ * This structure is used by both the device memory array and the queue
+ * memory variants of IMS.
+ */
+struct ims_slot {
+ u32 address_lo;
+ u32 address_hi;
+ u32 data;
+ u32 ctrl;
+} __packed;
+
+/*
+ * The IMS control word utilizes bit 0-2 for interrupt control. The remaining
+ * bits can contain auxiliary data.
+ */
+#define IMS_CONTROL_WORD_IRQMASK GENMASK(2, 0)
+#define IMS_CONTROL_WORD_AUXMASK GENMASK(31, 3)
+
+/* Auxiliary control word data related defines */
+enum {
+ IMS_AUXDATA_CONTROL_WORD,
+};
+
+/* Bit to mask the interrupt in ims_hw_slot::ctrl */
+#define IMS_CTRL_VECTOR_MASKBIT BIT(0)
+#define IMS_CTRL_PASID_ENABLE BIT(3)
+#define IMS_CTRL_PASID_SHIFT 12
+
+/* Set pasid and enable bit for the IMS entry */
+static inline u32 ims_ctrl_pasid_aux(unsigned int pasid, bool enable)
+{
+ u32 auxval = pasid << IMS_CTRL_PASID_SHIFT;
+
+ return enable ? auxval | IMS_CTRL_PASID_ENABLE : auxval;
+}
+
+/**
+ * struct ims_array_info - Information to create an IMS array domain
+ * @slots: Pointer to the start of the array
+ * @max_slots: Maximum number of slots in the array
+ */
+struct ims_array_info {
+ struct ims_slot __iomem *slots;
+ unsigned int max_slots;
+};
+
+struct pci_dev;
+struct irq_domain;
+
+struct irq_domain *pci_ims_array_create_msi_irq_domain(struct pci_dev *pdev,
+ struct ims_array_info *ims_info);
+
+#endif
--
2.7.4
On Fri, 26 Feb 2021 20:11:16 +0000,
Megha Dey <[email protected]> wrote:
>
> Generic IMS(Interrupt Message Store) irq chips and irq domain
> implementations for IMS based devices which store the interrupt messages
> in an array in device memory.
>
> Allocation and freeing of interrupts happens via the generic
> msi_domain_alloc/free_irqs() interface. No special purpose IMS magic
> required as long as the interrupt domain is stored in the underlying
> device struct. The irq_set_auxdata() is used to program the pasid into
> the IMS entry.
>
> [Megha: Fixed compile time errors
> Added necessary dependencies to IMS_MSI_ARRAY config
> Fixed polarity of IMS_VECTOR_CTRL
> Added reads after writes to flush writes to device
> Added set_desc ops to IMS msi domain ops
> Tested the IMS infrastructure with the IDXD driver]
>
> Reviewed-by: Tony Luck <[email protected]>
> Signed-off-by: Thomas Gleixner <[email protected]>
> Signed-off-by: Megha Dey <[email protected]>
> ---
> drivers/irqchip/Kconfig | 14 +++
> drivers/irqchip/Makefile | 1 +
> drivers/irqchip/irq-ims-msi.c | 211 ++++++++++++++++++++++++++++++++++++
> include/linux/irqchip/irq-ims-msi.h | 68 ++++++++++++
> 4 files changed, 294 insertions(+)
> create mode 100644 drivers/irqchip/irq-ims-msi.c
> create mode 100644 include/linux/irqchip/irq-ims-msi.h
>
> diff --git a/drivers/irqchip/Kconfig b/drivers/irqchip/Kconfig
> index e74fa20..2fb0c24 100644
> --- a/drivers/irqchip/Kconfig
> +++ b/drivers/irqchip/Kconfig
> @@ -586,4 +586,18 @@ config MST_IRQ
> help
> Support MStar Interrupt Controller.
>
> +config IMS_MSI
> + depends on PCI
> + select DEVICE_MSI
> + bool
> +
> +config IMS_MSI_ARRAY
> + bool "IMS Interrupt Message Store MSI controller for device memory storage arrays"
> + depends on PCI
> + select IMS_MSI
> + select GENERIC_MSI_IRQ_DOMAIN
> + help
> + Support for IMS Interrupt Message Store MSI controller
> + with IMS slot storage in a slot array in device memory
> +
> endmenu
> diff --git a/drivers/irqchip/Makefile b/drivers/irqchip/Makefile
> index c59b95a..e903201 100644
> --- a/drivers/irqchip/Makefile
> +++ b/drivers/irqchip/Makefile
> @@ -113,3 +113,4 @@ obj-$(CONFIG_LOONGSON_PCH_MSI) += irq-loongson-pch-msi.o
> obj-$(CONFIG_MST_IRQ) += irq-mst-intc.o
> obj-$(CONFIG_SL28CPLD_INTC) += irq-sl28cpld.o
> obj-$(CONFIG_MACH_REALTEK_RTL) += irq-realtek-rtl.o
> +obj-$(CONFIG_IMS_MSI) += irq-ims-msi.o
> diff --git a/drivers/irqchip/irq-ims-msi.c b/drivers/irqchip/irq-ims-msi.c
> new file mode 100644
> index 0000000..fa23207
> --- /dev/null
> +++ b/drivers/irqchip/irq-ims-msi.c
> @@ -0,0 +1,211 @@
> +// SPDX-License-Identifier: GPL-2.0
> +// (C) Copyright 2021 Thomas Gleixner <[email protected]>
> +/*
> + * Shared interrupt chips and irq domains for IMS devices
> + */
> +#include <linux/device.h>
> +#include <linux/slab.h>
> +#include <linux/msi.h>
> +#include <linux/irq.h>
> +#include <linux/irqdomain.h>
> +
> +#include <linux/irqchip/irq-ims-msi.h>
> +
> +#ifdef CONFIG_IMS_MSI_ARRAY
Given that this covers the whole driver, what is this #defined used
for? You might as well make the driver depend on this config option.
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
On Thu, Mar 25 2021 at 17:43, Marc Zyngier wrote:
> On Fri, 26 Feb 2021 20:11:16 +0000,
> Megha Dey <[email protected]> wrote:
>> +
>> +#include <linux/irqchip/irq-ims-msi.h>
>> +
>> +#ifdef CONFIG_IMS_MSI_ARRAY
>
> Given that this covers the whole driver, what is this #defined used
> for? You might as well make the driver depend on this config option.
That's a leftover from the initial version I wrote which had also
support for IMS_MSI_QUEUE to store the message in queue memory, but we
have no use case yet for it.
But yes, as things stand now it does not make any sense and IIRC at the
end they do not share anything in the C file except for some includes at
the very end.
Thanks,
tglx
Hi Thomas/Marc,
On 3/25/2021 12:07 PM, Thomas Gleixner wrote:
> On Thu, Mar 25 2021 at 17:43, Marc Zyngier wrote:
>> On Fri, 26 Feb 2021 20:11:16 +0000,
>> Megha Dey <[email protected]> wrote:
>>> +
>>> +#include <linux/irqchip/irq-ims-msi.h>
>>> +
>>> +#ifdef CONFIG_IMS_MSI_ARRAY
>> Given that this covers the whole driver, what is this #defined used
>> for? You might as well make the driver depend on this config option.
> That's a leftover from the initial version I wrote which had also
> support for IMS_MSI_QUEUE to store the message in queue memory, but we
> have no use case yet for it.
>
> But yes, as things stand now it does not make any sense and IIRC at the
> end they do not share anything in the C file except for some includes at
> the very end.
Sure, I will make this change.
>
> Thanks,
>
> tglx
>
>