2005-12-22 20:17:08

by Mark Maule

[permalink] [raw]
Subject: [PATCH 0/3] msi abstractions and support for altix

Resend #2: including linuxppc64-dev and linux-pci as well as PCI maintainer

Patch set to abstract portions of the MSI core so that it can be used on
architectures which don't use standard interrupt controllers.

Changes from last version based on review comments:

+ Change uintXX_t to uXX
+ Change _callouts to _ops
+ Renamed the _generic routines to _apic and moved them to a new file
msi-apic.c
+ Have each msi_arch_init() routine call msi_register() with the desired
msi ops for that platform.
+ Moved msi_address, msi_data, and related defs out of msi.h and into
msi-apic.c, replaced by shifts/masks.
+ Rolled msi-arch-init.patch and msi-callouts.patch into a single msi-ops.patch

Note: I don't have ia32 or non-altix ia64 gear with MSI capable cards.
Could some give this a sanity check to make sure I didn't break the
redefinition of the former msi_address/msi_data bits?

Mark

1/3 msi-ops.patch
Add an msi_arch_init() hook which can be used to perform platform
specific setup prior to msi use.

Define a set of msi ops to implement the platform-specific tasks:

setup - set up plumbing to get a vector directed at a default
cpu, and return the corresponding MSI bus address and data.
teardown - inverse of msi_setup
target - retarget a vector to a given cpu

Define the routine msi_register() called from msi_arch_init()
to set the desired ops.

Move a bunch of apic-specific code out of the msi core .h/.c and
into a new msi-apic.c file.

2/3 ia64-per-platform-device-vector.patch
For the ia64 arch, allow per-platform definitions of
IA64_FIRST_DEVICE_VECTOR and IA64_LAST_DEVICE_VECTOR.

3/3 msi-altix.patch
Altix specific callouts to implement MSI.


2005-12-22 20:17:15

by Mark Maule

[permalink] [raw]
Subject: [PATCH 2/3] per-platform IA64_{FIRST,LAST}_DEVICE_VECTOR definitions

Abstract IA64_FIRST_DEVICE_VECTOR/IA64_LAST_DEVICE_VECTOR since SN platforms
use a subset of the IA64 range. Implement this by making the above macros
global variables which the platform can override in it setup code.

Also add a reserve_irq_vector() routine used by SN to mark a vector's as
in-use when that weren't allocated through assign_irq_vector().

Signed-off-by: Mark Maule <[email protected]>

Index: msi/arch/ia64/kernel/irq_ia64.c
===================================================================
--- msi.orig/arch/ia64/kernel/irq_ia64.c 2005-12-21 22:59:09.199823700 -0600
+++ msi/arch/ia64/kernel/irq_ia64.c 2005-12-22 14:10:01.012860466 -0600
@@ -46,6 +46,10 @@

#define IRQ_DEBUG 0

+/* These can be overridden in platform_irq_init */
+int ia64_first_device_vector = IA64_DEF_FIRST_DEVICE_VECTOR;
+int ia64_last_device_vector = IA64_DEF_LAST_DEVICE_VECTOR;
+
/* default base addr of IPI table */
void __iomem *ipi_base_addr = ((void __iomem *)
(__IA64_UNCACHED_OFFSET | IA64_IPI_DEFAULT_BASE_ADDR));
@@ -60,7 +64,7 @@
};
EXPORT_SYMBOL(isa_irq_to_vector_map);

-static unsigned long ia64_vector_mask[BITS_TO_LONGS(IA64_NUM_DEVICE_VECTORS)];
+static unsigned long ia64_vector_mask[BITS_TO_LONGS(IA64_MAX_DEVICE_VECTORS)];

int
assign_irq_vector (int irq)
@@ -89,6 +93,17 @@
printk(KERN_WARNING "%s: double free!\n", __FUNCTION__);
}

+int
+reserve_irq_vector (int vector)
+{
+ if (vector < IA64_FIRST_DEVICE_VECTOR ||
+ vector > IA64_LAST_DEVICE_VECTOR)
+ return -EINVAL;
+
+ return test_and_set_bit(IA64_FIRST_DEVICE_VECTOR + vector,
+ ia64_vector_mask);
+}
+
#ifdef CONFIG_SMP
# define IS_RESCHEDULE(vec) (vec == IA64_IPI_RESCHEDULE)
#else
Index: msi/arch/ia64/sn/kernel/irq.c
===================================================================
--- msi.orig/arch/ia64/sn/kernel/irq.c 2005-12-21 22:59:09.199823700 -0600
+++ msi/arch/ia64/sn/kernel/irq.c 2005-12-22 14:10:01.024578027 -0600
@@ -203,6 +203,9 @@
int i;
irq_desc_t *base_desc = irq_desc;

+ ia64_first_device_vector = IA64_SN2_FIRST_DEVICE_VECTOR;
+ ia64_last_device_vector = IA64_SN2_LAST_DEVICE_VECTOR;
+
for (i = 0; i < NR_IRQS; i++) {
if (base_desc[i].handler == &no_irq_type) {
base_desc[i].handler = &irq_type_sn;
@@ -287,6 +290,7 @@
/* link it into the sn_irq[irq] list */
spin_lock(&sn_irq_info_lock);
list_add_rcu(&sn_irq_info->list, sn_irq_lh[sn_irq_info->irq_irq]);
+ reserve_irq_vector(sn_irq_info->irq_irq);
spin_unlock(&sn_irq_info_lock);

(void)register_intr_pda(sn_irq_info);
@@ -310,8 +314,11 @@
spin_lock(&sn_irq_info_lock);
list_del_rcu(&sn_irq_info->list);
spin_unlock(&sn_irq_info_lock);
+ if (list_empty(sn_irq_lh[sn_irq_info->irq_irq]))
+ free_irq_vector(sn_irq_info->irq_irq);
call_rcu(&sn_irq_info->rcu, sn_irq_info_free);
pci_dev_put(pci_dev);
+
}

static inline void
Index: msi/include/asm-ia64/hw_irq.h
===================================================================
--- msi.orig/include/asm-ia64/hw_irq.h 2005-12-21 22:59:09.200800164 -0600
+++ msi/include/asm-ia64/hw_irq.h 2005-12-22 14:10:01.046060224 -0600
@@ -47,9 +47,19 @@
#define IA64_CMC_VECTOR 0x1f /* corrected machine-check interrupt vector */
/*
* Vectors 0x20-0x2f are reserved for legacy ISA IRQs.
+ * Use vectors 0x30-0xe7 as the default device vector range for ia64.
+ * Platforms may choose to reduce this range in platform_irq_setup, but the
+ * platform range must fall within
+ * [IA64_DEF_FIRST_DEVICE_VECTOR..IA64_DEF_LAST_DEVICE_VECTOR]
*/
-#define IA64_FIRST_DEVICE_VECTOR 0x30
-#define IA64_LAST_DEVICE_VECTOR 0xe7
+extern int ia64_first_device_vector;
+extern int ia64_last_device_vector;
+
+#define IA64_DEF_FIRST_DEVICE_VECTOR 0x30
+#define IA64_DEF_LAST_DEVICE_VECTOR 0xe7
+#define IA64_FIRST_DEVICE_VECTOR ia64_first_device_vector
+#define IA64_LAST_DEVICE_VECTOR ia64_last_device_vector
+#define IA64_MAX_DEVICE_VECTORS (IA64_DEF_LAST_DEVICE_VECTOR - IA64_DEF_FIRST_DEVICE_VECTOR + 1)
#define IA64_NUM_DEVICE_VECTORS (IA64_LAST_DEVICE_VECTOR - IA64_FIRST_DEVICE_VECTOR + 1)

#define IA64_MCA_RENDEZ_VECTOR 0xe8 /* MCA rendez interrupt */
@@ -83,6 +93,7 @@

extern int assign_irq_vector (int irq); /* allocate a free vector */
extern void free_irq_vector (int vector);
+extern int reserve_irq_vector (int vector);
extern void ia64_send_ipi (int cpu, int vector, int delivery_mode, int redirect);
extern void register_percpu_irq (ia64_vector vec, struct irqaction *action);

Index: msi/drivers/pci/msi.c
===================================================================
--- msi.orig/drivers/pci/msi.c 2005-12-21 22:59:09.200800164 -0600
+++ msi/drivers/pci/msi.c 2005-12-22 14:10:19.301044796 -0600
@@ -35,7 +35,7 @@

#ifndef CONFIG_X86_IO_APIC
int vector_irq[NR_VECTORS] = { [0 ... NR_VECTORS - 1] = -1};
-u8 irq_vector[NR_IRQ_VECTORS] = { FIRST_DEVICE_VECTOR , 0 };
+u8 irq_vector[NR_IRQ_VECTORS];
#endif

static struct msi_ops *msi_ops;
@@ -377,6 +377,11 @@
printk(KERN_WARNING "PCI: MSI cache init failed\n");
return status;
}
+
+#ifndef CONFIG_X86_IO_APIC
+ irq_vector[0] = FIRST_DEVICE_VECTOR;
+#endif
+
last_alloc_vector = assign_irq_vector(AUTO_ASSIGN);
if (last_alloc_vector < 0) {
pci_msi_enable = 0;

2005-12-22 20:17:33

by Mark Maule

[permalink] [raw]
Subject: [PATCH 1/3] msi vector targeting abstractions

Abstract portions of the MSI core for platforms that do not use standard
APIC interrupt controllers. This is implemented through a new arch-specific
msi setup routine, and a set of msi ops which can be set on a per platform
basis.

Signed-off-by: Mark Maule <[email protected]>

Index: msi/drivers/pci/msi.c
===================================================================
--- msi.orig/drivers/pci/msi.c 2005-12-13 12:22:42.784269607 -0600
+++ msi/drivers/pci/msi.c 2005-12-21 22:59:09.200800164 -0600
@@ -23,8 +23,6 @@
#include "pci.h"
#include "msi.h"

-#define MSI_TARGET_CPU first_cpu(cpu_online_map)
-
static DEFINE_SPINLOCK(msi_lock);
static struct msi_desc* msi_desc[NR_IRQS] = { [0 ... NR_IRQS-1] = NULL };
static kmem_cache_t* msi_cachep;
@@ -40,6 +38,15 @@
u8 irq_vector[NR_IRQ_VECTORS] = { FIRST_DEVICE_VECTOR , 0 };
#endif

+static struct msi_ops *msi_ops;
+
+int
+msi_register(struct msi_ops *ops)
+{
+ msi_ops = ops;
+ return 0;
+}
+
static void msi_cache_ctor(void *p, kmem_cache_t *cache, unsigned long flags)
{
memset(p, 0, NR_IRQS * sizeof(struct msi_desc));
@@ -92,7 +99,7 @@
static void set_msi_affinity(unsigned int vector, cpumask_t cpu_mask)
{
struct msi_desc *entry;
- struct msg_address address;
+ u32 address_hi, address_lo;
unsigned int irq = vector;
unsigned int dest_cpu = first_cpu(cpu_mask);

@@ -108,28 +115,36 @@
if (!(pos = pci_find_capability(entry->dev, PCI_CAP_ID_MSI)))
return;

+ pci_read_config_dword(entry->dev, msi_upper_address_reg(pos),
+ &address_hi);
pci_read_config_dword(entry->dev, msi_lower_address_reg(pos),
- &address.lo_address.value);
- address.lo_address.value &= MSI_ADDRESS_DEST_ID_MASK;
- address.lo_address.value |= (cpu_physical_id(dest_cpu) <<
- MSI_TARGET_CPU_SHIFT);
- entry->msi_attrib.current_cpu = cpu_physical_id(dest_cpu);
+ &address_lo);
+
+ msi_ops->target(vector, dest_cpu, &address_hi, &address_lo);
+
+ pci_write_config_dword(entry->dev, msi_upper_address_reg(pos),
+ address_hi);
pci_write_config_dword(entry->dev, msi_lower_address_reg(pos),
- address.lo_address.value);
+ address_lo);
set_native_irq_info(irq, cpu_mask);
break;
}
case PCI_CAP_ID_MSIX:
{
- int offset = entry->msi_attrib.entry_nr * PCI_MSIX_ENTRY_SIZE +
- PCI_MSIX_ENTRY_LOWER_ADDR_OFFSET;
+ int offset_hi =
+ entry->msi_attrib.entry_nr * PCI_MSIX_ENTRY_SIZE +
+ PCI_MSIX_ENTRY_UPPER_ADDR_OFFSET;
+ int offset_lo =
+ entry->msi_attrib.entry_nr * PCI_MSIX_ENTRY_SIZE +
+ PCI_MSIX_ENTRY_LOWER_ADDR_OFFSET;
+
+ address_hi = readl(entry->mask_base + offset_hi);
+ address_lo = readl(entry->mask_base + offset_lo);

- address.lo_address.value = readl(entry->mask_base + offset);
- address.lo_address.value &= MSI_ADDRESS_DEST_ID_MASK;
- address.lo_address.value |= (cpu_physical_id(dest_cpu) <<
- MSI_TARGET_CPU_SHIFT);
- entry->msi_attrib.current_cpu = cpu_physical_id(dest_cpu);
- writel(address.lo_address.value, entry->mask_base + offset);
+ msi_ops->target(vector, dest_cpu, &address_hi, &address_lo);
+
+ writel(address_hi, entry->mask_base + offset_hi);
+ writel(address_lo, entry->mask_base + offset_lo);
set_native_irq_info(irq, cpu_mask);
break;
}
@@ -249,30 +264,6 @@
.set_affinity = set_msi_irq_affinity
};

-static void msi_data_init(struct msg_data *msi_data,
- unsigned int vector)
-{
- memset(msi_data, 0, sizeof(struct msg_data));
- msi_data->vector = (u8)vector;
- msi_data->delivery_mode = MSI_DELIVERY_MODE;
- msi_data->level = MSI_LEVEL_MODE;
- msi_data->trigger = MSI_TRIGGER_MODE;
-}
-
-static void msi_address_init(struct msg_address *msi_address)
-{
- unsigned int dest_id;
- unsigned long dest_phys_id = cpu_physical_id(MSI_TARGET_CPU);
-
- memset(msi_address, 0, sizeof(struct msg_address));
- msi_address->hi_address = (u32)0;
- dest_id = (MSI_ADDRESS_HEADER << MSI_ADDRESS_HEADER_SHIFT);
- msi_address->lo_address.u.dest_mode = MSI_PHYSICAL_MODE;
- msi_address->lo_address.u.redirection_hint = MSI_REDIRECTION_HINT_MODE;
- msi_address->lo_address.u.dest_id = dest_id;
- msi_address->lo_address.value |= (dest_phys_id << MSI_TARGET_CPU_SHIFT);
-}
-
static int msi_free_vector(struct pci_dev* dev, int vector, int reassign);
static int assign_msi_vector(void)
{
@@ -367,6 +358,20 @@
return status;
}

+ if ((status = msi_arch_init()) < 0) {
+ pci_msi_enable = 0;
+ printk(KERN_WARNING
+ "PCI: MSI arch init failed. MSI disabled.\n");
+ return status;
+ }
+
+ if (! msi_ops) {
+ printk(KERN_WARNING
+ "PCI: MSI ops not registered. MSI disabled.\n");
+ status = -EINVAL;
+ return status;
+ }
+
if ((status = msi_cache_init()) < 0) {
pci_msi_enable = 0;
printk(KERN_WARNING "PCI: MSI cache init failed\n");
@@ -510,9 +515,11 @@
**/
static int msi_capability_init(struct pci_dev *dev)
{
+ int status;
struct msi_desc *entry;
- struct msg_address address;
- struct msg_data data;
+ u32 address_lo;
+ u32 address_hi;
+ u32 data;
int pos, vector;
u16 control;

@@ -539,23 +546,26 @@
entry->mask_base = (void __iomem *)(long)msi_mask_bits_reg(pos,
is_64bit_address(control));
}
+ /* Configure MSI capability structure */
+ status = msi_ops->setup(dev, vector,
+ &address_hi,
+ &address_lo,
+ &data);
+ if (status < 0) {
+ kmem_cache_free(msi_cachep, entry);
+ return status;
+ }
/* Replace with MSI handler */
irq_handler_init(PCI_CAP_ID_MSI, vector, entry->msi_attrib.maskbit);
- /* Configure MSI capability structure */
- msi_address_init(&address);
- msi_data_init(&data, vector);
- entry->msi_attrib.current_cpu = ((address.lo_address.u.dest_id >>
- MSI_TARGET_CPU_SHIFT) & MSI_TARGET_CPU_MASK);
- pci_write_config_dword(dev, msi_lower_address_reg(pos),
- address.lo_address.value);
+
+ pci_write_config_dword(dev, msi_lower_address_reg(pos), address_lo);
if (is_64bit_address(control)) {
pci_write_config_dword(dev,
- msi_upper_address_reg(pos), address.hi_address);
- pci_write_config_word(dev,
- msi_data_reg(pos, 1), *((u32*)&data));
+ msi_upper_address_reg(pos), address_hi);
+ pci_write_config_word(dev, msi_data_reg(pos, 1), data);
} else
- pci_write_config_word(dev,
- msi_data_reg(pos, 0), *((u32*)&data));
+ pci_write_config_word(dev, msi_data_reg(pos, 0), data);
+
if (entry->msi_attrib.maskbit) {
unsigned int maskbits, temp;
/* All MSIs are unmasked by default, Mask them all */
@@ -590,13 +600,15 @@
struct msix_entry *entries, int nvec)
{
struct msi_desc *head = NULL, *tail = NULL, *entry = NULL;
- struct msg_address address;
- struct msg_data data;
+ u32 address_hi;
+ u32 address_lo;
+ u32 data;
int vector, pos, i, j, nr_entries, temp = 0;
u32 phys_addr, table_offset;
u16 control;
u8 bir;
void __iomem *base;
+ int status;

pos = pci_find_capability(dev, PCI_CAP_ID_MSIX);
/* Request & Map MSI-X table region */
@@ -643,18 +655,20 @@
/* Replace with MSI-X handler */
irq_handler_init(PCI_CAP_ID_MSIX, vector, 1);
/* Configure MSI-X capability structure */
- msi_address_init(&address);
- msi_data_init(&data, vector);
- entry->msi_attrib.current_cpu =
- ((address.lo_address.u.dest_id >>
- MSI_TARGET_CPU_SHIFT) & MSI_TARGET_CPU_MASK);
- writel(address.lo_address.value,
+ status = msi_ops->setup(dev, vector,
+ &address_hi,
+ &address_lo,
+ &data);
+ if (status < 0)
+ break;
+
+ writel(address_lo,
base + j * PCI_MSIX_ENTRY_SIZE +
PCI_MSIX_ENTRY_LOWER_ADDR_OFFSET);
- writel(address.hi_address,
+ writel(address_hi,
base + j * PCI_MSIX_ENTRY_SIZE +
PCI_MSIX_ENTRY_UPPER_ADDR_OFFSET);
- writel(*(u32*)&data,
+ writel(data,
base + j * PCI_MSIX_ENTRY_SIZE +
PCI_MSIX_ENTRY_DATA_OFFSET);
attach_msi_entry(entry, vector);
@@ -789,6 +803,8 @@
void __iomem *base;
unsigned long flags;

+ msi_ops->teardown(vector);
+
spin_lock_irqsave(&msi_lock, flags);
entry = msi_desc[vector];
if (!entry || entry->dev != dev) {
Index: msi/include/asm-i386/msi.h
===================================================================
--- msi.orig/include/asm-i386/msi.h 2005-12-13 12:22:42.785246074 -0600
+++ msi/include/asm-i386/msi.h 2005-12-21 23:18:10.602753071 -0600
@@ -12,4 +12,11 @@
#define LAST_DEVICE_VECTOR 232
#define MSI_TARGET_CPU_SHIFT 12

+static inline int msi_arch_init(void)
+{
+ extern struct msi_ops msi_apic_ops;
+ msi_register(&msi_apic_ops);
+ return 0;
+}
+
#endif /* ASM_MSI_H */
Index: msi/include/asm-x86_64/msi.h
===================================================================
--- msi.orig/include/asm-x86_64/msi.h 2005-12-13 12:22:42.786222541 -0600
+++ msi/include/asm-x86_64/msi.h 2005-12-21 23:18:20.733573956 -0600
@@ -13,4 +13,11 @@
#define LAST_DEVICE_VECTOR 232
#define MSI_TARGET_CPU_SHIFT 12

+static inline int msi_arch_init(void)
+{
+ extern struct msi_ops msi_apic_ops;
+ msi_register_apic(&msi_apic_ops);
+ return 0;
+}
+
#endif /* ASM_MSI_H */
Index: msi/include/asm-ia64/machvec.h
===================================================================
--- msi.orig/include/asm-ia64/machvec.h 2005-12-13 12:22:42.786222541 -0600
+++ msi/include/asm-ia64/machvec.h 2005-12-21 23:18:32.231445414 -0600
@@ -74,6 +74,7 @@
typedef unsigned short ia64_mv_readw_relaxed_t (const volatile void __iomem *);
typedef unsigned int ia64_mv_readl_relaxed_t (const volatile void __iomem *);
typedef unsigned long ia64_mv_readq_relaxed_t (const volatile void __iomem *);
+typedef int ia64_mv_msi_init_t (void);

static inline void
machvec_noop (void)
@@ -146,6 +147,7 @@
# define platform_readw_relaxed ia64_mv.readw_relaxed
# define platform_readl_relaxed ia64_mv.readl_relaxed
# define platform_readq_relaxed ia64_mv.readq_relaxed
+# define platform_msi_init ia64_mv.msi_init
# endif

/* __attribute__((__aligned__(16))) is required to make size of the
@@ -194,6 +196,7 @@
ia64_mv_readw_relaxed_t *readw_relaxed;
ia64_mv_readl_relaxed_t *readl_relaxed;
ia64_mv_readq_relaxed_t *readq_relaxed;
+ ia64_mv_msi_init_t *msi_init;
} __attribute__((__aligned__(16))); /* align attrib? see above comment */

#define MACHVEC_INIT(name) \
@@ -238,6 +241,7 @@
platform_readw_relaxed, \
platform_readl_relaxed, \
platform_readq_relaxed, \
+ platform_msi_init, \
}

extern struct ia64_machine_vector ia64_mv;
@@ -386,5 +390,9 @@
#ifndef platform_readq_relaxed
# define platform_readq_relaxed __ia64_readq_relaxed
#endif
+#ifndef platform_msi_init
+# define platform_msi_init { extern struct msi_ops msi_apic_ops; \
+ msi_register(&msi_apic_ops); return 0; }
+#endif

#endif /* _ASM_IA64_MACHVEC_H */
Index: msi/include/asm-ia64/machvec_sn2.h
===================================================================
--- msi.orig/include/asm-ia64/machvec_sn2.h 2005-12-13 12:22:42.787199008 -0600
+++ msi/include/asm-ia64/machvec_sn2.h 2005-12-13 16:09:49.257035213 -0600
@@ -71,6 +71,7 @@
extern ia64_mv_dma_sync_sg_for_device sn_dma_sync_sg_for_device;
extern ia64_mv_dma_mapping_error sn_dma_mapping_error;
extern ia64_mv_dma_supported sn_dma_supported;
+extern ia64_mv_msi_init_t sn_msi_init;

/*
* This stuff has dual use!
@@ -120,6 +121,7 @@
#define platform_dma_sync_sg_for_device sn_dma_sync_sg_for_device
#define platform_dma_mapping_error sn_dma_mapping_error
#define platform_dma_supported sn_dma_supported
+#define platform_msi_init sn_msi_init

#include <asm/sn/io.h>

Index: msi/include/asm-ia64/msi.h
===================================================================
--- msi.orig/include/asm-ia64/msi.h 2005-12-13 12:22:42.787199008 -0600
+++ msi/include/asm-ia64/msi.h 2005-12-13 16:09:49.268752815 -0600
@@ -14,4 +14,6 @@
#define ack_APIC_irq ia64_eoi
#define MSI_TARGET_CPU_SHIFT 4

+static inline int msi_arch_init(void) { return platform_msi_init(); }
+
#endif /* ASM_MSI_H */
Index: msi/arch/ia64/sn/pci/Makefile
===================================================================
--- msi.orig/arch/ia64/sn/pci/Makefile 2005-12-13 12:22:42.788175474 -0600
+++ msi/arch/ia64/sn/pci/Makefile 2005-12-13 16:09:49.296093887 -0600
@@ -3,8 +3,9 @@
# License. See the file "COPYING" in the main directory of this archive
# for more details.
#
-# Copyright (C) 2000-2004 Silicon Graphics, Inc. All Rights Reserved.
+# Copyright (C) 2000-2005 Silicon Graphics, Inc. All Rights Reserved.
#
# Makefile for the sn pci general routines.

obj-y := pci_dma.o tioca_provider.o tioce_provider.o pcibr/
+obj-$(CONFIG_PCI_MSI) += msi.o
Index: msi/arch/ia64/sn/pci/msi.c
===================================================================
--- /dev/null 1970-01-01 00:00:00.000000000 +0000
+++ msi/arch/ia64/sn/pci/msi.c 2005-12-21 22:59:02.713172526 -0600
@@ -0,0 +1,18 @@
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * Copyright (C) 2005 Silicon Graphics, Inc. All Rights Reserved.
+ */
+
+#include <asm/errno.h>
+
+int
+sn_msi_init(void)
+{
+ /*
+ * return error until MSI is supported on altix platforms
+ */
+ return -EINVAL;
+}
Index: msi/drivers/pci/Makefile
===================================================================
--- msi.orig/drivers/pci/Makefile 2005-12-21 16:08:58.222841575 -0600
+++ msi/drivers/pci/Makefile 2005-12-21 16:10:32.848440390 -0600
@@ -23,7 +23,7 @@
obj-$(CONFIG_PPC64) += setup-bus.o
obj-$(CONFIG_MIPS) += setup-bus.o setup-irq.o
obj-$(CONFIG_X86_VISWS) += setup-irq.o
-obj-$(CONFIG_PCI_MSI) += msi.o
+obj-$(CONFIG_PCI_MSI) += msi.o msi-apic.o

#
# ACPI Related PCI FW Functions
Index: msi/drivers/pci/msi-apic.c
===================================================================
--- /dev/null 1970-01-01 00:00:00.000000000 +0000
+++ msi/drivers/pci/msi-apic.c 2005-12-22 11:09:37.022232088 -0600
@@ -0,0 +1,102 @@
+/*
+ * MSI hooks for standard x86 apic
+ */
+
+#include <linux/pci.h>
+
+#include "msi.h"
+
+/*
+ * Shifts for APIC-based data
+ */
+
+#define MSI_DATA_VECTOR_SHIFT 0
+#define MSI_DATA_VECTOR(v) (((u8)v) << MSI_DATA_VECTOR_SHIFT)
+
+#define MSI_DATA_DELIVERY_SHIFT 8
+#define MSI_DATA_DELIVERY_FIXED (0 << MSI_DATA_DELIVERY_SHIFT)
+#define MSI_DATA_DELIVERY_LOWPRI (1 << MSI_DATA_DELIVERY_SHIFT)
+
+#define MSI_DATA_LEVEL_SHIFT 14
+#define MSI_DATA_LEVEL_DEASSERT (0 << MSI_DATA_LEVEL_SHIFT)
+#define MSI_DATA_LEVEL_ASSERT (1 << MSI_DATA_LEVEL_SHIFT)
+
+#define MSI_DATA_TRIGGER_SHIFT 15
+#define MSI_DATA_TRIGGER_EDGE (0 << MSI_DATA_TRIGGER_SHIFT)
+#define MSI_DATA_TRIGGER_LEVEL (1 << MSI_DATA_TRIGGER_SHIFT)
+
+/*
+ * Shift/mask fields for APIC-based bus address
+ */
+
+#define MSI_ADDR_HEADER 0xfee00000
+
+#define MSI_ADDR_DESTID_MASK 0xfff0000f
+#define MSI_ADDR_DESTID_CPU(cpu) ((cpu) << MSI_TARGET_CPU_SHIFT)
+
+#define MSI_ADDR_DESTMODE_SHIFT 2
+#define MSI_ADDR_DESTMODE_PHYS (0 << MSI_ADDR_DESTMODE_SHIFT)
+#define MSI_ADDR_DESTMODE_LOGIC (1 << MSI_ADDR_DESTMODE_SHIFT)
+
+#define MSI_ADDR_REDIRECTION_SHIFT 3
+#define MSI_ADDR_REDIRECTION_CPU (0 << MSI_ADDR_REDIRECTION_SHIFT)
+#define MSI_ADDR_REDIRECTION_LOWPRI (1 << MSI_ADDR_REDIRECTION_SHIFT)
+
+
+static void
+msi_target_apic(unsigned int vector,
+ unsigned int dest_cpu,
+ u32 *address_hi, /* in/out */
+ u32 *address_lo) /* in/out */
+{
+ u32 addr = *address_lo;
+
+ addr &= MSI_ADDR_DESTID_MASK;
+ addr |= MSI_ADDR_DESTID_CPU(cpu_physical_id(dest_cpu));
+
+ *address_lo = addr;
+}
+
+static int
+msi_setup_apic(struct pci_dev *pdev, /* unused in generic */
+ unsigned int vector,
+ u32 *address_hi,
+ u32 *address_lo,
+ u32 *data)
+{
+ unsigned long dest_phys_id;
+
+ dest_phys_id = cpu_physical_id(first_cpu(cpu_online_map));
+
+ *address_hi = 0;
+ *address_lo = MSI_ADDR_HEADER |
+ MSI_ADDR_DESTMODE_PHYS |
+ MSI_ADDR_REDIRECTION_CPU |
+ MSI_ADDR_DESTID_CPU(dest_phys_id);
+
+ *data = MSI_DATA_TRIGGER_EDGE |
+ MSI_DATA_LEVEL_ASSERT |
+ MSI_DATA_DELIVERY_FIXED |
+ MSI_DATA_VECTOR(vector);
+
+ return 0;
+}
+
+static void
+msi_teardown_apic(unsigned int vector)
+{
+ return; /* no-op */
+}
+
+/*
+ * Generic callouts used on most archs/platforms. Override with
+ * msi_register_callouts()
+ */
+
+struct msi_ops msi_apic_ops = {
+ .setup = msi_setup_apic,
+ .teardown = msi_teardown_apic,
+#ifdef CONFIG_SMP
+ .target = msi_target_apic,
+#endif
+};
Index: msi/drivers/pci/msi.h
===================================================================
--- msi.orig/drivers/pci/msi.h 2005-12-21 16:08:58.222841575 -0600
+++ msi/drivers/pci/msi.h 2005-12-21 23:06:48.292305590 -0600
@@ -69,67 +69,6 @@
#define msix_mask(address) (address | PCI_MSIX_FLAGS_BITMASK)
#define msix_is_pending(address) (address & PCI_MSIX_FLAGS_PENDMASK)

-/*
- * MSI Defined Data Structures
- */
-#define MSI_ADDRESS_HEADER 0xfee
-#define MSI_ADDRESS_HEADER_SHIFT 12
-#define MSI_ADDRESS_HEADER_MASK 0xfff000
-#define MSI_ADDRESS_DEST_ID_MASK 0xfff0000f
-#define MSI_TARGET_CPU_MASK 0xff
-#define MSI_DELIVERY_MODE 0
-#define MSI_LEVEL_MODE 1 /* Edge always assert */
-#define MSI_TRIGGER_MODE 0 /* MSI is edge sensitive */
-#define MSI_PHYSICAL_MODE 0
-#define MSI_LOGICAL_MODE 1
-#define MSI_REDIRECTION_HINT_MODE 0
-
-struct msg_data {
-#if defined(__LITTLE_ENDIAN_BITFIELD)
- __u32 vector : 8;
- __u32 delivery_mode : 3; /* 000b: FIXED | 001b: lowest prior */
- __u32 reserved_1 : 3;
- __u32 level : 1; /* 0: deassert | 1: assert */
- __u32 trigger : 1; /* 0: edge | 1: level */
- __u32 reserved_2 : 16;
-#elif defined(__BIG_ENDIAN_BITFIELD)
- __u32 reserved_2 : 16;
- __u32 trigger : 1; /* 0: edge | 1: level */
- __u32 level : 1; /* 0: deassert | 1: assert */
- __u32 reserved_1 : 3;
- __u32 delivery_mode : 3; /* 000b: FIXED | 001b: lowest prior */
- __u32 vector : 8;
-#else
-#error "Bitfield endianness not defined! Check your byteorder.h"
-#endif
-} __attribute__ ((packed));
-
-struct msg_address {
- union {
- struct {
-#if defined(__LITTLE_ENDIAN_BITFIELD)
- __u32 reserved_1 : 2;
- __u32 dest_mode : 1; /*0:physic | 1:logic */
- __u32 redirection_hint: 1; /*0: dedicated CPU
- 1: lowest priority */
- __u32 reserved_2 : 4;
- __u32 dest_id : 24; /* Destination ID */
-#elif defined(__BIG_ENDIAN_BITFIELD)
- __u32 dest_id : 24; /* Destination ID */
- __u32 reserved_2 : 4;
- __u32 redirection_hint: 1; /*0: dedicated CPU
- 1: lowest priority */
- __u32 dest_mode : 1; /*0:physic | 1:logic */
- __u32 reserved_1 : 2;
-#else
-#error "Bitfield endianness not defined! Check your byteorder.h"
-#endif
- }u;
- __u32 value;
- }lo_address;
- __u32 hi_address;
-} __attribute__ ((packed));
-
struct msi_desc {
struct {
__u8 type : 5; /* {0: unused, 5h:MSI, 11h:MSI-X} */
@@ -138,7 +77,7 @@
__u8 reserved: 1; /* reserved */
__u8 entry_nr; /* specific enabled entry */
__u8 default_vector; /* default pre-assigned vector */
- __u8 current_cpu; /* current destination cpu */
+ __u8 unused; /* formerly unused destination cpu*/
}msi_attrib;

struct {
Index: msi/include/linux/pci.h
===================================================================
--- msi.orig/include/linux/pci.h 2005-12-21 16:08:58.223818043 -0600
+++ msi/include/linux/pci.h 2005-12-21 16:10:32.847463922 -0600
@@ -478,6 +478,16 @@
u16 entry; /* driver uses to specify entry, OS writes */
};

+struct msi_ops {
+ int (*setup) (struct pci_dev *pdev, unsigned int vector,
+ u32 *addr_hi, u32 *addr_lo, u32 *data);
+ void (*teardown) (unsigned int vector);
+#ifdef CONFIG_SMP
+ void (*target) (unsigned int vector, unsigned int cpu,
+ u32 *addr_hi, u32 *addr_lo);
+#endif
+};
+
#ifndef CONFIG_PCI_MSI
static inline void pci_scan_msi_device(struct pci_dev *dev) {}
static inline int pci_enable_msi(struct pci_dev *dev) {return -1;}
@@ -486,6 +496,7 @@
struct msix_entry *entries, int nvec) {return -1;}
static inline void pci_disable_msix(struct pci_dev *dev) {}
static inline void msi_remove_pci_irq_vectors(struct pci_dev *dev) {}
+static inline int msi_register(struct msi_ops *ops) {return -1;}
#else
extern void pci_scan_msi_device(struct pci_dev *dev);
extern int pci_enable_msi(struct pci_dev *dev);
@@ -494,6 +505,7 @@
struct msix_entry *entries, int nvec);
extern void pci_disable_msix(struct pci_dev *dev);
extern void msi_remove_pci_irq_vectors(struct pci_dev *dev);
+extern int msi_register(struct msi_ops *ops);
#endif

extern void pci_block_user_cfg_access(struct pci_dev *dev);

2005-12-22 20:17:35

by Mark Maule

[permalink] [raw]
Subject: [PATCH 2/3] altix: msi support

MSI callouts for altix. Involves a fair amount of code reorg in sn irq.c
code as well as adding some extensions to the altix PCI provider abstaction.

Signed-off-by: Mark Maule <[email protected]>

Index: linux-2.6.14/arch/ia64/sn/pci/msi.c
===================================================================
--- linux-2.6.14.orig/arch/ia64/sn/pci/msi.c 2005-12-21 22:37:02.978262311 -0600
+++ linux-2.6.14/arch/ia64/sn/pci/msi.c 2005-12-21 22:38:24.090685912 -0600
@@ -6,13 +6,205 @@
* Copyright (C) 2005 Silicon Graphics, Inc. All Rights Reserved.
*/

-#include <asm/errno.h>
+#include <linux/types.h>
+#include <linux/pci.h>
+#include <linux/cpumask.h>
+
+#include <asm/msi.h>
+
+#include <asm/sn/addrs.h>
+#include <asm/sn/intr.h>
+#include <asm/sn/pcibus_provider_defs.h>
+#include <asm/sn/pcidev.h>
+#include <asm/sn/nodepda.h>
+
+struct sn_msi_info {
+ u64 pci_addr;
+ struct sn_irq_info *sn_irq_info;
+};
+
+static struct sn_msi_info *sn_msi_info;
+
+static void
+sn_msi_teardown(unsigned int vector)
+{
+ nasid_t nasid;
+ int widget;
+ struct pci_dev *pdev;
+ struct pcidev_info *sn_pdev;
+ struct sn_irq_info *sn_irq_info;
+ struct pcibus_bussoft *bussoft;
+ struct sn_pcibus_provider *provider;
+
+ sn_irq_info = sn_msi_info[vector].sn_irq_info;
+ if (sn_irq_info == NULL || sn_irq_info->irq_int_bit >= 0)
+ return;
+
+ sn_pdev = (struct pcidev_info *)sn_irq_info->irq_pciioinfo;
+ pdev = sn_pdev->pdi_linux_pcidev;
+ provider = SN_PCIDEV_BUSPROVIDER(pdev);
+
+ (*provider->dma_unmap)(pdev,
+ sn_msi_info[vector].pci_addr,
+ PCI_DMA_FROMDEVICE);
+ sn_msi_info[vector].pci_addr = 0;
+
+ bussoft = SN_PCIDEV_BUSSOFT(pdev);
+ nasid = NASID_GET(bussoft->bs_base);
+ widget = (nasid & 1) ?
+ TIO_SWIN_WIDGETNUM(bussoft->bs_base) :
+ SWIN_WIDGETNUM(bussoft->bs_base);
+
+ sn_intr_free(nasid, widget, sn_irq_info);
+ sn_msi_info[vector].sn_irq_info = NULL;
+
+ return;
+}

int
-sn_msi_init(void)
+sn_msi_setup(struct pci_dev *pdev, unsigned int vector,
+ u32 *addr_hi, u32 *addr_lo, u32 *data)
{
+ int widget;
+ int status;
+ nasid_t nasid;
+ u64 bus_addr;
+ struct sn_irq_info *sn_irq_info;
+ struct pcibus_bussoft *bussoft = SN_PCIDEV_BUSSOFT(pdev);
+ struct sn_pcibus_provider *provider = SN_PCIDEV_BUSPROVIDER(pdev);
+
+ if (bussoft == NULL)
+ return -EINVAL;
+
+ if (provider == NULL || provider->dma_map_consistent == NULL)
+ return -EINVAL;
+
+ /*
+ * Set up the vector plumbing. Let the prom (via sn_intr_alloc)
+ * decide which cpu to direct this msi at by default.
+ */
+
+ nasid = NASID_GET(bussoft->bs_base);
+ widget = (nasid & 1) ?
+ TIO_SWIN_WIDGETNUM(bussoft->bs_base) :
+ SWIN_WIDGETNUM(bussoft->bs_base);
+
+ sn_irq_info = kzalloc(sizeof(struct sn_irq_info), GFP_KERNEL);
+ if (! sn_irq_info)
+ return -ENOMEM;
+
+ status = sn_intr_alloc(nasid, widget, sn_irq_info, vector, -1, -1);
+ if (status) {
+ kfree(sn_irq_info);
+ return -ENOMEM;
+ }
+
+ sn_irq_info->irq_int_bit = -1; /* mark this as an MSI irq */
+ sn_irq_fixup(pdev, sn_irq_info);
+
+ /* Prom probably should fill these in, but doesn't ... */
+ sn_irq_info->irq_bridge_type = bussoft->bs_asic_type;
+ sn_irq_info->irq_bridge = (void *)bussoft->bs_base;
+
/*
- * return error until MSI is supported on altix platforms
+ * Map the xio address into bus space
*/
- return -EINVAL;
+ bus_addr = (*provider->dma_map_consistent)(pdev,
+ sn_irq_info->irq_xtalkaddr,
+ sizeof(sn_irq_info->irq_xtalkaddr),
+ SN_DMA_MSI|SN_DMA_ADDR_XIO);
+ if (! bus_addr) {
+ sn_intr_free(nasid, widget, sn_irq_info);
+ kfree(sn_irq_info);
+ return -ENOMEM;
+ }
+
+ sn_msi_info[vector].sn_irq_info = sn_irq_info;
+ sn_msi_info[vector].pci_addr = bus_addr;
+
+ *addr_hi = (u32)(bus_addr >> 32);
+ *addr_lo = (u32)(bus_addr & 0x00000000ffffffff);
+
+ /*
+ * In the SN platform, bit 16 is a "send vector" bit which
+ * must be present in order to move the vector through the system.
+ */
+ *data = 0x100 + (unsigned int)vector;
+
+#ifdef CONFIG_SMP
+ set_irq_affinity_info((vector & 0xff), sn_irq_info->irq_cpuid, 0);
+#endif
+
+ return 0;
+}
+
+static void
+sn_msi_target(unsigned int vector, unsigned int cpu,
+ u32 *addr_hi, u32 *addr_lo)
+{
+ int slice;
+ nasid_t nasid;
+ u64 bus_addr;
+ struct pci_dev *pdev;
+ struct pcidev_info *sn_pdev;
+ struct sn_irq_info *sn_irq_info;
+ struct sn_irq_info *new_irq_info;
+ struct sn_pcibus_provider *provider;
+
+ sn_irq_info = sn_msi_info[vector].sn_irq_info;
+ if (sn_irq_info == NULL || sn_irq_info->irq_int_bit >= 0)
+ return;
+
+ /*
+ * Release XIO resources for the old MSI PCI address
+ */
+
+ sn_pdev = (struct pcidev_info *)sn_irq_info->irq_pciioinfo;
+ pdev = sn_pdev->pdi_linux_pcidev;
+ provider = SN_PCIDEV_BUSPROVIDER(pdev);
+
+ bus_addr = (u64)(*addr_hi) << 32 | (u64)(*addr_lo);
+ (*provider->dma_unmap)(pdev, bus_addr, PCI_DMA_FROMDEVICE);
+ sn_msi_info[vector].pci_addr = 0;
+
+ nasid = cpuid_to_nasid(cpu);
+ slice = cpuid_to_slice(cpu);
+
+ new_irq_info = sn_retarget_vector(sn_irq_info, nasid, slice);
+ sn_msi_info[vector].sn_irq_info = new_irq_info;
+ if (new_irq_info == NULL)
+ return;
+
+ /*
+ * Map the xio address into bus space
+ */
+
+ bus_addr = (*provider->dma_map_consistent)(pdev,
+ new_irq_info->irq_xtalkaddr,
+ sizeof(new_irq_info->irq_xtalkaddr),
+ SN_DMA_MSI|SN_DMA_ADDR_XIO);
+
+ sn_msi_info[vector].pci_addr = bus_addr;
+ *addr_hi = (u32)(bus_addr >> 32);
+ *addr_lo = (u32)(bus_addr & 0x00000000ffffffff);
+}
+
+struct msi_ops sn_msi_ops = {
+ .setup = sn_msi_setup,
+ .teardown = sn_msi_teardown,
+#ifdef CONFIG_SMP
+ .target = sn_msi_target,
+#endif
+};
+
+int
+sn_msi_init(void)
+{
+ sn_msi_info =
+ kzalloc(sizeof(struct sn_msi_info) * NR_VECTORS, GFP_KERNEL);
+ if (! sn_msi_info)
+ return -ENOMEM;
+
+ msi_register(&sn_msi_ops);
+ return 0;
}
Index: linux-2.6.14/arch/ia64/sn/kernel/io_init.c
===================================================================
--- linux-2.6.14.orig/arch/ia64/sn/kernel/io_init.c 2005-12-21 22:35:23.436286704 -0600
+++ linux-2.6.14/arch/ia64/sn/kernel/io_init.c 2005-12-21 22:38:24.091685671 -0600
@@ -51,7 +51,7 @@
*/

static dma_addr_t
-sn_default_pci_map(struct pci_dev *pdev, unsigned long paddr, size_t size)
+sn_default_pci_map(struct pci_dev *pdev, unsigned long paddr, size_t size, int type)
{
return 0;
}
Index: linux-2.6.14/arch/ia64/sn/kernel/irq.c
===================================================================
--- linux-2.6.14.orig/arch/ia64/sn/kernel/irq.c 2005-12-21 22:37:58.120953676 -0600
+++ linux-2.6.14/arch/ia64/sn/kernel/irq.c 2005-12-21 22:38:24.092685429 -0600
@@ -25,11 +25,11 @@

int sn_force_interrupt_flag = 1;
extern int sn_ioif_inited;
-static struct list_head **sn_irq_lh;
+struct list_head **sn_irq_lh;
static spinlock_t sn_irq_info_lock = SPIN_LOCK_UNLOCKED; /* non-IRQ lock */

-static inline uint64_t sn_intr_alloc(nasid_t local_nasid, int local_widget,
- u64 sn_irq_info,
+uint64_t sn_intr_alloc(nasid_t local_nasid, int local_widget,
+ struct sn_irq_info *sn_irq_info,
int req_irq, nasid_t req_nasid,
int req_slice)
{
@@ -39,12 +39,13 @@

SAL_CALL_NOLOCK(ret_stuff, (u64) SN_SAL_IOIF_INTERRUPT,
(u64) SAL_INTR_ALLOC, (u64) local_nasid,
- (u64) local_widget, (u64) sn_irq_info, (u64) req_irq,
+ (u64) local_widget, __pa(sn_irq_info), (u64) req_irq,
(u64) req_nasid, (u64) req_slice);
+
return ret_stuff.status;
}

-static inline void sn_intr_free(nasid_t local_nasid, int local_widget,
+void sn_intr_free(nasid_t local_nasid, int local_widget,
struct sn_irq_info *sn_irq_info)
{
struct ia64_sal_retval ret_stuff;
@@ -113,73 +114,91 @@

static void sn_irq_info_free(struct rcu_head *head);

-static void sn_set_affinity_irq(unsigned int irq, cpumask_t mask)
+struct sn_irq_info *sn_retarget_vector(struct sn_irq_info *sn_irq_info,
+ nasid_t nasid, int slice)
{
- struct sn_irq_info *sn_irq_info, *sn_irq_info_safe;
- int cpuid, cpuphys;
+ int vector;
+ int cpuphys;
+ int64_t bridge;
+ int local_widget, status;
+ nasid_t local_nasid;
+ struct sn_irq_info *new_irq_info;
+ struct sn_pcibus_provider *pci_provider;

- cpuid = first_cpu(mask);
- cpuphys = cpu_physical_id(cpuid);
+ new_irq_info = kmalloc(sizeof(struct sn_irq_info), GFP_ATOMIC);
+ if (new_irq_info == NULL)
+ return NULL;

- list_for_each_entry_safe(sn_irq_info, sn_irq_info_safe,
- sn_irq_lh[irq], list) {
- uint64_t bridge;
- int local_widget, status;
- nasid_t local_nasid;
- struct sn_irq_info *new_irq_info;
- struct sn_pcibus_provider *pci_provider;
-
- new_irq_info = kmalloc(sizeof(struct sn_irq_info), GFP_ATOMIC);
- if (new_irq_info == NULL)
- break;
- memcpy(new_irq_info, sn_irq_info, sizeof(struct sn_irq_info));
-
- bridge = (uint64_t) new_irq_info->irq_bridge;
- if (!bridge) {
- kfree(new_irq_info);
- break; /* irq is not a device interrupt */
- }
+ memcpy(new_irq_info, sn_irq_info, sizeof(struct sn_irq_info));

- local_nasid = NASID_GET(bridge);
+ bridge = (uint64_t) new_irq_info->irq_bridge;
+ if (!bridge) {
+ kfree(new_irq_info);
+ return NULL; /* irq is not a device interrupt */
+ }

- if (local_nasid & 1)
- local_widget = TIO_SWIN_WIDGETNUM(bridge);
- else
- local_widget = SWIN_WIDGETNUM(bridge);
-
- /* Free the old PROM new_irq_info structure */
- sn_intr_free(local_nasid, local_widget, new_irq_info);
- /* Update kernels new_irq_info with new target info */
- unregister_intr_pda(new_irq_info);
-
- /* allocate a new PROM new_irq_info struct */
- status = sn_intr_alloc(local_nasid, local_widget,
- __pa(new_irq_info), irq,
- cpuid_to_nasid(cpuid),
- cpuid_to_slice(cpuid));
-
- /* SAL call failed */
- if (status) {
- kfree(new_irq_info);
- break;
- }
+ local_nasid = NASID_GET(bridge);

- new_irq_info->irq_cpuid = cpuid;
- register_intr_pda(new_irq_info);
+ if (local_nasid & 1)
+ local_widget = TIO_SWIN_WIDGETNUM(bridge);
+ else
+ local_widget = SWIN_WIDGETNUM(bridge);

- pci_provider = sn_pci_provider[new_irq_info->irq_bridge_type];
- if (pci_provider && pci_provider->target_interrupt)
- (pci_provider->target_interrupt)(new_irq_info);
-
- spin_lock(&sn_irq_info_lock);
- list_replace_rcu(&sn_irq_info->list, &new_irq_info->list);
- spin_unlock(&sn_irq_info_lock);
- call_rcu(&sn_irq_info->rcu, sn_irq_info_free);
+ vector = sn_irq_info->irq_irq;
+ /* Free the old PROM new_irq_info structure */
+ sn_intr_free(local_nasid, local_widget, new_irq_info);
+ /* Update kernels new_irq_info with new target info */
+ unregister_intr_pda(new_irq_info);
+
+ /* allocate a new PROM new_irq_info struct */
+ status = sn_intr_alloc(local_nasid, local_widget,
+ new_irq_info, vector,
+ nasid, slice);
+
+ /* SAL call failed */
+ if (status) {
+ kfree(new_irq_info);
+ return NULL;
+ }
+
+ cpuphys = nasid_slice_to_cpuid(nasid, slice);
+ new_irq_info->irq_cpuid = cpuphys;
+ register_intr_pda(new_irq_info);
+
+ pci_provider = sn_pci_provider[new_irq_info->irq_bridge_type];
+
+ /*
+ * If this represents a line interrupt, target it. If it's
+ * an msi (irq_int_bit < 0), it's already targeted.
+ */
+ if (new_irq_info->irq_int_bit >= 0 &&
+ pci_provider && pci_provider->target_interrupt)
+ (pci_provider->target_interrupt)(new_irq_info);
+
+ spin_lock(&sn_irq_info_lock);
+ list_replace_rcu(&sn_irq_info->list, &new_irq_info->list);
+ spin_unlock(&sn_irq_info_lock);
+ call_rcu(&sn_irq_info->rcu, sn_irq_info_free);

#ifdef CONFIG_SMP
- set_irq_affinity_info((irq & 0xff), cpuphys, 0);
+ set_irq_affinity_info((vector & 0xff), cpuphys, 0);
#endif
- }
+
+ return new_irq_info;
+}
+
+static void sn_set_affinity_irq(unsigned int irq, cpumask_t mask)
+{
+ struct sn_irq_info *sn_irq_info, *sn_irq_info_safe;
+ nasid_t nasid;
+ int slice;
+
+ nasid = cpuid_to_nasid(first_cpu(mask));
+ slice = cpuid_to_slice(first_cpu(mask));
+
+ list_for_each_entry_safe(sn_irq_info, sn_irq_info_safe,
+ sn_irq_lh[irq], list)
+ (void)sn_retarget_vector(sn_irq_info, nasid, slice);
}

struct hw_interrupt_type irq_type_sn = {
@@ -441,5 +460,4 @@

INIT_LIST_HEAD(sn_irq_lh[i]);
}
-
}
Index: linux-2.6.14/arch/ia64/sn/pci/pci_dma.c
===================================================================
--- linux-2.6.14.orig/arch/ia64/sn/pci/pci_dma.c 2005-12-21 22:35:23.446284291 -0600
+++ linux-2.6.14/arch/ia64/sn/pci/pci_dma.c 2005-12-21 22:38:24.092685429 -0600
@@ -11,7 +11,7 @@

#include <linux/module.h>
#include <asm/dma.h>
-#include <asm/sn/pcibr_provider.h>
+#include <asm/sn/intr.h>
#include <asm/sn/pcibus_provider_defs.h>
#include <asm/sn/pcidev.h>
#include <asm/sn/sn_sal.h>
@@ -113,7 +113,8 @@
* resources.
*/

- *dma_handle = provider->dma_map_consistent(pdev, phys_addr, size);
+ *dma_handle = provider->dma_map_consistent(pdev, phys_addr, size,
+ SN_DMA_ADDR_PHYS);
if (!*dma_handle) {
printk(KERN_ERR "%s: out of ATEs\n", __FUNCTION__);
free_pages((unsigned long)cpuaddr, get_order(size));
@@ -176,7 +177,7 @@
BUG_ON(dev->bus != &pci_bus_type);

phys_addr = __pa(cpu_addr);
- dma_addr = provider->dma_map(pdev, phys_addr, size);
+ dma_addr = provider->dma_map(pdev, phys_addr, size, SN_DMA_ADDR_PHYS);
if (!dma_addr) {
printk(KERN_ERR "%s: out of ATEs\n", __FUNCTION__);
return 0;
@@ -260,7 +261,8 @@
for (i = 0; i < nhwentries; i++, sg++) {
phys_addr = SG_ENT_PHYS_ADDRESS(sg);
sg->dma_address = provider->dma_map(pdev,
- phys_addr, sg->length);
+ phys_addr, sg->length,
+ SN_DMA_ADDR_PHYS);

if (!sg->dma_address) {
printk(KERN_ERR "%s: out of ATEs\n", __FUNCTION__);
Index: linux-2.6.14/arch/ia64/sn/pci/pcibr/pcibr_dma.c
===================================================================
--- linux-2.6.14.orig/arch/ia64/sn/pci/pcibr/pcibr_dma.c 2005-12-21 22:35:23.445284532 -0600
+++ linux-2.6.14/arch/ia64/sn/pci/pcibr/pcibr_dma.c 2005-12-21 22:38:24.093685188 -0600
@@ -41,7 +41,7 @@

static dma_addr_t
pcibr_dmamap_ate32(struct pcidev_info *info,
- uint64_t paddr, size_t req_size, uint64_t flags)
+ uint64_t paddr, size_t req_size, uint64_t flags, int dma_flags)
{

struct pcidev_info *pcidev_info = info->pdi_host_pcidev_info;
@@ -81,9 +81,12 @@
if (IS_PCIX(pcibus_info))
ate_flags &= ~(PCI32_ATE_PREF);

- xio_addr =
- IS_PIC_SOFT(pcibus_info) ? PHYS_TO_DMA(paddr) :
- PHYS_TO_TIODMA(paddr);
+ if (SN_DMA_ADDRTYPE(dma_flags == SN_DMA_ADDR_PHYS))
+ xio_addr = IS_PIC_SOFT(pcibus_info) ? PHYS_TO_DMA(paddr) :
+ PHYS_TO_TIODMA(paddr);
+ else
+ xio_addr = paddr;
+
offset = IOPGOFF(xio_addr);
ate = ate_flags | (xio_addr - offset);

@@ -91,6 +94,13 @@
if (IS_PIC_SOFT(pcibus_info)) {
ate |= (pcibus_info->pbi_hub_xid << PIC_ATE_TARGETID_SHFT);
}
+
+ /*
+ * If we're mapping for MSI, set the MSI bit in the ATE
+ */
+ if (dma_flags & SN_DMA_MSI)
+ ate |= PCI32_ATE_MSI;
+
ate_write(pcibus_info, ate_index, ate_count, ate);

/*
@@ -105,20 +115,27 @@
if (pcibus_info->pbi_devreg[internal_device] & PCIBR_DEV_SWAP_DIR)
ATE_SWAP_ON(pci_addr);

+
return pci_addr;
}

static dma_addr_t
pcibr_dmatrans_direct64(struct pcidev_info * info, uint64_t paddr,
- uint64_t dma_attributes)
+ uint64_t dma_attributes, int dma_flags)
{
struct pcibus_info *pcibus_info = (struct pcibus_info *)
((info->pdi_host_pcidev_info)->pdi_pcibus_info);
uint64_t pci_addr;

/* Translate to Crosstalk View of Physical Address */
- pci_addr = (IS_PIC_SOFT(pcibus_info) ? PHYS_TO_DMA(paddr) :
- PHYS_TO_TIODMA(paddr)) | dma_attributes;
+ if (SN_DMA_ADDRTYPE(dma_flags) == SN_DMA_ADDR_PHYS)
+ pci_addr = IS_PIC_SOFT(pcibus_info) ?
+ PHYS_TO_DMA(paddr) :
+ PHYS_TO_TIODMA(paddr) | dma_attributes;
+ else
+ pci_addr = IS_PIC_SOFT(pcibus_info) ?
+ paddr :
+ paddr | dma_attributes;

/* Handle Bus mode */
if (IS_PCIX(pcibus_info))
@@ -130,7 +147,9 @@
((uint64_t) pcibus_info->
pbi_hub_xid << PIC_PCI64_ATTR_TARG_SHFT);
} else
- pci_addr |= TIOCP_PCI64_CMDTYPE_MEM;
+ pci_addr |= (dma_flags & SN_DMA_MSI) ?
+ TIOCP_PCI64_CMDTYPE_MSI :
+ TIOCP_PCI64_CMDTYPE_MEM;

/* If PCI mode, func zero uses VCHAN0, every other func uses VCHAN1 */
if (!IS_PCIX(pcibus_info) && PCI_FUNC(info->pdi_linux_pcidev->devfn))
@@ -142,7 +161,7 @@

static dma_addr_t
pcibr_dmatrans_direct32(struct pcidev_info * info,
- uint64_t paddr, size_t req_size, uint64_t flags)
+ uint64_t paddr, size_t req_size, uint64_t flags, int dma_flags)
{

struct pcidev_info *pcidev_info = info->pdi_host_pcidev_info;
@@ -158,8 +177,14 @@
return 0;
}

- xio_addr = IS_PIC_SOFT(pcibus_info) ? PHYS_TO_DMA(paddr) :
- PHYS_TO_TIODMA(paddr);
+ if (dma_flags & SN_DMA_MSI)
+ return 0;
+
+ if (SN_DMA_ADDRTYPE(dma_flags) == SN_DMA_ADDR_PHYS)
+ xio_addr = IS_PIC_SOFT(pcibus_info) ? PHYS_TO_DMA(paddr) :
+ PHYS_TO_TIODMA(paddr);
+ else
+ xio_addr = paddr;

xio_base = pcibus_info->pbi_dir_xbase;
offset = xio_addr - xio_base;
@@ -331,7 +356,7 @@
*/

dma_addr_t
-pcibr_dma_map(struct pci_dev * hwdev, unsigned long phys_addr, size_t size)
+pcibr_dma_map(struct pci_dev * hwdev, unsigned long phys_addr, size_t size, int dma_flags)
{
dma_addr_t dma_handle;
struct pcidev_info *pcidev_info = SN_PCIDEV_INFO(hwdev);
@@ -348,11 +373,11 @@
*/

dma_handle = pcibr_dmatrans_direct64(pcidev_info, phys_addr,
- PCI64_ATTR_PREF);
+ PCI64_ATTR_PREF, dma_flags);
} else {
/* Handle 32-63 bit cards via direct mapping */
dma_handle = pcibr_dmatrans_direct32(pcidev_info, phys_addr,
- size, 0);
+ size, 0, dma_flags);
if (!dma_handle) {
/*
* It is a 32 bit card and we cannot do direct mapping,
@@ -360,7 +385,8 @@
*/

dma_handle = pcibr_dmamap_ate32(pcidev_info, phys_addr,
- size, PCI32_ATE_PREF);
+ size, PCI32_ATE_PREF,
+ dma_flags);
}
}

@@ -369,18 +395,18 @@

dma_addr_t
pcibr_dma_map_consistent(struct pci_dev * hwdev, unsigned long phys_addr,
- size_t size)
+ size_t size, int dma_flags)
{
dma_addr_t dma_handle;
struct pcidev_info *pcidev_info = SN_PCIDEV_INFO(hwdev);

if (hwdev->dev.coherent_dma_mask == ~0UL) {
dma_handle = pcibr_dmatrans_direct64(pcidev_info, phys_addr,
- PCI64_ATTR_BAR);
+ PCI64_ATTR_BAR, dma_flags);
} else {
dma_handle = (dma_addr_t) pcibr_dmamap_ate32(pcidev_info,
phys_addr, size,
- PCI32_ATE_BAR);
+ PCI32_ATE_BAR, dma_flags);
}

return dma_handle;
Index: linux-2.6.14/arch/ia64/sn/pci/tioca_provider.c
===================================================================
--- linux-2.6.14.orig/arch/ia64/sn/pci/tioca_provider.c 2005-12-21 22:35:23.446284291 -0600
+++ linux-2.6.14/arch/ia64/sn/pci/tioca_provider.c 2005-12-21 22:38:24.093685188 -0600
@@ -515,11 +515,17 @@
* use the GART mapped mode.
*/
static uint64_t
-tioca_dma_map(struct pci_dev *pdev, uint64_t paddr, size_t byte_count)
+tioca_dma_map(struct pci_dev *pdev, uint64_t paddr, size_t byte_count, int dma_flags)
{
uint64_t mapaddr;

/*
+ * Not supported for now ...
+ */
+ if (dma_flags & SN_DMA_MSI)
+ return 0;
+
+ /*
* If card is 64 or 48 bit addresable, use a direct mapping. 32
* bit direct is so restrictive w.r.t. where the memory resides that
* we don't use it even though CA has some support.
Index: linux-2.6.14/arch/ia64/sn/pci/tioce_provider.c
===================================================================
--- linux-2.6.14.orig/arch/ia64/sn/pci/tioce_provider.c 2005-12-21 22:35:23.445284532 -0600
+++ linux-2.6.14/arch/ia64/sn/pci/tioce_provider.c 2005-12-21 22:38:24.094684947 -0600
@@ -52,7 +52,8 @@
(ATE_PAGE((start)+(len)-1, pagesize) - ATE_PAGE(start, pagesize) + 1)

#define ATE_VALID(ate) ((ate) & (1UL << 63))
-#define ATE_MAKE(addr, ps) (((addr) & ~ATE_PAGEMASK(ps)) | (1UL << 63))
+#define ATE_MAKE(addr, ps, msi) \
+ (((addr) & ~ATE_PAGEMASK(ps)) | (1UL << 63) | ((msi)?(1UL << 62):0))

/*
* Flavors of ate-based mapping supported by tioce_alloc_map()
@@ -78,15 +79,17 @@
*
* 63 - must be 1 to indicate d64 mode to CE hardware
* 62 - barrier bit ... controlled with tioce_dma_barrier()
- * 61 - 0 since this is not an MSI transaction
+ * 61 - msi bit ... specified through dma_flags
* 60:54 - reserved, MBZ
*/
static uint64_t
-tioce_dma_d64(unsigned long ct_addr)
+tioce_dma_d64(unsigned long ct_addr, int dma_flags)
{
uint64_t bus_addr;

bus_addr = ct_addr | (1UL << 63);
+ if (dma_flags & SN_DMA_MSI)
+ bus_addr |= (1UL << 61);

return bus_addr;
}
@@ -143,7 +146,7 @@
*/
static uint64_t
tioce_alloc_map(struct tioce_kernel *ce_kern, int type, int port,
- uint64_t ct_addr, int len)
+ uint64_t ct_addr, int len, int dma_flags)
{
int i;
int j;
@@ -152,6 +155,7 @@
int entries;
int nates;
int pagesize;
+ int msi_capable, msi_wanted;
uint64_t *ate_shadow;
uint64_t *ate_reg;
uint64_t addr;
@@ -173,6 +177,7 @@
ate_reg = ce_mmr->ce_ure_ate3240;
pagesize = ce_kern->ce_ate3240_pagesize;
bus_base = TIOCE_M32_MIN;
+ msi_capable = 1;
break;
case TIOCE_ATE_M40:
first = 0;
@@ -181,6 +186,7 @@
ate_reg = ce_mmr->ce_ure_ate40;
pagesize = MB(64);
bus_base = TIOCE_M40_MIN;
+ msi_capable = 0;
break;
case TIOCE_ATE_M40S:
/*
@@ -193,11 +199,16 @@
ate_reg = ce_mmr->ce_ure_ate3240;
pagesize = GB(16);
bus_base = TIOCE_M40S_MIN;
+ msi_capable = 0;
break;
default:
return 0;
}

+ msi_wanted = dma_flags & SN_DMA_MSI;
+ if (msi_wanted && !msi_capable)
+ return 0;
+
nates = ATE_NPAGES(ct_addr, len, pagesize);
if (nates > entries)
return 0;
@@ -226,7 +237,7 @@
for (j = 0; j < nates; j++) {
uint64_t ate;

- ate = ATE_MAKE(addr, pagesize);
+ ate = ATE_MAKE(addr, pagesize, msi_wanted);
ate_shadow[i + j] = ate;
writeq(ate, &ate_reg[i + j]);
addr += pagesize;
@@ -253,7 +264,7 @@
* Map @paddr into 32-bit bus space of the CE associated with @pcidev_info.
*/
static uint64_t
-tioce_dma_d32(struct pci_dev *pdev, uint64_t ct_addr)
+tioce_dma_d32(struct pci_dev *pdev, uint64_t ct_addr, int dma_flags)
{
int dma_ok;
int port;
@@ -263,6 +274,9 @@
uint64_t ct_lower;
dma_addr_t bus_addr;

+ if (dma_flags & SN_DMA_MSI)
+ return 0;
+
ct_upper = ct_addr & ~0x3fffffffUL;
ct_lower = ct_addr & 0x3fffffffUL;

@@ -387,7 +401,7 @@
*/
static uint64_t
tioce_do_dma_map(struct pci_dev *pdev, uint64_t paddr, size_t byte_count,
- int barrier)
+ int barrier, int dma_flags)
{
unsigned long flags;
uint64_t ct_addr;
@@ -403,15 +417,18 @@
if (dma_mask < 0x7fffffffUL)
return 0;

- ct_addr = PHYS_TO_TIODMA(paddr);
+ if (SN_DMA_ADDRTYPE(dma_flags) == SN_DMA_ADDR_PHYS)
+ ct_addr = PHYS_TO_TIODMA(paddr);
+ else
+ ct_addr = paddr;

/*
* If the device can generate 64 bit addresses, create a D64 map.
- * Since this should never fail, bypass the rest of the checks.
*/
if (dma_mask == ~0UL) {
- mapaddr = tioce_dma_d64(ct_addr);
- goto dma_map_done;
+ mapaddr = tioce_dma_d64(ct_addr, dma_flags);
+ if (mapaddr)
+ goto dma_map_done;
}

pcidev_to_tioce(pdev, NULL, &ce_kern, &port);
@@ -454,18 +471,22 @@

if (byte_count > MB(64)) {
mapaddr = tioce_alloc_map(ce_kern, TIOCE_ATE_M40S,
- port, ct_addr, byte_count);
+ port, ct_addr, byte_count,
+ dma_flags);
if (!mapaddr)
mapaddr =
tioce_alloc_map(ce_kern, TIOCE_ATE_M40, -1,
- ct_addr, byte_count);
+ ct_addr, byte_count,
+ dma_flags);
} else {
mapaddr = tioce_alloc_map(ce_kern, TIOCE_ATE_M40, -1,
- ct_addr, byte_count);
+ ct_addr, byte_count,
+ dma_flags);
if (!mapaddr)
mapaddr =
tioce_alloc_map(ce_kern, TIOCE_ATE_M40S,
- port, ct_addr, byte_count);
+ port, ct_addr, byte_count,
+ dma_flags);
}
}

@@ -473,7 +494,7 @@
* 32-bit direct is the next mode to try
*/
if (!mapaddr && dma_mask >= 0xffffffffUL)
- mapaddr = tioce_dma_d32(pdev, ct_addr);
+ mapaddr = tioce_dma_d32(pdev, ct_addr, dma_flags);

/*
* Last resort, try 32-bit ATE-based map.
@@ -481,12 +502,12 @@
if (!mapaddr)
mapaddr =
tioce_alloc_map(ce_kern, TIOCE_ATE_M32, -1, ct_addr,
- byte_count);
+ byte_count, dma_flags);

spin_unlock_irqrestore(&ce_kern->ce_lock, flags);

dma_map_done:
- if (mapaddr & barrier)
+ if (mapaddr && barrier)
mapaddr = tioce_dma_barrier(mapaddr, 1);

return mapaddr;
@@ -502,9 +523,9 @@
* in the address.
*/
static uint64_t
-tioce_dma(struct pci_dev *pdev, uint64_t paddr, size_t byte_count)
+tioce_dma(struct pci_dev *pdev, uint64_t paddr, size_t byte_count, int dma_flags)
{
- return tioce_do_dma_map(pdev, paddr, byte_count, 0);
+ return tioce_do_dma_map(pdev, paddr, byte_count, 0, dma_flags);
}

/**
@@ -516,9 +537,9 @@
* Simply call tioce_do_dma_map() to create a map with the barrier bit set
* in the address.
*/ static uint64_t
-tioce_dma_consistent(struct pci_dev *pdev, uint64_t paddr, size_t byte_count)
+tioce_dma_consistent(struct pci_dev *pdev, uint64_t paddr, size_t byte_count, int dma_flags)
{
- return tioce_do_dma_map(pdev, paddr, byte_count, 1);
+ return tioce_do_dma_map(pdev, paddr, byte_count, 1, dma_flags);
}

/**
Index: linux-2.6.14/include/asm-ia64/sn/intr.h
===================================================================
--- linux-2.6.14.orig/include/asm-ia64/sn/intr.h 2005-12-21 22:35:23.446284291 -0600
+++ linux-2.6.14/include/asm-ia64/sn/intr.h 2005-12-21 22:38:24.094684947 -0600
@@ -3,13 +3,14 @@
* License. See the file "COPYING" in the main directory of this archive
* for more details.
*
- * Copyright (C) 1992 - 1997, 2000-2004 Silicon Graphics, Inc. All rights reserved.
+ * Copyright (C) 1992 - 1997, 2000-2005 Silicon Graphics, Inc. All rights reserved.
*/

#ifndef _ASM_IA64_SN_INTR_H
#define _ASM_IA64_SN_INTR_H

#include <linux/rcupdate.h>
+#include <asm/sn/types.h>

#define SGI_UART_VECTOR (0xe9)

@@ -40,6 +41,7 @@
int irq_cpuid; /* kernel logical cpuid */
int irq_irq; /* the IRQ number */
int irq_int_bit; /* Bridge interrupt pin */
+ /* <0 means MSI */
uint64_t irq_xtalkaddr; /* xtalkaddr IRQ is sent to */
int irq_bridge_type;/* pciio asic type (pciio.h) */
void *irq_bridge; /* bridge generating irq */
@@ -53,6 +55,12 @@
};

extern void sn_send_IPI_phys(int, long, int, int);
+extern uint64_t sn_intr_alloc(nasid_t, int,
+ struct sn_irq_info *,
+ int, nasid_t, int);
+extern void sn_intr_free(nasid_t, int, struct sn_irq_info *);
+extern struct sn_irq_info *sn_retarget_vector(struct sn_irq_info *, nasid_t, int);
+extern struct list_head **sn_irq_lh;

#define CPU_VECTOR_TO_IRQ(cpuid,vector) (vector)

Index: linux-2.6.14/include/asm-ia64/sn/pcibr_provider.h
===================================================================
--- linux-2.6.14.orig/include/asm-ia64/sn/pcibr_provider.h 2005-12-21 22:35:23.446284291 -0600
+++ linux-2.6.14/include/asm-ia64/sn/pcibr_provider.h 2005-12-21 22:38:24.094684947 -0600
@@ -3,7 +3,7 @@
* License. See the file "COPYING" in the main directory of this archive
* for more details.
*
- * Copyright (C) 1992-1997,2000-2004 Silicon Graphics, Inc. All rights reserved.
+ * Copyright (C) 1992-1997,2000-2005 Silicon Graphics, Inc. All rights reserved.
*/
#ifndef _ASM_IA64_SN_PCI_PCIBR_PROVIDER_H
#define _ASM_IA64_SN_PCI_PCIBR_PROVIDER_H
@@ -55,6 +55,7 @@
#define PCI32_ATE_V (0x1 << 0)
#define PCI32_ATE_CO (0x1 << 1)
#define PCI32_ATE_PREC (0x1 << 2)
+#define PCI32_ATE_MSI (0x1 << 2)
#define PCI32_ATE_PREF (0x1 << 3)
#define PCI32_ATE_BAR (0x1 << 4)
#define PCI32_ATE_ADDR_SHFT 12
@@ -129,8 +130,8 @@

extern int pcibr_init_provider(void);
extern void *pcibr_bus_fixup(struct pcibus_bussoft *, struct pci_controller *);
-extern dma_addr_t pcibr_dma_map(struct pci_dev *, unsigned long, size_t);
-extern dma_addr_t pcibr_dma_map_consistent(struct pci_dev *, unsigned long, size_t);
+extern dma_addr_t pcibr_dma_map(struct pci_dev *, unsigned long, size_t, int type);
+extern dma_addr_t pcibr_dma_map_consistent(struct pci_dev *, unsigned long, size_t, int type);
extern void pcibr_dma_unmap(struct pci_dev *, dma_addr_t, int);

/*
Index: linux-2.6.14/include/asm-ia64/sn/pcibus_provider_defs.h
===================================================================
--- linux-2.6.14.orig/include/asm-ia64/sn/pcibus_provider_defs.h 2005-12-21 22:35:23.446284291 -0600
+++ linux-2.6.14/include/asm-ia64/sn/pcibus_provider_defs.h 2005-12-21 22:38:24.095684706 -0600
@@ -3,7 +3,7 @@
* License. See the file "COPYING" in the main directory of this archive
* for more details.
*
- * Copyright (C) 1992 - 1997, 2000-2004 Silicon Graphics, Inc. All rights reserved.
+ * Copyright (C) 1992 - 1997, 2000-2005 Silicon Graphics, Inc. All rights reserved.
*/
#ifndef _ASM_IA64_SN_PCI_PCIBUS_PROVIDER_H
#define _ASM_IA64_SN_PCI_PCIBUS_PROVIDER_H
@@ -45,13 +45,24 @@
*/

struct sn_pcibus_provider {
- dma_addr_t (*dma_map)(struct pci_dev *, unsigned long, size_t);
- dma_addr_t (*dma_map_consistent)(struct pci_dev *, unsigned long, size_t);
+ dma_addr_t (*dma_map)(struct pci_dev *, unsigned long, size_t, int flags);
+ dma_addr_t (*dma_map_consistent)(struct pci_dev *, unsigned long, size_t, int flags);
void (*dma_unmap)(struct pci_dev *, dma_addr_t, int);
void * (*bus_fixup)(struct pcibus_bussoft *, struct pci_controller *);
void (*force_interrupt)(struct sn_irq_info *);
void (*target_interrupt)(struct sn_irq_info *);
};

+/*
+ * Flags used by the map interfaces
+ * bits 3:0 specifies format of passed in address
+ * bit 4 specifies that address is to be used for MSI
+ */
+
+#define SN_DMA_ADDRTYPE(x) ((x) & 0xf)
+#define SN_DMA_ADDR_PHYS 1 /* address is an xio address. */
+#define SN_DMA_ADDR_XIO 2 /* address is phys memory */
+#define SN_DMA_MSI 0x10 /* Bus address is to be used for MSI */
+
extern struct sn_pcibus_provider *sn_pci_provider[];
#endif /* _ASM_IA64_SN_PCI_PCIBUS_PROVIDER_H */
Index: linux-2.6.14/include/asm-ia64/sn/tiocp.h
===================================================================
--- linux-2.6.14.orig/include/asm-ia64/sn/tiocp.h 2005-12-21 22:35:23.446284291 -0600
+++ linux-2.6.14/include/asm-ia64/sn/tiocp.h 2005-12-21 22:38:24.095684706 -0600
@@ -3,13 +3,14 @@
* License. See the file "COPYING" in the main directory of this archive
* for more details.
*
- * Copyright (C) 2003-2004 Silicon Graphics, Inc. All rights reserved.
+ * Copyright (C) 2003-2005 Silicon Graphics, Inc. All rights reserved.
*/
#ifndef _ASM_IA64_SN_PCI_TIOCP_H
#define _ASM_IA64_SN_PCI_TIOCP_H

#define TIOCP_HOST_INTR_ADDR 0x003FFFFFFFFFFFFFUL
#define TIOCP_PCI64_CMDTYPE_MEM (0x1ull << 60)
+#define TIOCP_PCI64_CMDTYPE_MSI (0x3ull << 60)


/*****************************************************************************

2005-12-22 20:23:29

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH 0/3] msi abstractions and support for altix

On Thu, Dec 22, 2005 at 02:15:44PM -0600, Mark Maule wrote:
> Resend #2: including linuxppc64-dev and linux-pci as well as PCI maintainer

I'll wait for Resend #3 based on my previous comments before considering
adding it to my kernel trees:)

thanks,

greg k-h

2005-12-22 20:26:37

by Mark Maule

[permalink] [raw]
Subject: Re: [PATCH 0/3] msi abstractions and support for altix

On Thu, Dec 22, 2005 at 12:22:59PM -0800, Greg KH wrote:
> On Thu, Dec 22, 2005 at 02:15:44PM -0600, Mark Maule wrote:
> > Resend #2: including linuxppc64-dev and linux-pci as well as PCI maintainer
>
> I'll wait for Resend #3 based on my previous comments before considering
> adding it to my kernel trees:)
>

Resend #2 includes the correction to the irq_vector[] declaration, and I
responded to the question about setting irq_vector[0] if that's what you
mean ...

Mark

2005-12-22 20:34:40

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH 0/3] msi abstractions and support for altix

On Thu, Dec 22, 2005 at 02:26:27PM -0600, Mark Maule wrote:
> On Thu, Dec 22, 2005 at 12:22:59PM -0800, Greg KH wrote:
> > On Thu, Dec 22, 2005 at 02:15:44PM -0600, Mark Maule wrote:
> > > Resend #2: including linuxppc64-dev and linux-pci as well as PCI maintainer
> >
> > I'll wait for Resend #3 based on my previous comments before considering
> > adding it to my kernel trees:)
> >
>
> Resend #2 includes the correction to the irq_vector[] declaration, and I
> responded to the question about setting irq_vector[0] if that's what you
> mean ...

Sorry, but I missed that last response. Why do you set the [0] value in
a #ifdef now?

thanks,

greg k-h

2005-12-22 20:38:38

by Mark Maule

[permalink] [raw]
Subject: Re: [PATCH 0/3] msi abstractions and support for altix

On Thu, Dec 22, 2005 at 12:34:15PM -0800, Greg KH wrote:
> On Thu, Dec 22, 2005 at 02:26:27PM -0600, Mark Maule wrote:
> > On Thu, Dec 22, 2005 at 12:22:59PM -0800, Greg KH wrote:
> > > On Thu, Dec 22, 2005 at 02:15:44PM -0600, Mark Maule wrote:
> > > > Resend #2: including linuxppc64-dev and linux-pci as well as PCI maintainer
> > >
> > > I'll wait for Resend #3 based on my previous comments before considering
> > > adding it to my kernel trees:)
> > >
> >
> > Resend #2 includes the correction to the irq_vector[] declaration, and I
> > responded to the question about setting irq_vector[0] if that's what you
> > mean ...
>
> Sorry, but I missed that last response. Why do you set the [0] value in
> a #ifdef now?

Because on ia64 IA64_FIRST_DEVICE_VECTOR and IA64_LAST_DEVICE_VECTOR
(from which MSI FIRST_DEVICE_VECTOR/LAST_DEVICE_VECTOR are derived) are not
constants. The are now global variables (see change to asm-ia64/hw_irq.h)
to allow the platform to override them. Altix uses a reduced range of
vectors for devices, and this change was necessary to make assign_irq_vector()
to work on altix.

Mark

2005-12-22 20:50:25

by Matthew Wilcox

[permalink] [raw]
Subject: Re: [PATCH 0/3] msi abstractions and support for altix

On Thu, Dec 22, 2005 at 02:38:24PM -0600, Mark Maule wrote:
> Because on ia64 IA64_FIRST_DEVICE_VECTOR and IA64_LAST_DEVICE_VECTOR
> (from which MSI FIRST_DEVICE_VECTOR/LAST_DEVICE_VECTOR are derived) are not
> constants. The are now global variables (see change to asm-ia64/hw_irq.h)
> to allow the platform to override them. Altix uses a reduced range of
> vectors for devices, and this change was necessary to make assign_irq_vector()
> to work on altix.

To be honest, I think this is just adding a third layer of paper over
the crack in the wall. The original code assumed x86; the ia64 port
added enough emulation to make it look like x86 and now altix fixes a
couple of assumptions. I say: bleh.

What we actually need is an interface provided by the architecture that
allocates a new irq. I have a hankering to implement MSI on PA-RISC but
haven't found the time ...

2005-12-22 21:50:32

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH 0/3] msi abstractions and support for altix

On Thu, Dec 22, 2005 at 02:38:24PM -0600, Mark Maule wrote:
> On Thu, Dec 22, 2005 at 12:34:15PM -0800, Greg KH wrote:
> > On Thu, Dec 22, 2005 at 02:26:27PM -0600, Mark Maule wrote:
> > > On Thu, Dec 22, 2005 at 12:22:59PM -0800, Greg KH wrote:
> > > > On Thu, Dec 22, 2005 at 02:15:44PM -0600, Mark Maule wrote:
> > > > > Resend #2: including linuxppc64-dev and linux-pci as well as PCI maintainer
> > > >
> > > > I'll wait for Resend #3 based on my previous comments before considering
> > > > adding it to my kernel trees:)
> > > >
> > >
> > > Resend #2 includes the correction to the irq_vector[] declaration, and I
> > > responded to the question about setting irq_vector[0] if that's what you
> > > mean ...
> >
> > Sorry, but I missed that last response. Why do you set the [0] value in
> > a #ifdef now?
>
> Because on ia64 IA64_FIRST_DEVICE_VECTOR and IA64_LAST_DEVICE_VECTOR
> (from which MSI FIRST_DEVICE_VECTOR/LAST_DEVICE_VECTOR are derived) are not
> constants. The are now global variables (see change to asm-ia64/hw_irq.h)
> to allow the platform to override them. Altix uses a reduced range of
> vectors for devices, and this change was necessary to make assign_irq_vector()
> to work on altix.

I'm with Matthew on this one, that's not a real fix for this. What
would PPC64 do in this case?

thanks,

greg k-h

2005-12-23 15:32:34

by Mark Maule

[permalink] [raw]
Subject: Re: [PATCH 0/3] msi abstractions and support for altix

On Thu, Dec 22, 2005 at 01:44:46PM -0800, Greg KH wrote:
> On Thu, Dec 22, 2005 at 02:38:24PM -0600, Mark Maule wrote:
> > On Thu, Dec 22, 2005 at 12:34:15PM -0800, Greg KH wrote:
> > > On Thu, Dec 22, 2005 at 02:26:27PM -0600, Mark Maule wrote:
> > > > On Thu, Dec 22, 2005 at 12:22:59PM -0800, Greg KH wrote:
> > > > > On Thu, Dec 22, 2005 at 02:15:44PM -0600, Mark Maule wrote:
> > > > > > Resend #2: including linuxppc64-dev and linux-pci as well as PCI maintainer
> > > > >
> > > > > I'll wait for Resend #3 based on my previous comments before considering
> > > > > adding it to my kernel trees:)
> > > > >
> > > >
> > > > Resend #2 includes the correction to the irq_vector[] declaration, and I
> > > > responded to the question about setting irq_vector[0] if that's what you
> > > > mean ...
> > >
> > > Sorry, but I missed that last response. Why do you set the [0] value in
> > > a #ifdef now?
> >
> > Because on ia64 IA64_FIRST_DEVICE_VECTOR and IA64_LAST_DEVICE_VECTOR
> > (from which MSI FIRST_DEVICE_VECTOR/LAST_DEVICE_VECTOR are derived) are not
> > constants. The are now global variables (see change to asm-ia64/hw_irq.h)
> > to allow the platform to override them. Altix uses a reduced range of
> > vectors for devices, and this change was necessary to make assign_irq_vector()
> > to work on altix.
>
> I'm with Matthew on this one, that's not a real fix for this. What
> would PPC64 do in this case?

Using the existing framework, wouldn't PPC just define it's own
assign_irq_vector and {FIRST,LAST}_DEVICE_VECTOR and handle it however it
wants under the covers?

I agree that this is not a great solution, but it's what the existing framework
allowed. I'm willing to pursue a more general vector allocation scheme, but
I suspect that'll take some time.

Is this issue going to hold up forward progress of this patchset? IMO, this
set is a major step in generalizing the MSI code and I think the vector
generalizing code would best be handled by a separate effort.

Mark

Mark

2005-12-23 16:38:11

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH 0/3] msi abstractions and support for altix

On Fri, Dec 23, 2005 at 09:32:15AM -0600, Mark Maule wrote:
> On Thu, Dec 22, 2005 at 01:44:46PM -0800, Greg KH wrote:
> > On Thu, Dec 22, 2005 at 02:38:24PM -0600, Mark Maule wrote:
> > > On Thu, Dec 22, 2005 at 12:34:15PM -0800, Greg KH wrote:
> > > > On Thu, Dec 22, 2005 at 02:26:27PM -0600, Mark Maule wrote:
> > > > > On Thu, Dec 22, 2005 at 12:22:59PM -0800, Greg KH wrote:
> > > > > > On Thu, Dec 22, 2005 at 02:15:44PM -0600, Mark Maule wrote:
> > > > > > > Resend #2: including linuxppc64-dev and linux-pci as well as PCI maintainer
> > > > > >
> > > > > > I'll wait for Resend #3 based on my previous comments before considering
> > > > > > adding it to my kernel trees:)
> > > > > >
> > > > >
> > > > > Resend #2 includes the correction to the irq_vector[] declaration, and I
> > > > > responded to the question about setting irq_vector[0] if that's what you
> > > > > mean ...
> > > >
> > > > Sorry, but I missed that last response. Why do you set the [0] value in
> > > > a #ifdef now?
> > >
> > > Because on ia64 IA64_FIRST_DEVICE_VECTOR and IA64_LAST_DEVICE_VECTOR
> > > (from which MSI FIRST_DEVICE_VECTOR/LAST_DEVICE_VECTOR are derived) are not
> > > constants. The are now global variables (see change to asm-ia64/hw_irq.h)
> > > to allow the platform to override them. Altix uses a reduced range of
> > > vectors for devices, and this change was necessary to make assign_irq_vector()
> > > to work on altix.
> >
> > I'm with Matthew on this one, that's not a real fix for this. What
> > would PPC64 do in this case?
>
> Using the existing framework, wouldn't PPC just define it's own
> assign_irq_vector and {FIRST,LAST}_DEVICE_VECTOR and handle it however it
> wants under the covers?
>
> I agree that this is not a great solution, but it's what the existing framework
> allowed. I'm willing to pursue a more general vector allocation scheme, but
> I suspect that'll take some time.
>
> Is this issue going to hold up forward progress of this patchset? IMO, this
> set is a major step in generalizing the MSI code and I think the vector
> generalizing code would best be handled by a separate effort.

I don't know, let's see what the ppc64 developers say. If they are
happy with this implementation, then it might be ok...

Ben?

thanks,

greg k-h

2006-01-03 03:23:10

by Mark Maule

[permalink] [raw]
Subject: Re: [PATCH 0/3] msi abstractions and support for altix

On Thu, Dec 22, 2005 at 01:50:23PM -0700, Matthew Wilcox wrote:
> On Thu, Dec 22, 2005 at 02:38:24PM -0600, Mark Maule wrote:
> > Because on ia64 IA64_FIRST_DEVICE_VECTOR and IA64_LAST_DEVICE_VECTOR
> > (from which MSI FIRST_DEVICE_VECTOR/LAST_DEVICE_VECTOR are derived) are not
> > constants. The are now global variables (see change to asm-ia64/hw_irq.h)
> > to allow the platform to override them. Altix uses a reduced range of
> > vectors for devices, and this change was necessary to make assign_irq_vector()
> > to work on altix.
>
> To be honest, I think this is just adding a third layer of paper over
> the crack in the wall. The original code assumed x86; the ia64 port
> added enough emulation to make it look like x86 and now altix fixes a
> couple of assumptions. I say: bleh.
>
> What we actually need is an interface provided by the architecture that
> allocates a new irq. I have a hankering to implement MSI on PA-RISC but
> haven't found the time ...

Matt, Greg, et. al:

Did you guys have something in mind for a vector allocation interface? It
seems to me that assign_irq_vector() more or less does what we want,
but what is missing is a way for the platform to prime which vectors
are available to choose from.

One possibly better solution would be to call something in the init_IRQ path
that would set up the vector pool available to assign_irq_vector().

Any opinions on this? I would maintain that this effort should be done
independently of this patchset.

thanks
Mark

2006-01-03 06:51:29

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH 0/3] msi abstractions and support for altix

On Mon, Jan 02, 2006 at 09:22:49PM -0600, Mark Maule wrote:
> On Thu, Dec 22, 2005 at 01:50:23PM -0700, Matthew Wilcox wrote:
> > On Thu, Dec 22, 2005 at 02:38:24PM -0600, Mark Maule wrote:
> > > Because on ia64 IA64_FIRST_DEVICE_VECTOR and IA64_LAST_DEVICE_VECTOR
> > > (from which MSI FIRST_DEVICE_VECTOR/LAST_DEVICE_VECTOR are derived) are not
> > > constants. The are now global variables (see change to asm-ia64/hw_irq.h)
> > > to allow the platform to override them. Altix uses a reduced range of
> > > vectors for devices, and this change was necessary to make assign_irq_vector()
> > > to work on altix.
> >
> > To be honest, I think this is just adding a third layer of paper over
> > the crack in the wall. The original code assumed x86; the ia64 port
> > added enough emulation to make it look like x86 and now altix fixes a
> > couple of assumptions. I say: bleh.
> >
> > What we actually need is an interface provided by the architecture that
> > allocates a new irq. I have a hankering to implement MSI on PA-RISC but
> > haven't found the time ...
>
> Matt, Greg, et. al:
>
> Did you guys have something in mind for a vector allocation interface? It
> seems to me that assign_irq_vector() more or less does what we want,
> but what is missing is a way for the platform to prime which vectors
> are available to choose from.
>
> One possibly better solution would be to call something in the init_IRQ path
> that would set up the vector pool available to assign_irq_vector().
>
> Any opinions on this? I would maintain that this effort should be done
> independently of this patchset.

Care to write a patch showing how this would work?

And why would this be independant of your other changes?

thanks,

greg k-h

2006-01-03 22:39:34

by Grant Grundler

[permalink] [raw]
Subject: Re: [PATCH 1/3] msi vector targeting abstractions

On Thu, Dec 22, 2005 at 02:15:49PM -0600, Mark Maule wrote:
> Abstract portions of the MSI core for platforms that do not use standard
> APIC interrupt controllers. This is implemented through a new arch-specific
> msi setup routine, and a set of msi ops which can be set on a per platform
> basis.

...
> +
> + msi_ops->target(vector, dest_cpu, &address_hi, &address_lo);
> +
> + pci_write_config_dword(entry->dev, msi_upper_address_reg(pos),
> + address_hi);
> pci_write_config_dword(entry->dev, msi_lower_address_reg(pos),
> - address.lo_address.value);
> + address_lo);
> set_native_irq_info(irq, cpu_mask);
> break;
> }
...
> --- /dev/null 1970-01-01 00:00:00.000000000 +0000
> +++ msi/drivers/pci/msi-apic.c 2005-12-22 11:09:37.022232088 -0600
...
> +struct msi_ops msi_apic_ops = {
> + .setup = msi_setup_apic,
> + .teardown = msi_teardown_apic,
> +#ifdef CONFIG_SMP
> + .target = msi_target_apic,
> +#endif

Mark,
msi_target_apic() initializes address_lo parameter.
Even on a UP machine, we need inialize this value.

If target is called unconditionally, wouldn't it be better
for msi_target_apic() always be called?

It would also be good for msi_target_apic to validate the 'dest_cpu' is online.
Maybe a BUG_ON or something like that.

grant

ps. not done looking through this...and still curious to see where
other discussion about generic vector assignment leads.

2006-01-03 23:50:48

by Mark Maule

[permalink] [raw]
Subject: Re: [PATCH 1/3] msi vector targeting abstractions

On Tue, Jan 03, 2006 at 02:39:18PM -0800, Grant Grundler wrote:
> On Thu, Dec 22, 2005 at 02:15:49PM -0600, Mark Maule wrote:
> > Abstract portions of the MSI core for platforms that do not use standard
> > APIC interrupt controllers. This is implemented through a new arch-specific
> > msi setup routine, and a set of msi ops which can be set on a per platform
> > basis.
>
> ...
> > +
> > + msi_ops->target(vector, dest_cpu, &address_hi, &address_lo);
> > +
> > + pci_write_config_dword(entry->dev, msi_upper_address_reg(pos),
> > + address_hi);
> > pci_write_config_dword(entry->dev, msi_lower_address_reg(pos),
> > - address.lo_address.value);
> > + address_lo);
> > set_native_irq_info(irq, cpu_mask);
> > break;
> > }
> ...
> > --- /dev/null 1970-01-01 00:00:00.000000000 +0000
> > +++ msi/drivers/pci/msi-apic.c 2005-12-22 11:09:37.022232088 -0600
> ...
> > +struct msi_ops msi_apic_ops = {
> > + .setup = msi_setup_apic,
> > + .teardown = msi_teardown_apic,
> > +#ifdef CONFIG_SMP
> > + .target = msi_target_apic,
> > +#endif
>
> Mark,
> msi_target_apic() initializes address_lo parameter.
> Even on a UP machine, we need inialize this value.

Not sure what you mean here. target is used to retarget an existing MSI vector
to a different processor. In the case of apic, this appears to be accomplished
by swizzling the cpu in the low 32 bits of the msi address. Nothing needs to
change in the upper 32 bits.

>
> If target is called unconditionally, wouldn't it be better
> for msi_target_apic() always be called?

target is called through the msi_ops->target vector. SN does not use
msi_target_apic(), it uses sn_msi_target(). Other platforms can implement
target however they need to.

>
> It would also be good for msi_target_apic to validate the 'dest_cpu' is online.
> Maybe a BUG_ON or something like that.

That wasn't a check in the original code flow ... the main protection appears
to be in the upper levels in irq_affinity_write_proc().

>
> grant
>
> ps. not done looking through this...and still curious to see where
> other discussion about generic vector assignment leads.

2006-01-04 00:01:44

by Grant Grundler

[permalink] [raw]
Subject: Re: [PATCH 2/3] per-platform IA64_{FIRST,LAST}_DEVICE_VECTOR definitions

On Thu, Dec 22, 2005 at 02:15:57PM -0600, Mark Maule wrote:
> Abstract IA64_FIRST_DEVICE_VECTOR/IA64_LAST_DEVICE_VECTOR since SN platforms
> use a subset of the IA64 range. Implement this by making the above macros
> global variables which the platform can override in it setup code.
...
> Index: msi/arch/ia64/sn/kernel/irq.c
> ===================================================================
> --- msi.orig/arch/ia64/sn/kernel/irq.c 2005-12-21 22:59:09.199823700 -0600
> +++ msi/arch/ia64/sn/kernel/irq.c 2005-12-22 14:10:01.024578027 -0600
> @@ -203,6 +203,9 @@
> int i;
> irq_desc_t *base_desc = irq_desc;
>
> + ia64_first_device_vector = IA64_SN2_FIRST_DEVICE_VECTOR;
> + ia64_last_device_vector = IA64_SN2_LAST_DEVICE_VECTOR;

Shouldn't this chunk of diff go in "PATCH [2/3] altix: msi support"?
(typo: that should have been "3/3" in the original mail)

thanks,
grant

2006-01-04 00:20:49

by Grant Grundler

[permalink] [raw]
Subject: Re: [PATCH 1/3] msi vector targeting abstractions

On Tue, Jan 03, 2006 at 05:50:24PM -0600, Mark Maule wrote:
> > > +struct msi_ops msi_apic_ops = {
> > > + .setup = msi_setup_apic,
> > > + .teardown = msi_teardown_apic,
> > > +#ifdef CONFIG_SMP
> > > + .target = msi_target_apic,
> > > +#endif
> >
> > Mark,
> > msi_target_apic() initializes address_lo parameter.
> > Even on a UP machine, we need inialize this value.
>
> Not sure what you mean here. target is used to retarget an existing
> MSI vector to a different processor.

Right - I didn't realize the caller, set_msi_affinity(), was surrounded by
"#ifdef CONFIG_SMP".

But set_msi_affinity() appears to be dead code.
I couldn't find any calls to set_msi_affinity() in 2.6.14 or 2.6.15.
Greg, you want a patch to remove that?

thanks,
grant

2006-01-04 00:27:54

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH 1/3] msi vector targeting abstractions

On Tue, Jan 03, 2006 at 04:20:47PM -0800, Grant Grundler wrote:
> On Tue, Jan 03, 2006 at 05:50:24PM -0600, Mark Maule wrote:
> > > > +struct msi_ops msi_apic_ops = {
> > > > + .setup = msi_setup_apic,
> > > > + .teardown = msi_teardown_apic,
> > > > +#ifdef CONFIG_SMP
> > > > + .target = msi_target_apic,
> > > > +#endif
> > >
> > > Mark,
> > > msi_target_apic() initializes address_lo parameter.
> > > Even on a UP machine, we need inialize this value.
> >
> > Not sure what you mean here. target is used to retarget an existing
> > MSI vector to a different processor.
>
> Right - I didn't realize the caller, set_msi_affinity(), was surrounded by
> "#ifdef CONFIG_SMP".
>
> But set_msi_affinity() appears to be dead code.
> I couldn't find any calls to set_msi_affinity() in 2.6.14 or 2.6.15.
> Greg, you want a patch to remove that?

Yes please, that would be great to have.

thanks,

greg k-h

2006-01-04 03:52:39

by Mark Maule

[permalink] [raw]
Subject: Re: [PATCH 1/3] msi vector targeting abstractions

On Tue, Jan 03, 2006 at 04:27:38PM -0800, Greg KH wrote:
> On Tue, Jan 03, 2006 at 04:20:47PM -0800, Grant Grundler wrote:
> > On Tue, Jan 03, 2006 at 05:50:24PM -0600, Mark Maule wrote:
> > > > > +struct msi_ops msi_apic_ops = {
> > > > > + .setup = msi_setup_apic,
> > > > > + .teardown = msi_teardown_apic,
> > > > > +#ifdef CONFIG_SMP
> > > > > + .target = msi_target_apic,
> > > > > +#endif
> > > >
> > > > Mark,
> > > > msi_target_apic() initializes address_lo parameter.
> > > > Even on a UP machine, we need inialize this value.
> > >
> > > Not sure what you mean here. target is used to retarget an existing
> > > MSI vector to a different processor.
> >
> > Right - I didn't realize the caller, set_msi_affinity(), was surrounded by
> > "#ifdef CONFIG_SMP".
> >
> > But set_msi_affinity() appears to be dead code.
> > I couldn't find any calls to set_msi_affinity() in 2.6.14 or 2.6.15.
> > Greg, you want a patch to remove that?
>
> Yes please, that would be great to have.
>
> thanks,
>
> greg k-h

Is that really dead code? From msi.h:

#ifdef CONFIG_SMP
#define set_msi_irq_affinity set_msi_affinity
#else
#define set_msi_irq_affinity NULL
#endif

Mark

2006-01-10 17:00:48

by Mark Maule

[permalink] [raw]
Subject: Re: [PATCH 0/3] msi abstractions and support for altix

On Mon, Jan 02, 2006 at 10:07:19PM -0800, Greg KH wrote:
> On Mon, Jan 02, 2006 at 09:22:49PM -0600, Mark Maule wrote:
> > On Thu, Dec 22, 2005 at 01:50:23PM -0700, Matthew Wilcox wrote:
> > > On Thu, Dec 22, 2005 at 02:38:24PM -0600, Mark Maule wrote:
> > > > Because on ia64 IA64_FIRST_DEVICE_VECTOR and IA64_LAST_DEVICE_VECTOR
> > > > (from which MSI FIRST_DEVICE_VECTOR/LAST_DEVICE_VECTOR are derived) are not
> > > > constants. The are now global variables (see change to asm-ia64/hw_irq.h)
> > > > to allow the platform to override them. Altix uses a reduced range of
> > > > vectors for devices, and this change was necessary to make assign_irq_vector()
> > > > to work on altix.
> > >
> > > To be honest, I think this is just adding a third layer of paper over
> > > the crack in the wall. The original code assumed x86; the ia64 port
> > > added enough emulation to make it look like x86 and now altix fixes a
> > > couple of assumptions. I say: bleh.
> > >
> > > What we actually need is an interface provided by the architecture that
> > > allocates a new irq. I have a hankering to implement MSI on PA-RISC but
> > > haven't found the time ...
> >
> > Matt, Greg, et. al:
> >
> > Did you guys have something in mind for a vector allocation interface? It
> > seems to me that assign_irq_vector() more or less does what we want,
> > but what is missing is a way for the platform to prime which vectors
> > are available to choose from.
> >
> > One possibly better solution would be to call something in the init_IRQ path
> > that would set up the vector pool available to assign_irq_vector().
> >
> > Any opinions on this? I would maintain that this effort should be done
> > independently of this patchset.
>
> Care to write a patch showing how this would work?
>
> And why would this be independant of your other changes?
>
> thanks,
>
> greg k-h

Ok, looks like it's going to be a bit until I have time to work on the
vector allocation stuff.

In the mean time, would folks be recepteive to taking this portion of the
initial patchset:

[PATCH 1/4] msi archetecture init hook
http://lkml.org/lkml/2005/12/21/168

This would at least give us a graceful pci_enable_msi() failure on altix
until I find the time to work on the other stuff.

Mark

2006-01-10 17:03:50

by Christoph Hellwig

[permalink] [raw]
Subject: Re: [PATCH 0/3] msi abstractions and support for altix

> Ok, looks like it's going to be a bit until I have time to work on the
> vector allocation stuff.
>
> In the mean time, would folks be recepteive to taking this portion of the
> initial patchset:
>
> [PATCH 1/4] msi archetecture init hook
> http://lkml.org/lkml/2005/12/21/168
>
> This would at least give us a graceful pci_enable_msi() failure on altix
> until I find the time to work on the other stuff.

Personally I think your patchkit should just go in after all the other
comments have addresses [1]. It's a huge improvement over the mess that's
there currently.

[1] I don't remember if you posted a patchkit addressing everything else yet,
so maybe it's been done already.

2006-01-10 17:12:00

by Greg KH

[permalink] [raw]
Subject: Re: [PATCH 0/3] msi abstractions and support for altix

On Tue, Jan 10, 2006 at 05:03:39PM +0000, Christoph Hellwig wrote:
> > Ok, looks like it's going to be a bit until I have time to work on the
> > vector allocation stuff.
> >
> > In the mean time, would folks be recepteive to taking this portion of the
> > initial patchset:
> >
> > [PATCH 1/4] msi archetecture init hook
> > http://lkml.org/lkml/2005/12/21/168
> >
> > This would at least give us a graceful pci_enable_msi() failure on altix
> > until I find the time to work on the other stuff.
>
> Personally I think your patchkit should just go in after all the other
> comments have addresses [1]. It's a huge improvement over the mess that's
> there currently.

Yes, please repost your patches with all of the comments addressed and
we can look at it from there...

thanks,

greg k-h