2014-12-05 08:48:37

by Thomas Gleixner

[permalink] [raw]
Subject: [patch 3/5] iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare

The whole iommu setup for irq remapping is a convoluted mess. The
iommu detect function gets called from mem_init() and the prepare
callback gets called from enable_IR_x2apic() for unknown reasons.

Of course AMD and Intel setup differs in nonsensical ways. Intels
prepare callback is explicit while AMDs prepare callback is implicit
in setup_irq_remapping_ops() just to be called in the prepare call
again.

Because all of this gets called from enable_IR_x2apic() and the dmar
prepare function merily parses the ACPI tables, but does not allocate
memory we end up with memory allocation from irq disabled context
later on.

AMDs iommu code at least allocates the required memory from the
prepare function. That has issues as well, but thats not scope of this
patch.

The goal of this change is to distangle the allocation from the actual
enablement. There is no point to allocate memory from irq disabled
regions with GFP_ATOMIC just because it does not matter at that point
in the boot stage. It matters with physical hotplug later on.

There is another issue with the current setup. Due to the conversion
to stacked irqdomains we end up with a call into the irqdomain
allocation code from irq disabled context, but that code does
GFP_KERNEL allocations rightfully as there is no reason to do
preperatory allocations with GFP_ATOMIC.

That change caused the allocator code to complain about GFP_KERNEL
allocations invoked in atomic context. Boris provided a temporary
hackaround which changed the GFP flags if irq_domain_add() got called
from atomic context. Not pretty and we really dont want to get this
into a mainline release for obvious reasons.

Move the ACPI table parsing and the resulting memory allocations from
the enable to the prepare function. That allows to get rid of the
horrible hackaround in irq_domain_add() later.

Signed-off-by: Thomas Gleixner <[email protected]>
---
drivers/iommu/intel_irq_remapping.c | 64 ++++++++++++++++++++++++------------
1 file changed, 44 insertions(+), 20 deletions(-)

Index: tip/drivers/iommu/intel_irq_remapping.c
===================================================================
--- tip.orig/drivers/iommu/intel_irq_remapping.c
+++ tip/drivers/iommu/intel_irq_remapping.c
@@ -539,22 +539,57 @@ static int __init intel_irq_remapping_su
return 1;
}

-static int __init intel_enable_irq_remapping(void)
+static void __init intel_cleanup_irq_remapping(void)
+{
+ struct dmar_drhd_unit *drhd;
+ struct intel_iommu *iommu;
+
+ for_each_iommu(iommu, drhd) {
+ if (ecap_ir_support(iommu->ecap)) {
+ iommu_disable_irq_remapping(iommu);
+ intel_teardown_irq_remapping(iommu);
+ }
+ }
+
+ if (x2apic_supported())
+ pr_warn("Failed to enable irq remapping. You are vulnerable to irq-injection attacks.\n");
+}
+
+static int __init intel_prepare_irq_remapping(void)
{
struct dmar_drhd_unit *drhd;
struct intel_iommu *iommu;
- bool x2apic_present;
- int setup = 0;
- int eim = 0;

- x2apic_present = x2apic_supported();
+ if (dmar_table_init() < 0)
+ return -1;

if (parse_ioapics_under_ir() != 1) {
- printk(KERN_INFO "Not enable interrupt remapping\n");
+ printk(KERN_INFO "Not enabling interrupt remapping\n");
goto error;
}

- if (x2apic_present) {
+ for_each_iommu(iommu, drhd) {
+ if (!ecap_ir_support(iommu->ecap))
+ continue;
+
+ /* Do the allocations early */
+ if (intel_setup_irq_remapping(iommu))
+ goto error;
+ }
+ return 0;
+error:
+ intel_cleanup_irq_remapping();
+ return -1;
+}
+
+static int __init intel_enable_irq_remapping(void)
+{
+ struct dmar_drhd_unit *drhd;
+ struct intel_iommu *iommu;
+ int setup = 0;
+ int eim = 0;
+
+ if (x2apic_supported()) {
pr_info("Queued invalidation will be enabled to support x2apic and Intr-remapping.\n");

eim = !dmar_x2apic_optout();
@@ -622,9 +657,6 @@ static int __init intel_enable_irq_remap
if (!ecap_ir_support(iommu->ecap))
continue;

- if (intel_setup_irq_remapping(iommu))
- goto error;
-
iommu_set_irq_remapping(iommu, eim);
setup = 1;
}
@@ -639,15 +671,7 @@ static int __init intel_enable_irq_remap
return eim ? IRQ_REMAP_X2APIC_MODE : IRQ_REMAP_XAPIC_MODE;

error:
- for_each_iommu(iommu, drhd)
- if (ecap_ir_support(iommu->ecap)) {
- iommu_disable_irq_remapping(iommu);
- intel_teardown_irq_remapping(iommu);
- }
-
- if (x2apic_present)
- pr_warn("Failed to enable irq remapping. You are vulnerable to irq-injection attacks.\n");
-
+ intel_cleanup_irq_remapping();
return -1;
}

@@ -947,7 +971,7 @@ static struct irq_domain *intel_get_irq_

struct irq_remap_ops intel_irq_remap_ops = {
.supported = intel_irq_remapping_supported,
- .prepare = dmar_table_init,
+ .prepare = intel_prepare_irq_remapping,
.enable = intel_enable_irq_remapping,
.disable = disable_irq_remapping,
.reenable = reenable_irq_remapping,


Subject: [tip:x86/apic] iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare

Commit-ID: e9220e591375af6d02604c261999df570fba744f
Gitweb: http://git.kernel.org/tip/e9220e591375af6d02604c261999df570fba744f
Author: Thomas Gleixner <[email protected]>
AuthorDate: Fri, 5 Dec 2014 08:48:32 +0000
Committer: Thomas Gleixner <[email protected]>
CommitDate: Sat, 6 Dec 2014 00:19:25 +0100

iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare

The whole iommu setup for irq remapping is a convoluted mess. The
iommu detect function gets called from mem_init() and the prepare
callback gets called from enable_IR_x2apic() for unknown reasons.

Of course AMD and Intel setup differs in nonsensical ways. Intels
prepare callback is explicit while AMDs prepare callback is implicit
in setup_irq_remapping_ops() just to be called in the prepare call
again.

Because all of this gets called from enable_IR_x2apic() and the dmar
prepare function merily parses the ACPI tables, but does not allocate
memory we end up with memory allocation from irq disabled context
later on.

AMDs iommu code at least allocates the required memory from the
prepare function. That has issues as well, but thats not scope of this
patch.

The goal of this change is to distangle the allocation from the actual
enablement. There is no point to allocate memory from irq disabled
regions with GFP_ATOMIC just because it does not matter at that point
in the boot stage. It matters with physical hotplug later on.

There is another issue with the current setup. Due to the conversion
to stacked irqdomains we end up with a call into the irqdomain
allocation code from irq disabled context, but that code does
GFP_KERNEL allocations rightfully as there is no reason to do
preperatory allocations with GFP_ATOMIC.

That change caused the allocator code to complain about GFP_KERNEL
allocations invoked in atomic context. Boris provided a temporary
hackaround which changed the GFP flags if irq_domain_add() got called
from atomic context. Not pretty and we really dont want to get this
into a mainline release for obvious reasons.

Move the ACPI table parsing and the resulting memory allocations from
the enable to the prepare function. That allows to get rid of the
horrible hackaround in irq_domain_add() later.

Signed-off-by: Thomas Gleixner <[email protected]>
Tested-by: Borislav Petkov <[email protected]>
Acked-by: Joerg Roedel <[email protected]>
Cc: Jiang Liu <[email protected]>
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Thomas Gleixner <[email protected]>
---
drivers/iommu/intel_irq_remapping.c | 64 +++++++++++++++++++++++++------------
1 file changed, 44 insertions(+), 20 deletions(-)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index f6da3b2..a17e411 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -539,22 +539,57 @@ static int __init intel_irq_remapping_supported(void)
return 1;
}

-static int __init intel_enable_irq_remapping(void)
+static void __init intel_cleanup_irq_remapping(void)
+{
+ struct dmar_drhd_unit *drhd;
+ struct intel_iommu *iommu;
+
+ for_each_iommu(iommu, drhd) {
+ if (ecap_ir_support(iommu->ecap)) {
+ iommu_disable_irq_remapping(iommu);
+ intel_teardown_irq_remapping(iommu);
+ }
+ }
+
+ if (x2apic_supported())
+ pr_warn("Failed to enable irq remapping. You are vulnerable to irq-injection attacks.\n");
+}
+
+static int __init intel_prepare_irq_remapping(void)
{
struct dmar_drhd_unit *drhd;
struct intel_iommu *iommu;
- bool x2apic_present;
- int setup = 0;
- int eim = 0;

- x2apic_present = x2apic_supported();
+ if (dmar_table_init() < 0)
+ return -1;

if (parse_ioapics_under_ir() != 1) {
- printk(KERN_INFO "Not enable interrupt remapping\n");
+ printk(KERN_INFO "Not enabling interrupt remapping\n");
goto error;
}

- if (x2apic_present) {
+ for_each_iommu(iommu, drhd) {
+ if (!ecap_ir_support(iommu->ecap))
+ continue;
+
+ /* Do the allocations early */
+ if (intel_setup_irq_remapping(iommu))
+ goto error;
+ }
+ return 0;
+error:
+ intel_cleanup_irq_remapping();
+ return -1;
+}
+
+static int __init intel_enable_irq_remapping(void)
+{
+ struct dmar_drhd_unit *drhd;
+ struct intel_iommu *iommu;
+ int setup = 0;
+ int eim = 0;
+
+ if (x2apic_supported()) {
pr_info("Queued invalidation will be enabled to support x2apic and Intr-remapping.\n");

eim = !dmar_x2apic_optout();
@@ -622,9 +657,6 @@ static int __init intel_enable_irq_remapping(void)
if (!ecap_ir_support(iommu->ecap))
continue;

- if (intel_setup_irq_remapping(iommu))
- goto error;
-
iommu_set_irq_remapping(iommu, eim);
setup = 1;
}
@@ -639,15 +671,7 @@ static int __init intel_enable_irq_remapping(void)
return eim ? IRQ_REMAP_X2APIC_MODE : IRQ_REMAP_XAPIC_MODE;

error:
- for_each_iommu(iommu, drhd)
- if (ecap_ir_support(iommu->ecap)) {
- iommu_disable_irq_remapping(iommu);
- intel_teardown_irq_remapping(iommu);
- }
-
- if (x2apic_present)
- pr_warn("Failed to enable irq remapping. You are vulnerable to irq-injection attacks.\n");
-
+ intel_cleanup_irq_remapping();
return -1;
}

@@ -947,7 +971,7 @@ static struct irq_domain *intel_get_irq_domain(struct irq_alloc_info *info)

struct irq_remap_ops intel_irq_remap_ops = {
.supported = intel_irq_remapping_supported,
- .prepare = dmar_table_init,
+ .prepare = intel_prepare_irq_remapping,
.enable = intel_enable_irq_remapping,
.disable = disable_irq_remapping,
.reenable = reenable_irq_remapping,

2014-12-11 07:35:52

by Yinghai Lu

[permalink] [raw]
Subject: Re: [tip:x86/apic] iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare

On Fri, Dec 5, 2014 at 3:26 PM, tip-bot for Thomas Gleixner
<[email protected]> wrote:
> Commit-ID: e9220e591375af6d02604c261999df570fba744f
> Gitweb: http://git.kernel.org/tip/e9220e591375af6d02604c261999df570fba744f
> Author: Thomas Gleixner <[email protected]>
> AuthorDate: Fri, 5 Dec 2014 08:48:32 +0000
> Committer: Thomas Gleixner <[email protected]>
> CommitDate: Sat, 6 Dec 2014 00:19:25 +0100
>
> iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare
>
> The whole iommu setup for irq remapping is a convoluted mess. The
> iommu detect function gets called from mem_init() and the prepare
> callback gets called from enable_IR_x2apic() for unknown reasons.

Got

[ 134.510969] calling ahci_pci_driver_init+0x0/0x1b @ 1
[ 134.511387] ahci 0000:00:1f.2: version 3.0
[ 134.530941] alloc irq_desc for 91 on node 0
[ 134.531168] alloc irq_desc for 92 on node 0
[ 134.550728] alloc irq_desc for 93 on node 0
[ 134.550995] alloc irq_desc for 94 on node 0
[ 134.551199] alloc irq_desc for 95 on node 0
[ 134.570871] alloc irq_desc for 96 on node 0
[ 134.571090] alloc irq_desc for 97 on node 0
[ 134.571303] alloc irq_desc for 98 on node 0
[ 134.590974] alloc irq_desc for 99 on node 0
[ 134.591205] alloc irq_desc for 100 on node 0
[ 134.610882] alloc irq_desc for 101 on node 0
[ 134.611136] alloc irq_desc for 102 on node 0
[ 134.611364] alloc irq_desc for 103 on node 0
[ 134.630992] alloc irq_desc for 104 on node 0
[ 134.631232] alloc irq_desc for 105 on node 0
[ 134.650885] alloc irq_desc for 106 on node 0
[ 134.651246] ahci 0000:00:1f.2: SSS flag set, parallel bus scan disabled
[ 134.670926] ahci 0000:00:1f.2: AHCI 0001.0200 32 slots 6 ports 3
Gbps 0x3f impl SATA mode
[ 134.671349] ahci 0000:00:1f.2: flags: 64bit ncq sntf stag pm led
clo pio slum part ccc ems sxs
[ 134.691158] ahci 0000:00:1f.2: with iommu 3 : domain 10
[ 134.751560] BUG: unable to handle kernel NULL pointer dereference
at 0000000000000118
[ 134.751997] IP: [<ffffffff81eafe50>] modify_irte+0x40/0xd0
[ 134.770893] PGD 0
[ 134.771011] Oops: 0000 [#1] SMP
[ 134.771195] Modules linked in:
[ 134.771344] CPU: 0 PID: 2169 Comm: kworker/0:1 Tainted: G W
[ 134.811557] Workqueue: events work_for_cpu_fn
[ 134.830823] task: ffff881024725240 ti: ffff8810252f8000 task.ti:
ffff8810252f8000
[ 134.831176] RIP: 0010:[<ffffffff81eafe50>] [<ffffffff81eafe50>]
modify_irte+0x40/0xd0
[ 134.851029] RSP: 0000:ffff8810252fba18 EFLAGS: 00010096
[ 134.851276] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000be00bd
[ 134.871322] RDX: 0000000000000000 RSI: ffffffff81eafe3f RDI: 0000000000000046
[ 134.891061] RBP: ffff8810252fba48 R08: 0000000000000001 R09: 0000000000000001
[ 134.891393] R10: ffff881024725240 R11: 0000000000000292 R12: 0000000000000000
[ 134.911249] R13: 0000000000000096 R14: ffff881022b181d0 R15: ffff880079268260
[ 134.930824] FS: 0000000000000000(0000) GS:ffff88103de00000(0000)
knlGS:0000000000000000
[ 134.931202] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 134.950966] CR2: 0000000000000118 CR3: 0000000002c1a000 CR4: 00000000000007f0
[ 134.970775] Stack:
[ 134.970883] ffff8810252fba88 0000000000000046 ffff881022f6c660
ffff881026d00000
[ 134.971253] 000000000000005c ffff880079268200 ffff8810252fba58
ffffffff81eaff26
[ 134.991066] ffff8810252fba78 ffffffff81106761 ffff880079268200
ffff88103d889400
[ 135.010908] Call Trace:
[ 135.011038] [<ffffffff81eaff26>] intel_irq_remapping_activate+0x16/0x20
[ 135.030800] [<ffffffff81106761>] irq_domain_activate_irq+0x41/0x50
[ 135.031103] [<ffffffff8110674b>] irq_domain_activate_irq+0x2b/0x50
[ 135.050857] [<ffffffff81103f19>] irq_startup+0x29/0x70
[ 135.051091] [<ffffffff81102857>] __setup_irq+0x327/0x590
[ 135.070849] [<ffffffff81a3d820>] ? ahci_bad_pmp_check_ready+0x70/0x70
[ 135.071143] [<ffffffff81102c42>] request_threaded_irq+0xf2/0x150
[ 135.090972] [<ffffffff81a3d820>] ? ahci_bad_pmp_check_ready+0x70/0x70
[ 135.091295] [<ffffffff81a3ff90>] ? ahci_host_activate+0x180/0x180
[ 135.111014] [<ffffffff8110495f>] devm_request_threaded_irq+0x5f/0xb0
[ 135.130804] [<ffffffff81a3feb3>] ahci_host_activate+0xa3/0x180
[ 135.131097] [<ffffffff81a3d391>] ahci_init_one+0x9d1/0xac0
[ 135.150841] [<ffffffff8157d735>] local_pci_probe+0x45/0xa0
[ 135.151127] [<ffffffff810b8868>] work_for_cpu_fn+0x18/0x30
[ 135.170843] [<ffffffff810bbd24>] process_one_work+0x254/0x470
[ 135.171103] [<ffffffff810bbc89>] ? process_one_work+0x1b9/0x470
[ 135.190846] [<ffffffff810bce1b>] worker_thread+0x31b/0x4e0
[ 135.191115] [<ffffffff810ea3bd>] ? trace_hardirqs_on+0xd/0x10
[ 135.210920] [<ffffffff810bcb00>] ? pool_mayday_timeout+0x170/0x170
[ 135.211215] [<ffffffff810c1ff1>] kthread+0x101/0x110
[ 135.230902] [<ffffffff810ea3bd>] ? trace_hardirqs_on+0xd/0x10
[ 135.231157] [<ffffffff810c1ef0>] ? kthread_stop+0x100/0x100
[ 135.250930] [<ffffffff82015e6c>] ret_from_fork+0x7c/0xb0
[ 135.251178] [<ffffffff810c1ef0>] ? kthread_stop+0x100/0x100
[ 135.270969] Code: ec 10 48 85 ff 0f 84 90 00 00 00 48 c7 c7 80 34
e0 82 49 89 f6 e8 21 54 16 00 0f b7 53 08 49 89 c5 0f b7 43 0a 4c 8b
23 8d 1c 02 <49> 8b 84 24 18 01 00 00 48 63 fb 48 c1 e7 04 48 03 38 49
8b 06
[ 135.291699] RIP [<ffffffff81eafe50>] modify_irte+0x40/0xd0
[ 135.311051] RSP <ffff8810252fba18>
[ 135.311215] CR2: 0000000000000118
[ 135.330856] ---[ end trace fee039719f1667df ]---
[ 135.333024] BUG: unable to handle kernel paging request at ffffffffffffff98
[ 135.350911] IP: [<ffffffff810c2530>] kthread_data+0x10/0x20
[ 135.351230] PGD 2c1b067 PUD 2c1d067 PMD 0
[ 135.351443] Oops: 0000 [#2] SMP
[ 135.370998] Modules linked in:
[ 135.371168] CPU: 0 PID: 2169 Comm: kworker/0:1 Tainted: G D W
[ 135.412423] task: ffff881024725240 ti: ffff8810252f8000 task.ti:
ffff8810252f8000
[ 135.412798] RIP: 0010:[<ffffffff810c2530>] [<ffffffff810c2530>]
kthread_data+0x10/0x20
[ 135.431159] RSP: 0000:ffff8810252fb538 EFLAGS: 00010096
[ 135.450891] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000000000f
[ 135.451218] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff881024725240
[ 135.471044] RBP: ffff8810252fb538 R08: ffff8810247252d0 R09: 0000000000000001
[ 135.490873] R10: ffff881024725240 R11: 000000000000001a R12: ffff88103dfd2c40
[ 135.491237] R13: 0000000000000000 R14: 0000000000000000 R15: ffff881024725240
[ 135.511046] FS: 0000000000000000(0000) GS:ffff88103de00000(0000)
knlGS:0000000000000000
[ 135.530882] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[ 135.531160] CR2: 0000000000000028 CR3: 0000000002c1a000 CR4: 00000000000007f0
[ 135.550979] Stack:
[ 135.551074] ffff8810252fb558 ffffffff810bd065 ffff8810252fb558
ffff881024725240
[ 135.570927] ffff8810252fb678 ffffffff8200fc0b ffff881025ffec00
0000000000009000
[ 135.571302] ffff881024725240 ffff8810252fbfd8 ffff88103dfd3a40
ffff881024725240
[ 135.591114] Call Trace:
[ 135.591231] [<ffffffff810bd065>] wq_worker_sleeping+0x15/0xb0
[ 135.610996] [<ffffffff8200fc0b>] __schedule+0x18b/0xa70
[ 135.611237] [<ffffffff810ea3bd>] ? trace_hardirqs_on+0xd/0x10
[ 135.630988] [<ffffffff810a634a>] ? do_exit+0x88a/0x9f0
[ 135.631222] [<ffffffff810a634a>] ? do_exit+0x88a/0x9f0
[ 135.650932] [<ffffffff82010555>] schedule+0x65/0x70
[ 135.651186] [<ffffffff810a6415>] do_exit+0x955/0x9f0
[ 135.670899] [<ffffffff81054a08>] oops_end+0xb8/0xd0
[ 135.671136] [<ffffffff81ffa88a>] no_context+0x309/0x352
[ 135.671373] [<ffffffff81ffaa98>] __bad_area_nosemaphore+0x1c5/0x1e4
[ 135.691185] [<ffffffff81ffaaca>] bad_area_nosemaphore+0x13/0x15
[ 135.710934] [<ffffffff81093f26>] __do_page_fault+0x266/0x590
[ 135.711292] [<ffffffff810c8d20>] ? task_rq_lock+0x50/0xb0
[ 135.730941] [<ffffffff810c8d20>] ? task_rq_lock+0x50/0xb0
[ 135.731200] [<ffffffff820150f2>] ? _raw_spin_lock+0x62/0x70
[ 135.750949] [<ffffffff810c8d20>] ? task_rq_lock+0x50/0xb0
[ 135.751195] [<ffffffff810ea166>] ? trace_hardirqs_on_caller+0x16/0x260
[ 135.770989] [<ffffffff810e7c6f>] ? trace_hardirqs_off_caller+0x1f/0x160
[ 135.771309] [<ffffffff81094296>] do_page_fault+0x46/0x80
[ 135.791081] [<ffffffff82017c72>] page_fault+0x22/0x30
[ 135.791310] [<ffffffff81eafe3f>] ? modify_irte+0x2f/0xd0
[ 135.811037] [<ffffffff81eafe50>] ? modify_irte+0x40/0xd0
[ 135.811315] [<ffffffff81eafe3f>] ? modify_irte+0x2f/0xd0
[ 135.831150] [<ffffffff81eaff26>] intel_irq_remapping_activate+0x16/0x20
[ 135.831461] [<ffffffff81106761>] irq_domain_activate_irq+0x41/0x50
[ 135.851716] [<ffffffff8110674b>] irq_domain_activate_irq+0x2b/0x50
[ 135.852020] [<ffffffff81103f19>] irq_startup+0x29/0x70
[ 135.871401] [<ffffffff81102857>] __setup_irq+0x327/0x590
[ 135.871653] [<ffffffff81a3d820>] ? ahci_bad_pmp_check_ready+0x70/0x70
[ 135.891334] [<ffffffff81102c42>] request_threaded_irq+0xf2/0x150
[ 135.911099] [<ffffffff81a3d820>] ? ahci_bad_pmp_check_ready+0x70/0x70
[ 135.911416] [<ffffffff81a3ff90>] ? ahci_host_activate+0x180/0x180
[ 135.931274] [<ffffffff8110495f>] devm_request_threaded_irq+0x5f/0xb0
[ 135.931568] [<ffffffff81a3feb3>] ahci_host_activate+0xa3/0x180
[ 135.951093] [<ffffffff81a3d391>] ahci_init_one+0x9d1/0xac0
[ 135.951375] [<ffffffff8157d735>] local_pci_probe+0x45/0xa0
[ 135.971111] [<ffffffff810b8868>] work_for_cpu_fn+0x18/0x30
[ 135.971366] [<ffffffff810bbd24>] process_one_work+0x254/0x470
[ 135.991196] [<ffffffff810bbc89>] ? process_one_work+0x1b9/0x470
[ 135.991477] [<ffffffff810bce1b>] worker_thread+0x31b/0x4e0
[ 136.011132] [<ffffffff810ea3bd>] ? trace_hardirqs_on+0xd/0x10
[ 136.011393] [<ffffffff810bcb00>] ? pool_mayday_timeout+0x170/0x170
[ 136.031187] [<ffffffff810c1ff1>] kthread+0x101/0x110
[ 136.031420] [<ffffffff810ea3bd>] ? trace_hardirqs_on+0xd/0x10
[ 136.051244] [<ffffffff810c1ef0>] ? kthread_stop+0x100/0x100
[ 136.051494] [<ffffffff82015e6c>] ret_from_fork+0x7c/0xb0
[ 136.071210] [<ffffffff810c1ef0>] ? kthread_stop+0x100/0x100
[ 136.071495] Code: 00 48 89 e5 5d 48 8b 40 88 48 c1 e8 02 83 e0 01
c3 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 8b 87 b8 08 00 00
55 48 89 e5 <48> 8b 40 98 5d c3 66 2e 0f 1f 84 00 00 00 00 00 66 66 66
66 90
[ 136.111619] RIP [<ffffffff810c2530>] kthread_data+0x10/0x20
[ 136.131069] RSP <ffff8810252fb538>
[ 136.131253] CR2: ffffffffffffff98
[ 136.131406] ---[ end trace fee039719f1667e0 ]---
[ 136.151131] Fixing recursive fault but reboot is needed!

It is in tip/apic

Thanks

Yinghai

2014-12-11 14:33:46

by Jiang Liu

[permalink] [raw]
Subject: Re: [tip:x86/apic] iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare

On 2014/12/11 15:35, Yinghai Lu wrote:
> On Fri, Dec 5, 2014 at 3:26 PM, tip-bot for Thomas Gleixner
> <[email protected]> wrote:
>> Commit-ID: e9220e591375af6d02604c261999df570fba744f
>> Gitweb: http://git.kernel.org/tip/e9220e591375af6d02604c261999df570fba744f
>> Author: Thomas Gleixner <[email protected]>
>> AuthorDate: Fri, 5 Dec 2014 08:48:32 +0000
>> Committer: Thomas Gleixner <[email protected]>
>> CommitDate: Sat, 6 Dec 2014 00:19:25 +0100
>>
>> iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare
>>
>> The whole iommu setup for irq remapping is a convoluted mess. The
>> iommu detect function gets called from mem_init() and the prepare
>> callback gets called from enable_IR_x2apic() for unknown reasons.
>
> Got
>
Hi Yinghai,
From following log messages, it seems that the AHCI controllers
allocates 16 MSI/MSI-X interrupt, and triggers NULL pointer reference
when enabling interrupts for AHCI.
It doesn't trigger panic with this code path (allocate/enable
MSI/MSI-X interrupts with IR enabled) on my test system. So could you
please help to get more info with the attached test patch?
Thanks!
Gerry

> [ 134.510969] calling ahci_pci_driver_init+0x0/0x1b @ 1
> [ 134.511387] ahci 0000:00:1f.2: version 3.0
> [ 134.530941] alloc irq_desc for 91 on node 0
> [ 134.531168] alloc irq_desc for 92 on node 0
> [ 134.550728] alloc irq_desc for 93 on node 0
> [ 134.550995] alloc irq_desc for 94 on node 0
> [ 134.551199] alloc irq_desc for 95 on node 0
> [ 134.570871] alloc irq_desc for 96 on node 0
> [ 134.571090] alloc irq_desc for 97 on node 0
> [ 134.571303] alloc irq_desc for 98 on node 0
> [ 134.590974] alloc irq_desc for 99 on node 0
> [ 134.591205] alloc irq_desc for 100 on node 0
> [ 134.610882] alloc irq_desc for 101 on node 0
> [ 134.611136] alloc irq_desc for 102 on node 0
> [ 134.611364] alloc irq_desc for 103 on node 0
> [ 134.630992] alloc irq_desc for 104 on node 0
> [ 134.631232] alloc irq_desc for 105 on node 0
> [ 134.650885] alloc irq_desc for 106 on node 0
> [ 134.651246] ahci 0000:00:1f.2: SSS flag set, parallel bus scan disabled
> [ 134.670926] ahci 0000:00:1f.2: AHCI 0001.0200 32 slots 6 ports 3
> Gbps 0x3f impl SATA mode
> [ 134.671349] ahci 0000:00:1f.2: flags: 64bit ncq sntf stag pm led
> clo pio slum part ccc ems sxs
> [ 134.691158] ahci 0000:00:1f.2: with iommu 3 : domain 10
> [ 134.751560] BUG: unable to handle kernel NULL pointer dereference
> at 0000000000000118
> [ 134.751997] IP: [<ffffffff81eafe50>] modify_irte+0x40/0xd0
> [ 134.770893] PGD 0
> [ 134.771011] Oops: 0000 [#1] SMP
> [ 134.771195] Modules linked in:
> [ 134.771344] CPU: 0 PID: 2169 Comm: kworker/0:1 Tainted: G W
> [ 134.811557] Workqueue: events work_for_cpu_fn
> [ 134.830823] task: ffff881024725240 ti: ffff8810252f8000 task.ti:
> ffff8810252f8000
> [ 134.831176] RIP: 0010:[<ffffffff81eafe50>] [<ffffffff81eafe50>]
> modify_irte+0x40/0xd0
> [ 134.851029] RSP: 0000:ffff8810252fba18 EFLAGS: 00010096
> [ 134.851276] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 0000000000be00bd
> [ 134.871322] RDX: 0000000000000000 RSI: ffffffff81eafe3f RDI: 0000000000000046
> [ 134.891061] RBP: ffff8810252fba48 R08: 0000000000000001 R09: 0000000000000001
> [ 134.891393] R10: ffff881024725240 R11: 0000000000000292 R12: 0000000000000000
> [ 134.911249] R13: 0000000000000096 R14: ffff881022b181d0 R15: ffff880079268260
> [ 134.930824] FS: 0000000000000000(0000) GS:ffff88103de00000(0000)
> knlGS:0000000000000000
> [ 134.931202] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 134.950966] CR2: 0000000000000118 CR3: 0000000002c1a000 CR4: 00000000000007f0
> [ 134.970775] Stack:
> [ 134.970883] ffff8810252fba88 0000000000000046 ffff881022f6c660
> ffff881026d00000
> [ 134.971253] 000000000000005c ffff880079268200 ffff8810252fba58
> ffffffff81eaff26
> [ 134.991066] ffff8810252fba78 ffffffff81106761 ffff880079268200
> ffff88103d889400
> [ 135.010908] Call Trace:
> [ 135.011038] [<ffffffff81eaff26>] intel_irq_remapping_activate+0x16/0x20
> [ 135.030800] [<ffffffff81106761>] irq_domain_activate_irq+0x41/0x50
> [ 135.031103] [<ffffffff8110674b>] irq_domain_activate_irq+0x2b/0x50
> [ 135.050857] [<ffffffff81103f19>] irq_startup+0x29/0x70
> [ 135.051091] [<ffffffff81102857>] __setup_irq+0x327/0x590
> [ 135.070849] [<ffffffff81a3d820>] ? ahci_bad_pmp_check_ready+0x70/0x70
> [ 135.071143] [<ffffffff81102c42>] request_threaded_irq+0xf2/0x150
> [ 135.090972] [<ffffffff81a3d820>] ? ahci_bad_pmp_check_ready+0x70/0x70
> [ 135.091295] [<ffffffff81a3ff90>] ? ahci_host_activate+0x180/0x180
> [ 135.111014] [<ffffffff8110495f>] devm_request_threaded_irq+0x5f/0xb0
> [ 135.130804] [<ffffffff81a3feb3>] ahci_host_activate+0xa3/0x180
> [ 135.131097] [<ffffffff81a3d391>] ahci_init_one+0x9d1/0xac0
> [ 135.150841] [<ffffffff8157d735>] local_pci_probe+0x45/0xa0
> [ 135.151127] [<ffffffff810b8868>] work_for_cpu_fn+0x18/0x30
> [ 135.170843] [<ffffffff810bbd24>] process_one_work+0x254/0x470
> [ 135.171103] [<ffffffff810bbc89>] ? process_one_work+0x1b9/0x470
> [ 135.190846] [<ffffffff810bce1b>] worker_thread+0x31b/0x4e0
> [ 135.191115] [<ffffffff810ea3bd>] ? trace_hardirqs_on+0xd/0x10
> [ 135.210920] [<ffffffff810bcb00>] ? pool_mayday_timeout+0x170/0x170
> [ 135.211215] [<ffffffff810c1ff1>] kthread+0x101/0x110
> [ 135.230902] [<ffffffff810ea3bd>] ? trace_hardirqs_on+0xd/0x10
> [ 135.231157] [<ffffffff810c1ef0>] ? kthread_stop+0x100/0x100
> [ 135.250930] [<ffffffff82015e6c>] ret_from_fork+0x7c/0xb0
> [ 135.251178] [<ffffffff810c1ef0>] ? kthread_stop+0x100/0x100
> [ 135.270969] Code: ec 10 48 85 ff 0f 84 90 00 00 00 48 c7 c7 80 34
> e0 82 49 89 f6 e8 21 54 16 00 0f b7 53 08 49 89 c5 0f b7 43 0a 4c 8b
> 23 8d 1c 02 <49> 8b 84 24 18 01 00 00 48 63 fb 48 c1 e7 04 48 03 38 49
> 8b 06
> [ 135.291699] RIP [<ffffffff81eafe50>] modify_irte+0x40/0xd0
> [ 135.311051] RSP <ffff8810252fba18>
> [ 135.311215] CR2: 0000000000000118
> [ 135.330856] ---[ end trace fee039719f1667df ]---
> [ 135.333024] BUG: unable to handle kernel paging request at ffffffffffffff98
> [ 135.350911] IP: [<ffffffff810c2530>] kthread_data+0x10/0x20
> [ 135.351230] PGD 2c1b067 PUD 2c1d067 PMD 0
> [ 135.351443] Oops: 0000 [#2] SMP
> [ 135.370998] Modules linked in:
> [ 135.371168] CPU: 0 PID: 2169 Comm: kworker/0:1 Tainted: G D W
> [ 135.412423] task: ffff881024725240 ti: ffff8810252f8000 task.ti:
> ffff8810252f8000
> [ 135.412798] RIP: 0010:[<ffffffff810c2530>] [<ffffffff810c2530>]
> kthread_data+0x10/0x20
> [ 135.431159] RSP: 0000:ffff8810252fb538 EFLAGS: 00010096
> [ 135.450891] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000000000f
> [ 135.451218] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff881024725240
> [ 135.471044] RBP: ffff8810252fb538 R08: ffff8810247252d0 R09: 0000000000000001
> [ 135.490873] R10: ffff881024725240 R11: 000000000000001a R12: ffff88103dfd2c40
> [ 135.491237] R13: 0000000000000000 R14: 0000000000000000 R15: ffff881024725240
> [ 135.511046] FS: 0000000000000000(0000) GS:ffff88103de00000(0000)
> knlGS:0000000000000000
> [ 135.530882] CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> [ 135.531160] CR2: 0000000000000028 CR3: 0000000002c1a000 CR4: 00000000000007f0
> [ 135.550979] Stack:
> [ 135.551074] ffff8810252fb558 ffffffff810bd065 ffff8810252fb558
> ffff881024725240
> [ 135.570927] ffff8810252fb678 ffffffff8200fc0b ffff881025ffec00
> 0000000000009000
> [ 135.571302] ffff881024725240 ffff8810252fbfd8 ffff88103dfd3a40
> ffff881024725240
> [ 135.591114] Call Trace:
> [ 135.591231] [<ffffffff810bd065>] wq_worker_sleeping+0x15/0xb0
> [ 135.610996] [<ffffffff8200fc0b>] __schedule+0x18b/0xa70
> [ 135.611237] [<ffffffff810ea3bd>] ? trace_hardirqs_on+0xd/0x10
> [ 135.630988] [<ffffffff810a634a>] ? do_exit+0x88a/0x9f0
> [ 135.631222] [<ffffffff810a634a>] ? do_exit+0x88a/0x9f0
> [ 135.650932] [<ffffffff82010555>] schedule+0x65/0x70
> [ 135.651186] [<ffffffff810a6415>] do_exit+0x955/0x9f0
> [ 135.670899] [<ffffffff81054a08>] oops_end+0xb8/0xd0
> [ 135.671136] [<ffffffff81ffa88a>] no_context+0x309/0x352
> [ 135.671373] [<ffffffff81ffaa98>] __bad_area_nosemaphore+0x1c5/0x1e4
> [ 135.691185] [<ffffffff81ffaaca>] bad_area_nosemaphore+0x13/0x15
> [ 135.710934] [<ffffffff81093f26>] __do_page_fault+0x266/0x590
> [ 135.711292] [<ffffffff810c8d20>] ? task_rq_lock+0x50/0xb0
> [ 135.730941] [<ffffffff810c8d20>] ? task_rq_lock+0x50/0xb0
> [ 135.731200] [<ffffffff820150f2>] ? _raw_spin_lock+0x62/0x70
> [ 135.750949] [<ffffffff810c8d20>] ? task_rq_lock+0x50/0xb0
> [ 135.751195] [<ffffffff810ea166>] ? trace_hardirqs_on_caller+0x16/0x260
> [ 135.770989] [<ffffffff810e7c6f>] ? trace_hardirqs_off_caller+0x1f/0x160
> [ 135.771309] [<ffffffff81094296>] do_page_fault+0x46/0x80
> [ 135.791081] [<ffffffff82017c72>] page_fault+0x22/0x30
> [ 135.791310] [<ffffffff81eafe3f>] ? modify_irte+0x2f/0xd0
> [ 135.811037] [<ffffffff81eafe50>] ? modify_irte+0x40/0xd0
> [ 135.811315] [<ffffffff81eafe3f>] ? modify_irte+0x2f/0xd0
> [ 135.831150] [<ffffffff81eaff26>] intel_irq_remapping_activate+0x16/0x20
> [ 135.831461] [<ffffffff81106761>] irq_domain_activate_irq+0x41/0x50
> [ 135.851716] [<ffffffff8110674b>] irq_domain_activate_irq+0x2b/0x50
> [ 135.852020] [<ffffffff81103f19>] irq_startup+0x29/0x70
> [ 135.871401] [<ffffffff81102857>] __setup_irq+0x327/0x590
> [ 135.871653] [<ffffffff81a3d820>] ? ahci_bad_pmp_check_ready+0x70/0x70
> [ 135.891334] [<ffffffff81102c42>] request_threaded_irq+0xf2/0x150
> [ 135.911099] [<ffffffff81a3d820>] ? ahci_bad_pmp_check_ready+0x70/0x70
> [ 135.911416] [<ffffffff81a3ff90>] ? ahci_host_activate+0x180/0x180
> [ 135.931274] [<ffffffff8110495f>] devm_request_threaded_irq+0x5f/0xb0
> [ 135.931568] [<ffffffff81a3feb3>] ahci_host_activate+0xa3/0x180
> [ 135.951093] [<ffffffff81a3d391>] ahci_init_one+0x9d1/0xac0
> [ 135.951375] [<ffffffff8157d735>] local_pci_probe+0x45/0xa0
> [ 135.971111] [<ffffffff810b8868>] work_for_cpu_fn+0x18/0x30
> [ 135.971366] [<ffffffff810bbd24>] process_one_work+0x254/0x470
> [ 135.991196] [<ffffffff810bbc89>] ? process_one_work+0x1b9/0x470
> [ 135.991477] [<ffffffff810bce1b>] worker_thread+0x31b/0x4e0
> [ 136.011132] [<ffffffff810ea3bd>] ? trace_hardirqs_on+0xd/0x10
> [ 136.011393] [<ffffffff810bcb00>] ? pool_mayday_timeout+0x170/0x170
> [ 136.031187] [<ffffffff810c1ff1>] kthread+0x101/0x110
> [ 136.031420] [<ffffffff810ea3bd>] ? trace_hardirqs_on+0xd/0x10
> [ 136.051244] [<ffffffff810c1ef0>] ? kthread_stop+0x100/0x100
> [ 136.051494] [<ffffffff82015e6c>] ret_from_fork+0x7c/0xb0
> [ 136.071210] [<ffffffff810c1ef0>] ? kthread_stop+0x100/0x100
> [ 136.071495] Code: 00 48 89 e5 5d 48 8b 40 88 48 c1 e8 02 83 e0 01
> c3 66 2e 0f 1f 84 00 00 00 00 00 66 66 66 66 90 48 8b 87 b8 08 00 00
> 55 48 89 e5 <48> 8b 40 98 5d c3 66 2e 0f 1f 84 00 00 00 00 00 66 66 66
> 66 90
> [ 136.111619] RIP [<ffffffff810c2530>] kthread_data+0x10/0x20
> [ 136.131069] RSP <ffff8810252fb538>
> [ 136.131253] CR2: ffffffffffffff98
> [ 136.131406] ---[ end trace fee039719f1667e0 ]---
> [ 136.151131] Fixing recursive fault but reboot is needed!
>
> It is in tip/apic
>
> Thanks
>
> Yinghai
>


Attachments:
0001-.patch (1.23 kB)

2014-12-11 17:57:07

by Yinghai Lu

[permalink] [raw]
Subject: Re: [tip:x86/apic] iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare

On Thu, Dec 11, 2014 at 6:33 AM, Jiang Liu <[email protected]> wrote:
> On 2014/12/11 15:35, Yinghai Lu wrote:
>> On Fri, Dec 5, 2014 at 3:26 PM, tip-bot for Thomas Gleixner
>> <[email protected]> wrote:
>>> Commit-ID: e9220e591375af6d02604c261999df570fba744f
>>> Gitweb: http://git.kernel.org/tip/e9220e591375af6d02604c261999df570fba744f
>>> Author: Thomas Gleixner <[email protected]>
>>> AuthorDate: Fri, 5 Dec 2014 08:48:32 +0000
>>> Committer: Thomas Gleixner <[email protected]>
>>> CommitDate: Sat, 6 Dec 2014 00:19:25 +0100
>>>
>>> iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare
>>>
>>> The whole iommu setup for irq remapping is a convoluted mess. The
>>> iommu detect function gets called from mem_init() and the prepare
>>> callback gets called from enable_IR_x2apic() for unknown reasons.
>>
>> Got
>>
> Hi Yinghai,
> From following log messages, it seems that the AHCI controllers
> allocates 16 MSI/MSI-X interrupt, and triggers NULL pointer reference
> when enabling interrupts for AHCI.
> It doesn't trigger panic with this code path (allocate/enable
> MSI/MSI-X interrupts with IR enabled) on my test system. So could you
> please help to get more info with the attached test patch?

[ 113.486917] calling ahci_pci_driver_init+0x0/0x1b @ 1
[ 113.487299] ahci 0000:00:1f.2: version 3.0
[ 113.507317] ahci 0000:00:1f.2: SSS flag set, parallel bus scan disabled
[ 113.507713] ahci 0000:00:1f.2: AHCI 0001.0200 32 slots 6 ports 3
Gbps 0x3f impl SATA mode
[ 113.527019] ahci 0000:00:1f.2: flags: 64bit ncq sntf stag pm led
clo pio slum part ccc ems sxs
[ 113.583977] iommu: chip_data ffff881022b97740, iommu
ffff88103d80ae00, index 32, subindex 0, ir_table ffff88103d802af0,
table_base ffff881026c00000, queue ffff88102770c000
[ 113.597261] iommu: chip_data ffff881022b97780, iommu
(null), index 0, subindex 0, ir_table (null), table_base
(null), queue (null)
[ 113.617124] BUG: unable to handle kernel NULL pointer dereference
at 0000000000000118
[ 113.636983] IP: [<ffffffff81eacca0>] modify_irte+0x40/0xd0
[ 113.637253] PGD 0
[ 113.656978] Oops: 0000 [#1] SMP
[ 113.657133] Modules linked in:
[ 113.657322] CPU: 0 PID: 2531 Comm: kworker/0:1 Tainted: G W
[ 113.696953] Workqueue: events work_for_cpu_fn
[ 113.697189] task: ffff881025535240 ti: ffff881025528000 task.ti:
ffff881025528000
[ 113.716965] RIP: 0010:[<ffffffff81eacca0>] [<ffffffff81eacca0>]
modify_irte+0x40/0xd0

2014-12-11 20:30:37

by Thomas Gleixner

[permalink] [raw]
Subject: Re: [tip:x86/apic] iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare

On Thu, 11 Dec 2014, Yinghai Lu wrote:
> On Thu, Dec 11, 2014 at 6:33 AM, Jiang Liu <[email protected]> wrote:
> > On 2014/12/11 15:35, Yinghai Lu wrote:
> >> On Fri, Dec 5, 2014 at 3:26 PM, tip-bot for Thomas Gleixner
> >> <[email protected]> wrote:
> >>> Commit-ID: e9220e591375af6d02604c261999df570fba744f
> >>> Gitweb: http://git.kernel.org/tip/e9220e591375af6d02604c261999df570fba744f
> >>> Author: Thomas Gleixner <[email protected]>
> >>> AuthorDate: Fri, 5 Dec 2014 08:48:32 +0000
> >>> Committer: Thomas Gleixner <[email protected]>
> >>> CommitDate: Sat, 6 Dec 2014 00:19:25 +0100
> >>>
> >>> iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare
> >>>
> >>> The whole iommu setup for irq remapping is a convoluted mess. The
> >>> iommu detect function gets called from mem_init() and the prepare
> >>> callback gets called from enable_IR_x2apic() for unknown reasons.
> >>
> >> Got
> >>
> > Hi Yinghai,
> > From following log messages, it seems that the AHCI controllers
> > allocates 16 MSI/MSI-X interrupt, and triggers NULL pointer reference
> > when enabling interrupts for AHCI.
> > It doesn't trigger panic with this code path (allocate/enable
> > MSI/MSI-X interrupts with IR enabled) on my test system. So could you
> > please help to get more info with the attached test patch?
>
> [ 113.486917] calling ahci_pci_driver_init+0x0/0x1b @ 1
> [ 113.487299] ahci 0000:00:1f.2: version 3.0
> [ 113.507317] ahci 0000:00:1f.2: SSS flag set, parallel bus scan disabled
> [ 113.507713] ahci 0000:00:1f.2: AHCI 0001.0200 32 slots 6 ports 3
> Gbps 0x3f impl SATA mode
> [ 113.527019] ahci 0000:00:1f.2: flags: 64bit ncq sntf stag pm led
> clo pio slum part ccc ems sxs
> [ 113.583977] iommu: chip_data ffff881022b97740, iommu
> ffff88103d80ae00, index 32, subindex 0, ir_table ffff88103d802af0,
> table_base ffff881026c00000, queue ffff88102770c000
> [ 113.597261] iommu: chip_data ffff881022b97780, iommu
> (null), index 0, subindex 0, ir_table (null), table_base
> (null), queue (null)

So irq_2_iommu is empty. That's a multi MSI, and that's the second
interrupt which gets enabled.

The patch below should fix it.

Thanks,

tglx

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index ff35b0336d2b..46da573a4746 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -1131,7 +1131,7 @@ static int intel_irq_remapping_alloc(struct irq_domain *domain,
{
struct intel_iommu *iommu = domain->host_data;
struct irq_alloc_info *info = arg;
- struct intel_ir_data *data;
+ struct intel_ir_data *data, *ird;
struct irq_data *irq_data;
struct irq_cfg *irq_cfg;
int i, ret, index;
@@ -1176,14 +1176,20 @@ static int intel_irq_remapping_alloc(struct irq_domain *domain,
}

if (i > 0) {
- data = kzalloc(sizeof(*data), GFP_KERNEL);
- if (!data)
+ ird = kzalloc(sizeof(*data), GFP_KERNEL);
+ if (!ird)
goto out_free_data;
+ /* Initialize the common data */
+ ird->irq_2_iommu = data->irq_2_iommu;
+ ird->irq_2_iommu.sub_handle = i;
+ } else {
+ ird = data;
}
+
irq_data->hwirq = (index << 16) + i;
- irq_data->chip_data = data;
+ irq_data->chip_data = ird;
irq_data->chip = &intel_ir_chip;
- intel_irq_remapping_prepare_irte(data, irq_cfg, info, index, i);
+ intel_irq_remapping_prepare_irte(ird, irq_cfg, info, index, i);
irq_set_status_flags(virq + i, IRQ_MOVE_PCNTXT);
}
return 0;

2014-12-12 02:04:33

by Yinghai Lu

[permalink] [raw]
Subject: Re: [tip:x86/apic] iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare

On Thu, Dec 11, 2014 at 12:30 PM, Thomas Gleixner <[email protected]> wrote:

> So irq_2_iommu is empty. That's a multi MSI, and that's the second
> interrupt which gets enabled.
>
> The patch below should fix it.
>

Yes, that fixes the problem.

Assume you will fold it into

commit 289472f461d922507f75dd2451770282adb3a99b
Author: Jiang Liu <[email protected]>
Date: Tue Nov 25 13:53:19 2014 +0800

iommu/vt-d: Enhance Intel IR driver to suppport hierarchy irqdomain

Thanks

Yinghai

2015-04-27 22:46:10

by Yinghai Lu

[permalink] [raw]
Subject: Re: [tip:x86/apic] iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare

On Thu, Dec 11, 2014 at 6:04 PM, Yinghai Lu <[email protected]> wrote:
> On Thu, Dec 11, 2014 at 12:30 PM, Thomas Gleixner <[email protected]> wrote:
>
>> So irq_2_iommu is empty. That's a multi MSI, and that's the second
>> interrupt which gets enabled.
>>
>> The patch below should fix it.
>>
>
> Yes, that fixes the problem.
>
> Assume you will fold it into
>
> commit 289472f461d922507f75dd2451770282adb3a99b
> Author: Jiang Liu <[email protected]>
> Date: Tue Nov 25 13:53:19 2014 +0800
>
> iommu/vt-d: Enhance Intel IR driver to suppport hierarchy irqdomain
>

Looks like you did not put your fix in new tip/x86/apic with Jiang's patchset.

Yinghai

2015-04-29 08:15:55

by Jiang Liu

[permalink] [raw]
Subject: Re: [tip:x86/apic] iommu/vt-d: Move iommu preparatory allocations to irq_remap_ops.prepare

On 2015/4/28 6:46, Yinghai Lu wrote:
> On Thu, Dec 11, 2014 at 6:04 PM, Yinghai Lu <[email protected]> wrote:
>> On Thu, Dec 11, 2014 at 12:30 PM, Thomas Gleixner <[email protected]> wrote:
>>
>>> So irq_2_iommu is empty. That's a multi MSI, and that's the second
>>> interrupt which gets enabled.
>>>
>>> The patch below should fix it.
>>>
>>
>> Yes, that fixes the problem.
>>
>> Assume you will fold it into
>>
>> commit 289472f461d922507f75dd2451770282adb3a99b
>> Author: Jiang Liu <[email protected]>
>> Date: Tue Nov 25 13:53:19 2014 +0800
>>
>> iommu/vt-d: Enhance Intel IR driver to suppport hierarchy irqdomain
>>
>
> Looks like you did not put your fix in new tip/x86/apic with Jiang's patchset.
Hi Yinghai,
Sorry, this patch got lost when I was reorganizing the patch
set. I will send out a formal patch for it soon.
Thanks!
Gerry

>
> Yinghai
>