2009-09-04 16:58:51

by Stefan Assmann

[permalink] [raw]
Subject: [RFC][PATCH 0/2] boot interrupts on Intel X58 and 55x0

This patchset is meant to disable boot interrupts on Intel X58 and 55x0
chipsets (Tylersburg). A lot of effort from Kei Tokunaga has gone into these
patches. Thanks a lot Kei!

The reason why this consists of 2 patches is that the PCI config space of the
configuration device to disable boot interrupts on these chipsets is not
always accessible by default. The first patch is to ensure that the device is
visible while the second patch applies the necessary changes to stop the
generation of boot interrupts. We're not really sure whether the final X58 and
55x0 chipsets have the configuration device visible or not, so patch #1 might
be superfluous but we've seen at least 2 machines where this is not the case.
That's one of the reasons why this patchset is marked as RFC. The other reason
is more serious namely the onboard NIC (8086:10c9) is malfunctioning on some
of our test system if the second patch is applied. It fails to acquire an IP
from DHCP and we're pretty clueless on this issue right now.
Help is greatly appreciated!


A quick summary of why boot interrupts are better off than on.

Boot interrupts will be generated by the chipset if the interrupt line of a
non-primary IO-APIC is masked and an IRQ arrives there. In that case a boot
interrupt will be forwarded to the PIC _and_ primary IO-APIC. We're not quite
sure why it arrives at the primary IO-APIC as well but it has been observed on
various chipsets.

As there will be no interrupt handler installed (for the boot interrupt) on
the primary IO-APIC the interrupt will be counted as spurious, which can
result in disabling the entire interrupt line by the kernel in case of too
many spurious interrupts. The problem only shows up if the primary IO-APIC
already has an interrupt handler installed on that line, otherwise that line
would be masked anyway and the boot interrupt silently ignored (which makes it
tricky to observe).

When does this become a problem?

Any device connected to a non-primary IO-APIC (that doesn't use MSIs) will
trigger the generation of boot interrupts if it's IO-APIC pin is masked. There
can be many reasons for that for example:
- The interrupt is shared and a buggy device driver (from another device)
causes the interrupt to get disabled by the kernel.
- The RT kernel masks interrupt lines during handling (threaded IRQ-handling).
- Kei reported from issues in the case of kdump when the first kernel disables
the IO-APICs before the second kernel starts booting.

It becomes a problem when too many interrupts are counted as spurious (the
boot interrupts cannot be handled because the kernel doesn't expect them) and
the kernel decides to better bring down the interrupt line.


2009-09-04 16:58:54

by Stefan Assmann

[permalink] [raw]
Subject: [RFC][PATCH 1/2] show Intel QuickPath Interconnect Routing and Protocol Layer Registers in PCI config space

On some machines with Intel X58 or Intel 55x0 chipset the Intel QuickPath
Interconnect Routing and Protocol Layer Registers or not exposed to PCI config
space. Access to these is necessary to disable boot interrupts in the QPI
Protocol Interrupt Control Register.
DEVHIDE1 provides a method to (un)hide the PCI config space of devices inside
the IOH and is altered to guarantee that PCI config space is accessible.
See Intel document #320838-003, sections 17.6.5.10 and 17.11.2.21.

Signed-off-by: Stefan Assmann <[email protected]>
---
drivers/pci/quirks.c | 24 ++++++++++++++++++++++++
include/linux/pci_ids.h | 1 +
2 files changed, 25 insertions(+)

--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -1612,6 +1612,30 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_IN
DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_ESB_10, quirk_disable_intel_boot_interrupt);

/*
+ * Show Intel QuickPath Interconnect Routing and Protocol Layer Registers
+ * in PCI config space (ensures access to QPIPINTRC).
+ * See Intel document #320838-003, sections 17.6.5.10 and 17.11.2.21.
+ */
+#define INTEL_X58_55x0_DEVHIDE1_REG 0xf2
+#define INTEL_X58_55x0_HIDE_DEV17_FUN0 (1<<12)
+#define INTEL_X58_55x0_HIDE_DEV17_FUN1 (1<<13)
+
+static void quirk_show_intel_qpi_conf_register(struct pci_dev *dev)
+{
+ u16 pci_config_word;
+
+ pci_read_config_word(dev, INTEL_X58_55x0_DEVHIDE1_REG, &pci_config_word);
+ pci_config_word &= ~(INTEL_X58_55x0_HIDE_DEV17_FUN0 |
+ INTEL_X58_55x0_HIDE_DEV17_FUN1);
+ pci_write_config_word(dev, INTEL_X58_55x0_DEVHIDE1_REG, pci_config_word);
+
+ /* need to rescan the bus for changes to take effect */
+ pci_rescan_bus(dev->bus);
+}
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_IOAT_TBG9, quirk_show_intel_qpi_conf_register);
+DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_IOAT_TBG9, quirk_show_intel_qpi_conf_register);
+
+/*
* disable boot interrupts on HT-1000
*/
#define BC_HT1000_FEATURE_REG 0x64
--- a/include/linux/pci_ids.h
+++ b/include/linux/pci_ids.h
@@ -2508,6 +2508,7 @@
#define PCI_DEVICE_ID_INTEL_IOAT_TBG5 0x342a
#define PCI_DEVICE_ID_INTEL_IOAT_TBG6 0x342b
#define PCI_DEVICE_ID_INTEL_IOAT_TBG7 0x342c
+#define PCI_DEVICE_ID_INTEL_IOAT_TBG9 0x342e
#define PCI_DEVICE_ID_INTEL_IOAT_TBG0 0x3430
#define PCI_DEVICE_ID_INTEL_IOAT_TBG1 0x3431
#define PCI_DEVICE_ID_INTEL_IOAT_TBG2 0x3432

2009-09-04 16:59:07

by Stefan Assmann

[permalink] [raw]
Subject: [RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

Disable boot interrupts on Intel X58, 55x0 systems by setting the Disable PCI
INTx Routing to ICH bit (default is 0).

Disable PCI INTx Routing to ICH: When this bit is set, local INTx messages
received from the CB DMA/PCI Express ports of the IOH are not routed to legacy
ICH - they are either converted into MSI via the integrated I/OxAPIC (if the
I/OxAPIC mask bit is clear in the appropriate entries) or cause no further action
(when mask bit is set). When this bit is clear, local INTx messages received from
the CB DMA/PCI Express ports of the IOH are routed to legacy ICH, provided the
corresponding mask bit in the IOAPIC is set.
See Intel document #321328-001, section 19.10.2.27.

Signed-off-by: Stefan Assmann <[email protected]>
---

drivers/pci/quirks.c | 24 ++++++++++++++++++++++++
include/linux/pci_ids.h | 1 +
2 files changed, 25 insertions(+)

--- a/drivers/pci/quirks.c
+++ b/drivers/pci/quirks.c
@@ -1635,6 +1635,30 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_IN
DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_IOAT_TBG9, quirk_show_intel_qpi_conf_register);

/*
+ * Disable boot interrupts on Intel X58, 55x0 (Tylersburg).
+ * See Intel document #321328-001, section 19.10.2.27.
+ * (Disable PCI INTx Routing to ICH)
+ */
+#define INTEL_X58_55x0_QPIPINTRC_OFFSET 0xe0
+#define INTEL_X58_55x0_QPIPINTRC_BIT (1<<25)
+static void quirk_disable_intel_tylersburg_boot_interrupt(struct pci_dev *dev)
+{
+ u32 pci_config_dword;
+
+ if (noioapicquirk)
+ return;
+
+ pci_read_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET, &pci_config_dword);
+ pci_config_dword |= INTEL_X58_55x0_QPIPINTRC_BIT;
+ pci_write_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET, pci_config_dword);
+
+ printk(KERN_INFO "disabled boot interrupt on device 0x%04x:0x%04x\n",
+ dev->vendor, dev->device);
+}
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_QPI_TBG15, quirk_disable_intel_tylersburg_boot_interrupt);
+DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_QPI_TBG15, quirk_disable_intel_tylersburg_boot_interrupt);
+
+/*
* disable boot interrupts on HT-1000
*/
#define BC_HT1000_FEATURE_REG 0x64
--- a/include/linux/pci_ids.h
+++ b/include/linux/pci_ids.h
@@ -2504,6 +2504,7 @@
#define PCI_DEVICE_ID_INTEL_ICH9_7 0x2916
#define PCI_DEVICE_ID_INTEL_ICH9_8 0x2918
#define PCI_DEVICE_ID_INTEL_82855PM_HB 0x3340
+#define PCI_DEVICE_ID_INTEL_QPI_TBG15 0x3428
#define PCI_DEVICE_ID_INTEL_IOAT_TBG4 0x3429
#define PCI_DEVICE_ID_INTEL_IOAT_TBG5 0x342a
#define PCI_DEVICE_ID_INTEL_IOAT_TBG6 0x342b

2009-09-04 17:06:55

by Daniel Walker

[permalink] [raw]
Subject: Re: [RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

On Fri, 2009-09-04 at 12:55 -0400, Stefan Assmann wrote:
> + * Disable boot interrupts on Intel X58, 55x0 (Tylersburg).
> + * See Intel document #321328-001, section 19.10.2.27.
> + * (Disable PCI INTx Routing to ICH)
> + */
> +#define INTEL_X58_55x0_QPIPINTRC_OFFSET 0xe0
> +#define INTEL_X58_55x0_QPIPINTRC_BIT (1<<25)
> +static void quirk_disable_intel_tylersburg_boot_interrupt(struct
> pci_dev *dev)
> +{
> + u32 pci_config_dword;
> +
> + if (noioapicquirk)
> + return;
> +
> + pci_read_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET,
> &pci_config_dword);
> + pci_config_dword |= INTEL_X58_55x0_QPIPINTRC_BIT;
> + pci_write_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET,
> pci_config_dword);
> +
> + printk(KERN_INFO "disabled boot interrupt on device 0x%04x:0x%
> 04x\n",
> + dev->vendor, dev->device);
> +}
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_QPI_TBG15, quirk_disable_intel_tylersburg_boot_interrupt);
> +DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_QPI_TBG15, quirk_disable_intel_tylersburg_boot_interrupt);


These lines are wildly long .. Could you reduce these down to a max of
80 characters..

Daniel

2009-09-04 17:15:26

by Yinghai Lu

[permalink] [raw]
Subject: Re: [RFC][PATCH 1/2] show Intel QuickPath Interconnect Routing and Protocol Layer Registers in PCI config space

On Fri, Sep 4, 2009 at 9:55 AM, Stefan Assmann<[email protected]> wrote:
> On some machines with Intel X58 or Intel 55x0 chipset the Intel QuickPath
> Interconnect Routing and Protocol Layer Registers or not exposed to PCI config
> space. Access to these is necessary to disable boot interrupts in the QPI
> Protocol Interrupt Control Register.
> DEVHIDE1 provides a method to (un)hide the PCI config space of devices inside
> the IOH and is altered to guarantee that PCI config space is accessible.
> See Intel document #320838-003, sections 17.6.5.10 and 17.11.2.21.
>
> Signed-off-by: Stefan Assmann <[email protected]>
> ---
> ?drivers/pci/quirks.c ? ?| ? 24 ++++++++++++++++++++++++
> ?include/linux/pci_ids.h | ? ?1 +
> ?2 files changed, 25 insertions(+)
>
> --- a/drivers/pci/quirks.c
> +++ b/drivers/pci/quirks.c
> @@ -1612,6 +1612,30 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_IN
> ?DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL, ? PCI_DEVICE_ID_INTEL_ESB_10, ? ?quirk_disable_intel_boot_interrupt);
>
> ?/*
> + * Show Intel QuickPath Interconnect Routing and Protocol Layer Registers
> + * in PCI config space (ensures access to QPIPINTRC).
> + * See Intel document #320838-003, sections 17.6.5.10 and 17.11.2.21.
> + */
> +#define INTEL_X58_55x0_DEVHIDE1_REG ? ?0xf2
> +#define INTEL_X58_55x0_HIDE_DEV17_FUN0 (1<<12)
> +#define INTEL_X58_55x0_HIDE_DEV17_FUN1 (1<<13)
> +
> +static void quirk_show_intel_qpi_conf_register(struct pci_dev *dev)
> +{
> + ? ? ? u16 pci_config_word;
> +
> + ? ? ? pci_read_config_word(dev, INTEL_X58_55x0_DEVHIDE1_REG, &pci_config_word);
> + ? ? ? pci_config_word &= ~(INTEL_X58_55x0_HIDE_DEV17_FUN0 |
> + ? ? ? ? ? ? ? ? ? ? ? ? ? ?INTEL_X58_55x0_HIDE_DEV17_FUN1);
> + ? ? ? pci_write_config_word(dev, INTEL_X58_55x0_DEVHIDE1_REG, pci_config_word);
> +
> + ? ? ? /* need to rescan the bus for changes to take effect */
> + ? ? ? pci_rescan_bus(dev->bus);

better to check if those bits are set already...

can you move the quirk to EARLY to avoid rescan bus?

YH

> +}
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, ? PCI_DEVICE_ID_INTEL_IOAT_TBG9, ?quirk_show_intel_qpi_conf_register);
> +DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL, ?PCI_DEVICE_ID_INTEL_IOAT_TBG9, ?quirk_show_intel_qpi_conf_register);
> +
> +/*
> ?* disable boot interrupts on HT-1000
> ?*/
> ?#define BC_HT1000_FEATURE_REG ? ? ? ? ?0x64
> --- a/include/linux/pci_ids.h
> +++ b/include/linux/pci_ids.h
> @@ -2508,6 +2508,7 @@
> ?#define PCI_DEVICE_ID_INTEL_IOAT_TBG5 ?0x342a
> ?#define PCI_DEVICE_ID_INTEL_IOAT_TBG6 ?0x342b
> ?#define PCI_DEVICE_ID_INTEL_IOAT_TBG7 ?0x342c
> +#define PCI_DEVICE_ID_INTEL_IOAT_TBG9 ?0x342e
> ?#define PCI_DEVICE_ID_INTEL_IOAT_TBG0 ?0x3430
> ?#define PCI_DEVICE_ID_INTEL_IOAT_TBG1 ?0x3431
> ?#define PCI_DEVICE_ID_INTEL_IOAT_TBG2 ?0x3432
> --
> To unsubscribe from this list: send the line "unsubscribe linux-acpi" in
> the body of a message to [email protected]
> More majordomo info at ?http://vger.kernel.org/majordomo-info.html
>

2009-09-05 09:03:45

by Stefan Assmann

[permalink] [raw]
Subject: Re: [RFC][PATCH 1/2] show Intel QuickPath Interconnect Routing and Protocol Layer Registers in PCI config space

On 04.09.2009 19:07, Yinghai Lu wrote:
> On Fri, Sep 4, 2009 at 9:55 AM, Stefan Assmann<[email protected]> wrote:
>> On some machines with Intel X58 or Intel 55x0 chipset the Intel QuickPath
>> Interconnect Routing and Protocol Layer Registers or not exposed to PCI config
>> space. Access to these is necessary to disable boot interrupts in the QPI
>> Protocol Interrupt Control Register.
>> DEVHIDE1 provides a method to (un)hide the PCI config space of devices inside
>> the IOH and is altered to guarantee that PCI config space is accessible.
>> See Intel document #320838-003, sections 17.6.5.10 and 17.11.2.21.
>>
>> Signed-off-by: Stefan Assmann <[email protected]>
>> ---
>> drivers/pci/quirks.c | 24 ++++++++++++++++++++++++
>> include/linux/pci_ids.h | 1 +
>> 2 files changed, 25 insertions(+)
>>
>> --- a/drivers/pci/quirks.c
>> +++ b/drivers/pci/quirks.c
>> @@ -1612,6 +1612,30 @@ DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_IN
>> DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_ESB_10, quirk_disable_intel_boot_interrupt);
>>
>> /*
>> + * Show Intel QuickPath Interconnect Routing and Protocol Layer Registers
>> + * in PCI config space (ensures access to QPIPINTRC).
>> + * See Intel document #320838-003, sections 17.6.5.10 and 17.11.2.21.
>> + */
>> +#define INTEL_X58_55x0_DEVHIDE1_REG 0xf2
>> +#define INTEL_X58_55x0_HIDE_DEV17_FUN0 (1<<12)
>> +#define INTEL_X58_55x0_HIDE_DEV17_FUN1 (1<<13)
>> +
>> +static void quirk_show_intel_qpi_conf_register(struct pci_dev *dev)
>> +{
>> + u16 pci_config_word;
>> +
>> + pci_read_config_word(dev, INTEL_X58_55x0_DEVHIDE1_REG, &pci_config_word);
>> + pci_config_word &= ~(INTEL_X58_55x0_HIDE_DEV17_FUN0 |
>> + INTEL_X58_55x0_HIDE_DEV17_FUN1);
>> + pci_write_config_word(dev, INTEL_X58_55x0_DEVHIDE1_REG, pci_config_word);
>> +
>> + /* need to rescan the bus for changes to take effect */
>> + pci_rescan_bus(dev->bus);

Hi Yinghai,

> better to check if those bits are set already...

If that's really an issue I'll change it, however it shouldn't cause any
troubles to set an already set bit. I'm not really sure if it's worth
the effort. It has the advantage of possibly avoiding pci_rescan_bus,
you're right about that.

> can you move the quirk to EARLY to avoid rescan bus?

Already tried that and it didn't work.

Stefan
--
Stefan Assmann | Red Hat GmbH
Software Engineer | Otto-Hahn-Strasse 20, 85609 Dornach
| HR: Amtsgericht Muenchen HRB 153243
| GF: Brendan Lane, Charlie Peters,
sassmann at redhat.com | Michael Cunningham, Charles Cachera

2009-09-05 09:09:19

by Stefan Assmann

[permalink] [raw]
Subject: Re: [RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

On 04.09.2009 19:06, Daniel Walker wrote:
> On Fri, 2009-09-04 at 12:55 -0400, Stefan Assmann wrote:
>> + * Disable boot interrupts on Intel X58, 55x0 (Tylersburg).
>> + * See Intel document #321328-001, section 19.10.2.27.
>> + * (Disable PCI INTx Routing to ICH)
>> + */
>> +#define INTEL_X58_55x0_QPIPINTRC_OFFSET 0xe0
>> +#define INTEL_X58_55x0_QPIPINTRC_BIT (1<<25)
>> +static void quirk_disable_intel_tylersburg_boot_interrupt(struct
>> pci_dev *dev)
>> +{
>> + u32 pci_config_dword;
>> +
>> + if (noioapicquirk)
>> + return;
>> +
>> + pci_read_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET,
>> &pci_config_dword);
>> + pci_config_dword |= INTEL_X58_55x0_QPIPINTRC_BIT;
>> + pci_write_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET,
>> pci_config_dword);
>> +
>> + printk(KERN_INFO "disabled boot interrupt on device 0x%04x:0x%
>> 04x\n",
>> + dev->vendor, dev->device);
>> +}
>> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_QPI_TBG15, quirk_disable_intel_tylersburg_boot_interrupt);
>> +DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_QPI_TBG15, quirk_disable_intel_tylersburg_boot_interrupt);
>
>
> These lines are wildly long .. Could you reduce these down to a max of
> 80 characters..

Hi Daniel,

you're right about the lines being to long, however if you take a peek
at drivers/pci/quirks.c you'll see that all the quirks are done this
way. :)

Stefan
--
Stefan Assmann | Red Hat GmbH
Software Engineer | Otto-Hahn-Strasse 20, 85609 Dornach
| HR: Amtsgericht Muenchen HRB 153243
| GF: Brendan Lane, Charlie Peters,
sassmann at redhat.com | Michael Cunningham, Charles Cachera

2009-09-05 14:47:18

by Daniel Walker

[permalink] [raw]
Subject: Re: [RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

On Sat, 2009-09-05 at 11:07 +0200, Stefan Assmann wrote:
> On 04.09.2009 19:06, Daniel Walker wrote:
> > On Fri, 2009-09-04 at 12:55 -0400, Stefan Assmann wrote:
> >> + * Disable boot interrupts on Intel X58, 55x0 (Tylersburg).
> >> + * See Intel document #321328-001, section 19.10.2.27.
> >> + * (Disable PCI INTx Routing to ICH)
> >> + */
> >> +#define INTEL_X58_55x0_QPIPINTRC_OFFSET 0xe0
> >> +#define INTEL_X58_55x0_QPIPINTRC_BIT (1<<25)
> >> +static void quirk_disable_intel_tylersburg_boot_interrupt(struct
> >> pci_dev *dev)
> >> +{
> >> + u32 pci_config_dword;
> >> +
> >> + if (noioapicquirk)
> >> + return;
> >> +
> >> + pci_read_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET,
> >> &pci_config_dword);
> >> + pci_config_dword |= INTEL_X58_55x0_QPIPINTRC_BIT;
> >> + pci_write_config_dword(dev, INTEL_X58_55x0_QPIPINTRC_OFFSET,
> >> pci_config_dword);
> >> +
> >> + printk(KERN_INFO "disabled boot interrupt on device 0x%04x:0x%
> >> 04x\n",
> >> + dev->vendor, dev->device);
> >> +}
> >> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_QPI_TBG15, quirk_disable_intel_tylersburg_boot_interrupt);
> >> +DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_QPI_TBG15, quirk_disable_intel_tylersburg_boot_interrupt);
> >
> >
> > These lines are wildly long .. Could you reduce these down to a max of
> > 80 characters..
>
> Hi Daniel,
>
> you're right about the lines being to long, however if you take a peek
> at drivers/pci/quirks.c you'll see that all the quirks are done this
> way. :)

I looked, the whole thing is a mess .. Really all these lines need to be
cleaned up.. It doesn't help to have you add more to it tho. Especially
the tab's or spaces between the commas, I can say I understand why that
is.. If there is a good reason why it's like that I'm all ears..

Daniel

2009-09-05 16:21:21

by Olaf Dabrunz

[permalink] [raw]
Subject: Re: [RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

On 05-Sep-09, Daniel Walker wrote:
> On Sat, 2009-09-05 at 11:07 +0200, Stefan Assmann wrote:
> > On 04.09.2009 19:06, Daniel Walker wrote:
> > > On Fri, 2009-09-04 at 12:55 -0400, Stefan Assmann wrote:
> > >> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_QPI_TBG15, quirk_disable_intel_tylersburg_boot_interrupt);
> > >> +DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_QPI_TBG15, quirk_disable_intel_tylersburg_boot_interrupt);
> > >
> > >
> > > These lines are wildly long .. Could you reduce these down to a max of
> > > 80 characters..
> >
> > Hi Daniel,
> >
> > you're right about the lines being to long, however if you take a peek
> > at drivers/pci/quirks.c you'll see that all the quirks are done this
> > way. :)
>
> I looked, the whole thing is a mess .. Really all these lines need to be
> cleaned up.. It doesn't help to have you add more to it tho. Especially
> the tab's or spaces between the commas, I can say I understand why that
> is.. If there is a good reason why it's like that I'm all ears..

It makes adding/deleting/modifying the entries easier, with one-line
editor operations. And if all data for one ID-to-quirk relationship is
on one line, with tabs to make for nice columns, I can more easily see
which device is linked to which quirk. That is why I usually prefer one
line per entry. And the entries in quirks.c are long, but not too long
for that IMHO.

I agree that the 80 columns restriction is a good thing in almost all
cases, as it limits the amount of information that is put on a single
line, and it limits the number of indentation levels within a function.

But in this case I think we are better off with one line per entry, even
if the line becomes longer than 80 characters. The information on one
line is not that much (one macro with 3 parameters), only the
identifiers try to be readable and discernible and therefore become
long.

So this will only be a problem for terminals that are limited to 80
rows, and I do not think that anyone is that restricted nowadays.

Regards,

--
Olaf Dabrunz (Olaf.Dabrunz <at> gmx.net)

2009-09-05 17:07:38

by Daniel Walker

[permalink] [raw]
Subject: Re: [RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

On Sat, 2009-09-05 at 18:18 +0200, Olaf Dabrunz wrote:
> On 05-Sep-09, Daniel Walker wrote:
> > On Sat, 2009-09-05 at 11:07 +0200, Stefan Assmann wrote:
> > > On 04.09.2009 19:06, Daniel Walker wrote:
> > > > On Fri, 2009-09-04 at 12:55 -0400, Stefan Assmann wrote:
> > > >> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_QPI_TBG15, quirk_disable_intel_tylersburg_boot_interrupt);
> > > >> +DECLARE_PCI_FIXUP_RESUME(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_QPI_TBG15, quirk_disable_intel_tylersburg_boot_interrupt);
> > > >
> > > >
> > > > These lines are wildly long .. Could you reduce these down to a max of
> > > > 80 characters..
> > >
> > > Hi Daniel,
> > >
> > > you're right about the lines being to long, however if you take a peek
> > > at drivers/pci/quirks.c you'll see that all the quirks are done this
> > > way. :)
> >
> > I looked, the whole thing is a mess .. Really all these lines need to be
> > cleaned up.. It doesn't help to have you add more to it tho. Especially
> > the tab's or spaces between the commas, I can say I understand why that
> > is.. If there is a good reason why it's like that I'm all ears..
>
> It makes adding/deleting/modifying the entries easier, with one-line
> editor operations. And if all data for one ID-to-quirk relationship is
> on one line, with tabs to make for nice columns, I can more easily see
> which device is linked to which quirk. That is why I usually prefer one
> line per entry. And the entries in quirks.c are long, but not too long
> for that IMHO.

Firstly I would say these entries aren't consistent at all.. Some are
comma tab delimited, some are comma space delimited, and some are mixed
space plus tab delimited plus comma delimited.. Many are actually broken
down into less than 80 max length already..

I'm not sure I see how this makes "adding/deleting/modifying" easier ..
We have lots of other situations where lines are broken down less than
80, function arguments, macros, structures all sorts of stuff.

Taking this block for instance,

DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
PCI_DEVICE_ID_TIGON3_5780,
quirk_msi_intx_disable_bug);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
PCI_DEVICE_ID_TIGON3_5780S,
quirk_msi_intx_disable_bug);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
PCI_DEVICE_ID_TIGON3_5714,
quirk_msi_intx_disable_bug);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
PCI_DEVICE_ID_TIGON3_5714S,
quirk_msi_intx_disable_bug);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
PCI_DEVICE_ID_TIGON3_5715,
quirk_msi_intx_disable_bug);
DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
PCI_DEVICE_ID_TIGON3_5715S,
quirk_msi_intx_disable_bug);

In this block there is a fairly clear pattern you can follow. One
problem with using tabs is that in order to get the perceive benefit
your suggesting you would be forced to have a terminal at a certain
size. Without that the single lines just end up line wrapped but in a
less consistent way. You also have the problem that people have to
deliberately add this formatting , sometimes one tab is enough,
sometimes you need two tabs, maybe someone uses spaces instead .. So you
end up with people potentially using all sorts of different methods and
maybe not getting it right (not to mention it's special formatting just
for this file)..

> So this will only be a problem for terminals that are limited to 80
> rows, and I do not think that anyone is that restricted nowadays.

It's more than that.. In order to un-wrap the lines in this file you
have to have a terminal at over 130 characters in width.

I think it would be better to create a script that parses this file and
produces consistent list in the format your describing, just for review
purposes.. That way it takes the human element out of it, and it still
lets you look at the formatting that your looking for.

Daniel

Subject: Re: [RFC][PATCH 0/2] boot interrupts on Intel X58 and 55x0

On Fri, 04 Sep 2009, Stefan Assmann wrote:
> is more serious namely the onboard NIC (8086:10c9) is malfunctioning on some
> of our test system if the second patch is applied. It fails to acquire an IP
> from DHCP and we're pretty clueless on this issue right now.
> Help is greatly appreciated!

Did you read Intel documet 320839-009, August/2009, already? It is at:
http://www.intel.com/Assets/PDF/specupdate/320839.pdf

I see some stuff there that might be a problem, many of the "issues" are
related to interrupt handling. Errata 40 (EOI to IOAPIC can be blocked)
even states that one should avoid the X58 IOH IOAPIC on stepping B2
completely, and use the IOAPIC in the ICH10R instead.

And the B2 stepping is the one in most motherboards (B3 is quite new).

--
"One disk to rule them all, One disk to find them. One disk to bring
them all and in the darkness grind them. In the Land of Redmond
where the shadows lie." -- The Silicon Valley Tarot
Henrique Holschuh

Subject: Re: [RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

On Sat, 05 Sep 2009, Daniel Walker wrote:
> I'm not sure I see how this makes "adding/deleting/modifying" easier ..
> We have lots of other situations where lines are broken down less than
> 80, function arguments, macros, structures all sorts of stuff.
>
> Taking this block for instance,
>
> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> PCI_DEVICE_ID_TIGON3_5780,
> quirk_msi_intx_disable_bug);
> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> PCI_DEVICE_ID_TIGON3_5780S,
> quirk_msi_intx_disable_bug);
> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> PCI_DEVICE_ID_TIGON3_5714,
> quirk_msi_intx_disable_bug);
> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> PCI_DEVICE_ID_TIGON3_5714S,
> quirk_msix_intx_disable_bug);
> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> PCI_DEVICE_ID_TIGON3_5715,
> quirk_msi_intx_disable_bug);
> DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> PCI_DEVICE_ID_TIGON3_5715S,
> quirk_msi_intx_disable_bug);

You must have an alien brain of some sort if reading that is easier than the
tabular (single-line) format for you.

Please try finding the one entry in there which is different (I changed one
of them) with a quick glance. Chances are you'll just skip over it and
think all of them do the same thing.

--
"One disk to rule them all, One disk to find them. One disk to bring
them all and in the darkness grind them. In the Land of Redmond
where the shadows lie." -- The Silicon Valley Tarot
Henrique Holschuh

2009-09-07 01:53:35

by Daniel Walker

[permalink] [raw]
Subject: Re: [RFC][PATCH 2/2] disable boot interrupts on Intel X58 and 55x0

On Sun, 2009-09-06 at 22:37 -0300, Henrique de Moraes Holschuh wrote:
> On Sat, 05 Sep 2009, Daniel Walker wrote:
> > I'm not sure I see how this makes "adding/deleting/modifying" easier ..
> > We have lots of other situations where lines are broken down less than
> > 80, function arguments, macros, structures all sorts of stuff.
> >
> > Taking this block for instance,
> >
> > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> > PCI_DEVICE_ID_TIGON3_5780,
> > quirk_msi_intx_disable_bug);
> > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> > PCI_DEVICE_ID_TIGON3_5780S,
> > quirk_msi_intx_disable_bug);
> > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> > PCI_DEVICE_ID_TIGON3_5714,
> > quirk_msi_intx_disable_bug);
> > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> > PCI_DEVICE_ID_TIGON3_5714S,
> > quirk_msx_intx_disable_bug);
> > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> > PCI_DEVICE_ID_TIGON3_5715,
> > quirk_msi_intx_disable_bug);
> > DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_BROADCOM,
> > PCI_DEVICE_ID_TIGON3_5715S,
> > quirk_msi_intx_disable_bug);
>
> You must have an alien brain of some sort if reading that is easier than the
> tabular (single-line) format for you.

My terminal isn't at 130 ..

> Please try finding the one entry in there which is different (I changed one
> of them) with a quick glance. Chances are you'll just skip over it and
> think all of them do the same thing.

msix , the letters didn't line up, the semicolons don't line up either..
Even with tabs all this information is so similar your bound to overlook
small errors like that. Even if you think all this matter, assume
someone has a term at 80 then every thing is wrapper. That would put you
in the same situation.

Daniel

Daniel

2009-09-07 08:25:44

by Stefan Assmann

[permalink] [raw]
Subject: Re: [RFC][PATCH 0/2] boot interrupts on Intel X58 and 55x0

On 07.09.2009 03:33, Henrique de Moraes Holschuh wrote:
> On Fri, 04 Sep 2009, Stefan Assmann wrote:
>> is more serious namely the onboard NIC (8086:10c9) is malfunctioning on some
>> of our test system if the second patch is applied. It fails to acquire an IP
>> from DHCP and we're pretty clueless on this issue right now.
>> Help is greatly appreciated!
>
> Did you read Intel documet 320839-009, August/2009, already? It is at:
> http://www.intel.com/Assets/PDF/specupdate/320839.pdf
>
> I see some stuff there that might be a problem, many of the "issues" are
> related to interrupt handling. Errata 40 (EOI to IOAPIC can be blocked)
> even states that one should avoid the X58 IOH IOAPIC on stepping B2
> completely, and use the IOAPIC in the ICH10R instead.

Hi Henrique,

good catch, I missed the errata. After reading the details of errata 40
I assume that it's not directly related because the onboard NIC is
connected to the IOH IO-APIC in the first place. No interaction with
another IO-APIC should be involved. The second reason why it's most
likely unrelated is that it only happens when booted with pci=nomsi
(should have mentioned that earlier, sorry) and the errata states:
"The End of Interrupt (EOI) message targeted to the I/OxAPIC will be
blocked in the case that an INTx from a PCIe device and an MSI are
pending inside the Intel X58 Express Chipset."

I've also looked through the other errata and while some of them look
pretty wild I couldn't find anything that might be the root cause for
the observed problem. Same goes for the 55x0 August/2009 spec update.
http://www.intel.com/Assets/PDF/specupdate/321329.pdf

Stefan
--
Stefan Assmann | Red Hat GmbH
Software Engineer | Otto-Hahn-Strasse 20, 85609 Dornach
| HR: Amtsgericht Muenchen HRB 153243
| GF: Brendan Lane, Charlie Peters,
sassmann at redhat.com | Michael Cunningham, Charles Cachera