2018-11-14 14:57:18

by Bharat Kumar Gogada

[permalink] [raw]
Subject: [PATCH] PCI: Enable SERR# forwarding for Type-1 PCI devices

As per Figure 6-3 in PCIe r4.0, sec 6.2.6, ERR_ messages
will be forwarded from the secondary interface to the primary interface,
if the SERR# Enable bit in the Bridge Control register is set.
Currently PCI_BRIDGE_CTL_SERR is being enabled only in
ACPI flow.
This patch enables PCI_BRIDGE_CTL_SERR for Type-1 PCI device.

Signed-off-by: Bharat Kumar Gogada <[email protected]>
---
drivers/pci/probe.c | 20 ++++++++++++++++++--
1 file changed, 18 insertions(+), 2 deletions(-)

diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
index b1c05b5..ed71e8e 100644
--- a/drivers/pci/probe.c
+++ b/drivers/pci/probe.c
@@ -1841,8 +1841,6 @@ static void program_hpp_type0(struct pci_dev *dev, struct hpp_type0 *hpp)
pci_write_config_byte(dev, PCI_SEC_LATENCY_TIMER,
hpp->latency_timer);
pci_read_config_word(dev, PCI_BRIDGE_CONTROL, &pci_bctl);
- if (hpp->enable_serr)
- pci_bctl |= PCI_BRIDGE_CTL_SERR;
if (hpp->enable_perr)
pci_bctl |= PCI_BRIDGE_CTL_PARITY;
pci_write_config_word(dev, PCI_BRIDGE_CONTROL, pci_bctl);
@@ -2114,6 +2112,23 @@ static void pci_configure_eetlp_prefix(struct pci_dev *dev)
#endif
}

+static void pci_configure_serr(struct pci_dev *dev)
+{
+ if (dev->hdr_type == PCI_HEADER_TYPE_BRIDGE) {
+ u16 control;
+
+ /*
+ * A Type-1 PCI bridge will not forward ERR_ messages coming
+ * from an endpoint if SERR# forwarding is not enabled.
+ */
+ pci_read_config_word(dev, PCI_BRIDGE_CONTROL, &control);
+ if (!(control & PCI_BRIDGE_CTL_SERR)) {
+ control |= PCI_BRIDGE_CTL_SERR;
+ pci_write_config_word(dev, PCI_BRIDGE_CONTROL, control);
+ }
+ }
+}
+
static void pci_configure_device(struct pci_dev *dev)
{
struct hotplug_params hpp;
@@ -2124,6 +2139,7 @@ static void pci_configure_device(struct pci_dev *dev)
pci_configure_relaxed_ordering(dev);
pci_configure_ltr(dev);
pci_configure_eetlp_prefix(dev);
+ pci_configure_serr(dev);

memset(&hpp, 0, sizeof(hpp));
ret = pci_get_hp_params(dev, &hpp);
--
2.7.4



2018-12-06 15:48:50

by Bharat Kumar Gogada

[permalink] [raw]
Subject: RE: [PATCH] PCI: Enable SERR# forwarding for Type-1 PCI devices

Hi All,

Please let me know if anyone has any issue with this patch.

Regards,
Bharat
> As per Figure 6-3 in PCIe r4.0, sec 6.2.6, ERR_ messages will be forwarded
> from the secondary interface to the primary interface, if the SERR# Enable
> bit in the Bridge Control register is set.
> Currently PCI_BRIDGE_CTL_SERR is being enabled only in ACPI flow.
> This patch enables PCI_BRIDGE_CTL_SERR for Type-1 PCI device.
>
> Signed-off-by: Bharat Kumar Gogada <[email protected]>
> ---
> drivers/pci/probe.c | 20 ++++++++++++++++++--
> 1 file changed, 18 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c index
> b1c05b5..ed71e8e 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -1841,8 +1841,6 @@ static void program_hpp_type0(struct pci_dev
> *dev, struct hpp_type0 *hpp)
> pci_write_config_byte(dev, PCI_SEC_LATENCY_TIMER,
> hpp->latency_timer);
> pci_read_config_word(dev, PCI_BRIDGE_CONTROL,
> &pci_bctl);
> - if (hpp->enable_serr)
> - pci_bctl |= PCI_BRIDGE_CTL_SERR;
> if (hpp->enable_perr)
> pci_bctl |= PCI_BRIDGE_CTL_PARITY;
> pci_write_config_word(dev, PCI_BRIDGE_CONTROL,
> pci_bctl); @@ -2114,6 +2112,23 @@ static void
> pci_configure_eetlp_prefix(struct pci_dev *dev) #endif }
>
> +static void pci_configure_serr(struct pci_dev *dev) {
> + if (dev->hdr_type == PCI_HEADER_TYPE_BRIDGE) {
> + u16 control;
> +
> + /*
> + * A Type-1 PCI bridge will not forward ERR_ messages
> coming
> + * from an endpoint if SERR# forwarding is not enabled.
> + */
> + pci_read_config_word(dev, PCI_BRIDGE_CONTROL,
> &control);
> + if (!(control & PCI_BRIDGE_CTL_SERR)) {
> + control |= PCI_BRIDGE_CTL_SERR;
> + pci_write_config_word(dev, PCI_BRIDGE_CONTROL,
> control);
> + }
> + }
> +}
> +
> static void pci_configure_device(struct pci_dev *dev) {
> struct hotplug_params hpp;
> @@ -2124,6 +2139,7 @@ static void pci_configure_device(struct pci_dev
> *dev)
> pci_configure_relaxed_ordering(dev);
> pci_configure_ltr(dev);
> pci_configure_eetlp_prefix(dev);
> + pci_configure_serr(dev);
>
> memset(&hpp, 0, sizeof(hpp));
> ret = pci_get_hp_params(dev, &hpp);
> --
> 2.7.4


2019-02-02 00:46:39

by Bjorn Helgaas

[permalink] [raw]
Subject: Re: [PATCH] PCI: Enable SERR# forwarding for Type-1 PCI devices

[+cc others who have commented or been copied on this ancient issue]

On Wed, Nov 14, 2018 at 08:17:01PM +0530, Bharat Kumar Gogada wrote:
> As per Figure 6-3 in PCIe r4.0, sec 6.2.6, ERR_ messages
> will be forwarded from the secondary interface to the primary interface,
> if the SERR# Enable bit in the Bridge Control register is set.
> Currently PCI_BRIDGE_CTL_SERR is being enabled only in
> ACPI flow.
> This patch enables PCI_BRIDGE_CTL_SERR for Type-1 PCI device.

I apologize for being so slow to respond to this.

I applied it to pci/aer for v5.1 with the following changelog:

PCI: Enable SERR# forwarding for all bridges

As per Figure 6-3 in PCIe r4.0, sec 6.2.6, ERR_ messages will be forwarded
from the secondary interface to the primary interface, if the SERR# Enable
bit in the Bridge Control register is set.

It seems clear that an ACPI hotplug parameter method (_HPP or _HPX) that
tells us to "enable SERR in the command register" (ACPI v6.2, sec 6.2.8,
6.2.9.1) refers to PCI_COMMAND_SERR, which enables reporting of errors by
the function itself.

For bridges, we also interpreted that to mean we should enable
PCI_BRIDGE_CTL_SERR, which enables *forwarding* of errors by the bridge.
But we didn't enable PCI_BRIDGE_CTL_SERR anywhere else, which means we
never enabled it for non-ACPI systems or ACPI systems that didn't supply
hotplug parameters.

That means errors reported below bridges were often never forwarded up to a
Root Port where they could be signaled via AER.

Enable PCI_BRIDGE_CTL_SERR for all bridges so we can get better error
reporting for downstream devices.

Signed-off-by: Bharat Kumar Gogada <[email protected]>
[bhelgaas: changelog]
Signed-off-by: Bjorn Helgaas <[email protected]>

> Signed-off-by: Bharat Kumar Gogada <[email protected]>
> ---
> drivers/pci/probe.c | 20 ++++++++++++++++++--
> 1 file changed, 18 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/pci/probe.c b/drivers/pci/probe.c
> index b1c05b5..ed71e8e 100644
> --- a/drivers/pci/probe.c
> +++ b/drivers/pci/probe.c
> @@ -1841,8 +1841,6 @@ static void program_hpp_type0(struct pci_dev *dev, struct hpp_type0 *hpp)
> pci_write_config_byte(dev, PCI_SEC_LATENCY_TIMER,
> hpp->latency_timer);
> pci_read_config_word(dev, PCI_BRIDGE_CONTROL, &pci_bctl);
> - if (hpp->enable_serr)
> - pci_bctl |= PCI_BRIDGE_CTL_SERR;
> if (hpp->enable_perr)
> pci_bctl |= PCI_BRIDGE_CTL_PARITY;
> pci_write_config_word(dev, PCI_BRIDGE_CONTROL, pci_bctl);
> @@ -2114,6 +2112,23 @@ static void pci_configure_eetlp_prefix(struct pci_dev *dev)
> #endif
> }
>
> +static void pci_configure_serr(struct pci_dev *dev)
> +{
> + if (dev->hdr_type == PCI_HEADER_TYPE_BRIDGE) {
> + u16 control;
> +
> + /*
> + * A Type-1 PCI bridge will not forward ERR_ messages coming
> + * from an endpoint if SERR# forwarding is not enabled.
> + */
> + pci_read_config_word(dev, PCI_BRIDGE_CONTROL, &control);
> + if (!(control & PCI_BRIDGE_CTL_SERR)) {
> + control |= PCI_BRIDGE_CTL_SERR;
> + pci_write_config_word(dev, PCI_BRIDGE_CONTROL, control);
> + }
> + }
> +}
> +
> static void pci_configure_device(struct pci_dev *dev)
> {
> struct hotplug_params hpp;
> @@ -2124,6 +2139,7 @@ static void pci_configure_device(struct pci_dev *dev)
> pci_configure_relaxed_ordering(dev);
> pci_configure_ltr(dev);
> pci_configure_eetlp_prefix(dev);
> + pci_configure_serr(dev);
>
> memset(&hpp, 0, sizeof(hpp));
> ret = pci_get_hp_params(dev, &hpp);
> --
> 2.7.4
>