2020-09-22 21:47:38

by Sean V Kelley

Subject: [PATCH v6 05/10] PCI/AER: Apply function level reset to RCiEP on fatal error

From: Qiuxu Zhuo <[email protected]>

Attempt to do function level reset for an RCiEP associated with an
RCEC device on fatal error.

Signed-off-by: Qiuxu Zhuo <[email protected]>
---
drivers/pci/pcie/err.c | 31 ++++++++++++++++++++++---------
1 file changed, 22 insertions(+), 9 deletions(-)

diff --git a/drivers/pci/pcie/err.c b/drivers/pci/pcie/err.c
index e575fa6cee63..5380ecc41506 100644
--- a/drivers/pci/pcie/err.c
+++ b/drivers/pci/pcie/err.c
@@ -169,6 +169,17 @@ static void pci_bridge_walk(struct pci_dev *bridge, int (*cb)(struct pci_dev *,
cb(bridge, userdata);
}

+static pci_ers_result_t flr_on_rciep(struct pci_dev *dev)
+{
+ if (!pcie_has_flr(dev))
+ return PCI_ERS_RESULT_NONE;
+
+ if (pcie_flr(dev))
+ return PCI_ERS_RESULT_DISCONNECT;
+
+ return PCI_ERS_RESULT_RECOVERED;
+}
+
pci_ers_result_t pcie_do_recovery(struct pci_dev *dev,
pci_channel_state_t state,
pci_ers_result_t (*reset_subordinate_devices)(struct pci_dev *pdev))
@@ -195,15 +206,17 @@ pci_ers_result_t pcie_do_recovery(struct pci_dev *dev,
if (state == pci_channel_io_frozen) {
pci_bridge_walk(bridge, report_frozen_detected, &status);
if (type == PCI_EXP_TYPE_RC_END) {
- pci_warn(dev, "link reset not possible for RCiEP\n");
- status = PCI_ERS_RESULT_NONE;
- goto failed;
- }
-
- status = reset_subordinate_devices(bridge);
- if (status != PCI_ERS_RESULT_RECOVERED) {
- pci_warn(dev, "subordinate device reset failed\n");
- goto failed;
+ status = flr_on_rciep(dev);
+ if (status != PCI_ERS_RESULT_RECOVERED) {
+ pci_warn(dev, "function level reset failed\n");
+ goto failed;
+ }
+ } else {
+ status = reset_subordinate_devices(bridge);
+ if (status != PCI_ERS_RESULT_RECOVERED) {
+ pci_warn(dev, "subordinate device reset failed\n");
+ goto failed;
+ }
}
} else {
pci_bridge_walk(bridge, report_normal_detected, &status);
--
2.28.0

2020-09-25 22:00:19

by Bjorn Helgaas

Subject: Re: [PATCH v6 05/10] PCI/AER: Apply function level reset to RCiEP on fatal error

On Tue, Sep 22, 2020 at 02:38:54PM -0700, Sean V Kelley wrote:
> From: Qiuxu Zhuo <[email protected]>
>
> Attempt to do function level reset for an RCiEP associated with an
> RCEC device on fatal error.
>
> Signed-off-by: Qiuxu Zhuo <[email protected]>
> ---
> drivers/pci/pcie/err.c | 31 ++++++++++++++++++++++---------
> 1 file changed, 22 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/pci/pcie/err.c b/drivers/pci/pcie/err.c
> index e575fa6cee63..5380ecc41506 100644
> --- a/drivers/pci/pcie/err.c
> +++ b/drivers/pci/pcie/err.c
> @@ -169,6 +169,17 @@ static void pci_bridge_walk(struct pci_dev *bridge, int (*cb)(struct pci_dev *,
> cb(bridge, userdata);
> }
>
> +static pci_ers_result_t flr_on_rciep(struct pci_dev *dev)
> +{
> + if (!pcie_has_flr(dev))
> + return PCI_ERS_RESULT_NONE;
> +
> + if (pcie_flr(dev))
> + return PCI_ERS_RESULT_DISCONNECT;
> +
> + return PCI_ERS_RESULT_RECOVERED;
> +}

Either we reset the device or we didn't; there is no third option.

If I read it correctly, nothing in pcie_do_recovery() cares about the
difference between PCI_ERS_RESULT_NONE and PCI_ERS_RESULT_DISCONNECT.
The status does get returned, but only edr_handle_event() looks at it,
and it doesn't care about the difference either.

So I think this should just return PCI_ERS_RESULT_RECOVERED or
PCI_ERS_RESULT_DISCONNECT.

> pci_ers_result_t pcie_do_recovery(struct pci_dev *dev,
> pci_channel_state_t state,
> pci_ers_result_t (*reset_subordinate_devices)(struct pci_dev *pdev))
> @@ -195,15 +206,17 @@ pci_ers_result_t pcie_do_recovery(struct pci_dev *dev,
> if (state == pci_channel_io_frozen) {
> pci_bridge_walk(bridge, report_frozen_detected, &status);
> if (type == PCI_EXP_TYPE_RC_END) {
> - pci_warn(dev, "link reset not possible for RCiEP\n");
> - status = PCI_ERS_RESULT_NONE;
> - goto failed;
> - }
> -
> - status = reset_subordinate_devices(bridge);
> - if (status != PCI_ERS_RESULT_RECOVERED) {
> - pci_warn(dev, "subordinate device reset failed\n");
> - goto failed;
> + status = flr_on_rciep(dev);
> + if (status != PCI_ERS_RESULT_RECOVERED) {
> + pci_warn(dev, "function level reset failed\n");
> + goto failed;
> + }
> + } else {
> + status = reset_subordinate_devices(bridge);
> + if (status != PCI_ERS_RESULT_RECOVERED) {
> + pci_warn(dev, "subordinate device reset failed\n");
> + goto failed;
> + }
> }
> } else {
> pci_bridge_walk(bridge, report_normal_detected, &status);
> --
> 2.28.0
>

2020-09-25 22:56:22

by Kelley, Sean V

Subject: Re: [PATCH v6 05/10] PCI/AER: Apply function level reset to RCiEP on fatal error

On 25 Sep 2020, at 14:58, Bjorn Helgaas wrote:

> On Tue, Sep 22, 2020 at 02:38:54PM -0700, Sean V Kelley wrote:
>> From: Qiuxu Zhuo <[email protected]>
>>
>> Attempt to do function level reset for an RCiEP associated with an
>> RCEC device on fatal error.
>>
>> Signed-off-by: Qiuxu Zhuo <[email protected]>
>> ---
>> drivers/pci/pcie/err.c | 31 ++++++++++++++++++++++---------
>> 1 file changed, 22 insertions(+), 9 deletions(-)
>>
>> diff --git a/drivers/pci/pcie/err.c b/drivers/pci/pcie/err.c
>> index e575fa6cee63..5380ecc41506 100644
>> --- a/drivers/pci/pcie/err.c
>> +++ b/drivers/pci/pcie/err.c
>> @@ -169,6 +169,17 @@ static void pci_bridge_walk(struct pci_dev
>> *bridge, int (*cb)(struct pci_dev *,
>> cb(bridge, userdata);
>> }
>>
>> +static pci_ers_result_t flr_on_rciep(struct pci_dev *dev)
>> +{
>> + if (!pcie_has_flr(dev))
>> + return PCI_ERS_RESULT_NONE;
>> +
>> + if (pcie_flr(dev))
>> + return PCI_ERS_RESULT_DISCONNECT;
>> +
>> + return PCI_ERS_RESULT_RECOVERED;
>> +}
>
> Either we reset the device or we didn't; there is no third option.
>
> If I read it correctly, nothing in pcie_do_recovery() cares about the
> difference between PCI_ERS_RESULT_NONE and PCI_ERS_RESULT_DISCONNECT.
> The status does get returned, but only edr_handle_event() looks at it,
> and it doesn't care about the difference either.
>
> So I think this should just return PCI_ERS_RESULT_RECOVERED or
> PCI_ERS_RESULT_DISCONNECT.

You raise a good point. As currently implemented, the usage requires
either a reset or not result. Will change.

Thanks,

Sean

>
>> pci_ers_result_t pcie_do_recovery(struct pci_dev *dev,
>> pci_channel_state_t state,
>> pci_ers_result_t (*reset_subordinate_devices)(struct pci_dev
>> *pdev))
>> @@ -195,15 +206,17 @@ pci_ers_result_t pcie_do_recovery(struct
>> pci_dev *dev,
>> if (state == pci_channel_io_frozen) {
>> pci_bridge_walk(bridge, report_frozen_detected, &status);
>> if (type == PCI_EXP_TYPE_RC_END) {
>> - pci_warn(dev, "link reset not possible for RCiEP\n");
>> - status = PCI_ERS_RESULT_NONE;
>> - goto failed;
>> - }
>> -
>> - status = reset_subordinate_devices(bridge);
>> - if (status != PCI_ERS_RESULT_RECOVERED) {
>> - pci_warn(dev, "subordinate device reset failed\n");
>> - goto failed;
>> + status = flr_on_rciep(dev);
>> + if (status != PCI_ERS_RESULT_RECOVERED) {
>> + pci_warn(dev, "function level reset failed\n");
>> + goto failed;
>> + }
>> + } else {
>> + status = reset_subordinate_devices(bridge);
>> + if (status != PCI_ERS_RESULT_RECOVERED) {
>> + pci_warn(dev, "subordinate device reset failed\n");
>> + goto failed;
>> + }
>> }
>> } else {
>> pci_bridge_walk(bridge, report_normal_detected, &status);
>> --
>> 2.28.0
>>