2022-01-06 16:03:42

by Jim Quinlan

[permalink] [raw]
Subject: [PATCH v11 0/7] PCI: brcmstb: root port turns on sub-device power

v11 -- Only one commit has changed from v10: "PCI: brcmstb: Add mechanism
to turn on subdev regulators". Lorenzo requested that this commit's
code be moved from pci/pci.h and pci/bus.c to
pci/controller/pcie-brcmstb.c as a stopgap measure until we figure
out a better way of offering this feature for all. This will most
likely involve Pali's upcoming changes to refactor PCI host probe
code.

v10 -- Bindings commit example: in comment, refer to bridge under
controller node as a root port. (Pali)
-- Bindings commit example: remove three properties that are not
appropriate for a PCIe endpoint node. (Rob)

v9 -- Simplify where this mechanism works: instead of looking for
regulators below every bridge, just look for them at the
bridge under the root bus (root port). Now there is no
modification of portdrv_{pci,core}.c in this submission.
-- Although Pali is working on support for probing native
PCIe controller drivers, this work may take some time to
implement and it still might not be able to accomodate
our driver's requirements (e.g. vreg suspend/resume control).
-- Move regulator suspend/resume control to Brcm RC driver. It
must reside there because (a) in order to know when to
initiate linkup during resume and (b) to turn on the
regulators before any config-space accesses occur.
-- Commit message spelling, word choice (Bjorn, Krzysztof)
-- Refactor a small commit that was ignoring a funcs' return
values (Bjorn).
-- Here is a summary of this mechanism:

If:
-- PCIe RC driver sets pci_ops {add,remove)_bus to
pci_subdev_regulators_{add,remove}_bus during its probe.
-- There is a DT node "RB" under the host bridge DT node.
-- During the RC driver's pci_host_probe() the add_bus callback
is invoked where (bus->parent && pci_is_root_bus(bus->parent)
is true

Then:
-- A struct subdev_regulators structure will be allocated and
assigned to bus->dev.driver_data.
-- regulator_bulk_{get,enable}() will be invoked on &bus->dev
and the former will search for and process any
vpcie{12v,3v3,3v3aux}-supply properties that reside in node "RB".
-- The regulators will be turned off/on for any unbind/bind operations.
-- The regulators will be turned off/on for any suspend/resumes, but
only if the RC driver handles this on its own. This will appear
in a later commit for the pcie-brcmstb.c driver.

v8 -- Only the two binding commits and the "Change brcm_phy_stop()" commit
are unchanged.
-- The code has been moved to portdrv_pci.c and bus.c. The regulators
are placed in bus->dev.driver_data (bus->sysdata is already occupied
by the Broadcom PCIe). Two functions, pci_subdev_regulators_{add,remove}_bus()
are created to turn the regulators on/off. The pcie_portdriver also sets
its pci_driver methods suspend and resume when the conditions are
right for this feature. (Robh for suggestions, although I probably
erred in following them).
-- Have the root complex return 0xffffffff on accesses even when
the link is down and the HW doesn't support such accesses (PaliR).
-- Just call devm_bulk_regulator_get() on standard supplies; don't
bother pre-scanning the DT for them (MarkB).

v7 -- RobH suggested putting the "vpcixxx-supply" property under the
bridge-node rather than the endpoint device node. Also, he said to
use the pci-ops add_bus/remove methods. Doing so simplifies the
code greatly and three commits were dropped. Thanks!

-- Rob also suggested (I think) having this patchset be a general
feature which is activated by an OF property under the bridge node.
I tried to do that but realized that our root complex driver
controls the regulators with its dev_pm_ops and there is no way to
transfer this control when using general mechanism. Note that
although the regulator core deals with suspend, our RC driver wants
the right to sometimes to preclude this for WOL scenarios.

-- One commit was added to change the response to the return value of
the pci_ops add_bus() method. Currently, an error causes a WARNING,
a dev_err(...), and continues to return the child bus. The
modification was, for returning -ENOLINK only, to skip WARNING &
dev_err() and return NULL. This is necessary for our RC HW, as if
the code continues on it will do a pci_read_config_dword() for the
vendor/id, and our HW flags a CPU abort (instead of returning
0xffffffff) when the is no pcie-link established.

[NOTE: MarkB, I did not add one of your two "Reviewed-by"s
because the commit had a decent amount of change.]

v6 -- Dropped the idea of a placeholder regulator
property (brcm-ep-a-supply). (MarkB)
-- device_initialize() now called once. Two
commits were added for this. (GKH)
-- In two cases, separated a single function
into two or more functions (MarkB)
-- "(void)foo();" => "foo()". Note that although
foo() returns an int, in this instance it is being
invoked within a function returning void, and foo()
already executes a dev_err() on error. (MarkB)
-- Added a commit to correct PCIe interrupts in YAML.
-- Removed "device_type = "pci";" for the EP node
in the YAML example.
-- Updated the URL related to the voltage regulator
names on GitHub. Note that I added vpciev3v3aux.

v5 [NOTE: It has been a while since v4. Sorry]
-- See "PCI: allow for callback to prepare nascent subdev"
commit message for the cornerstone of this patchset
and the reasons behind it. This is a new commit.
-- The RC driver now looks into its DT children and
turns on regulators for a sub-device, and this occurs
prior to PCIe link as it must.
-- Dropped commits not related to the focus of this patchset.

v4 [NOTE: I'm not sure this fixes RobH and MarkB constraints but I'd
like to use this pullreq as a basis for future discussion.]
[Commit: Add bindings for ...]
-- Fix syntax error in YAML bindings example (RobH)
-- {vpcie12v,vpcie3v3}-supply props are back in root complex DT node
(I believe RobH said this was okay)
[Commit: Add control of ..]
-- Do not do global search for regulator; now we look specifically
for the property {vpcie12v,vpcie3v3}-supply in the root complex
DT node and then call devm_regulator_bulk_get() (MarkB)
-- Use devm_regulator_bulk_get() (Bjorn)
-- s/EP/slot0 device/ (Bjorn)
-- Spelling, capitalization (Bjorn)
-- Have brcm_phy_stop() return a void (Bjorn)
[Commit: Do not turn off ...]
-- Capitalization (Bjorn)
[Commit: Check return value ...]
-- Commit message content (Bjorn)
-- Move 6/6 hunk to 2/6 where it belongs (Bjorn)
-- Move the rest of 6/6 before all other commits (Bjorn)

v3 -- Driver now searches for EP DT subnode for any regulators to turn on.
If present, these regulators have the property names
"vpcie12v-supply" and "vpcie3v3-supply". The existence of these
regulators in the EP subnode are currently pending as a pullreq
in pci-bus.yaml at
https://github.com/devicetree-org/dt-schema/pull/54
(MarkB, RobH).
-- Check return of brcm_set_regulators() (Florian)
-- Specify one regulator string per line for easier update (Florian)
-- Author/Committer/Signoff email changed from that of V2 from
'[email protected]' to '[email protected]'.

v2 -- Use regulator bulk API rather than multiple calls (MarkB).

v1 -- Bindings are added for fixed regulators that may power the EP device.
-- The brcmstb RC driver is modified to control these regulators
during probe, suspend, and resume.
-- 7216 type SOCs have additional error reporting HW and a
panic handler is added to dump its info.
-- A missing return value check is added.

Jim Quinlan (7):
PCI: brcmstb: Fix function return value handling
dt-bindings: PCI: Correct brcmstb interrupts, interrupt-map.
dt-bindings: PCI: Add bindings for Brcmstb EP voltage regulators
PCI: brcmstb: Split brcm_pcie_setup() into two funcs
PCI: brcmstb: Add mechanism to turn on subdev regulators
PCI: brcmstb: Add control of subdevice voltage regulators
PCI: brcmstb: Do not turn off WOL regulators on suspend

.../bindings/pci/brcm,stb-pcie.yaml | 27 +-
drivers/pci/controller/pcie-brcmstb.c | 284 +++++++++++++++---
2 files changed, 273 insertions(+), 38 deletions(-)


base-commit: c9e6606c7fe92b50a02ce51dda82586ebdf99b48
prerequisite-patch-id: 0905430e81a95900a1366916fe2940b848317a7c
prerequisite-patch-id: 710896210c50354d87f6025fe0bd1b89981138eb
prerequisite-patch-id: 97d3886cb911cb12ef3d514fdfff2a0ab11e8570
prerequisite-patch-id: 241f1e1878fc177d941f4982ca12779a29feb62b
prerequisite-patch-id: d856608825e2294297db5d7f88f8c180f3e5a1f2
prerequisite-patch-id: 92bcbc9772fb4d248157bcf35e799ac37be8ee45
prerequisite-patch-id: 6f4b1aac459bb54523ade0e87c04e9d6c45bd9f5
prerequisite-patch-id: 090ee7a3112a4ecb03805b23ed10e2c96b3b34ed
prerequisite-patch-id: b3de6b4a4dd7a2f57b86e6548eb345df59de497b
--
2.17.1



2022-01-06 16:03:47

by Jim Quinlan

[permalink] [raw]
Subject: [PATCH v11 1/7] PCI: brcmstb: Fix function return value handling

Do at least a dev_err() on some calls to reset_control_rearm() and
brcm_phy_stop(). In some cases it may not make sense to return this error
value "above" as doing so will cause more trouble than is warranted.

Signed-off-by: Jim Quinlan <[email protected]>
Acked-by: Florian Fainelli <[email protected]>
---
drivers/pci/controller/pcie-brcmstb.c | 28 +++++++++++++++++++++------
1 file changed, 22 insertions(+), 6 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 1fc7bd49a7ad..9ed79ddb6a83 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -1146,11 +1146,23 @@ static int brcm_pcie_suspend(struct device *dev)
int ret;

brcm_pcie_turn_off(pcie);
- ret = brcm_phy_stop(pcie);
- reset_control_rearm(pcie->rescal);
+ /*
+ * If brcm_phy_stop() returns an error, just dev_err(). If we
+ * return the error it will cause the suspend to fail and this is a
+ * forgivable offense that will probably be erased on resume.
+ */
+ if (brcm_phy_stop(pcie))
+ dev_err(dev, "Could not stop phy for suspend\n");
+
+ ret = reset_control_rearm(pcie->rescal);
+ if (ret) {
+ dev_err(dev, "Could not rearm rescal reset\n");
+ return ret;
+ }
+
clk_disable_unprepare(pcie->clk);

- return ret;
+ return 0;
}

static int brcm_pcie_resume(struct device *dev)
@@ -1161,7 +1173,9 @@ static int brcm_pcie_resume(struct device *dev)
int ret;

base = pcie->base;
- clk_prepare_enable(pcie->clk);
+ ret = clk_prepare_enable(pcie->clk);
+ if (ret)
+ return ret;

ret = reset_control_reset(pcie->rescal);
if (ret)
@@ -1202,8 +1216,10 @@ static void __brcm_pcie_remove(struct brcm_pcie *pcie)
{
brcm_msi_remove(pcie);
brcm_pcie_turn_off(pcie);
- brcm_phy_stop(pcie);
- reset_control_rearm(pcie->rescal);
+ if (brcm_phy_stop(pcie))
+ dev_err(pcie->dev, "Could not stop phy\n");
+ if (reset_control_rearm(pcie->rescal))
+ dev_err(pcie->dev, "Could not rearm rescal reset\n");
clk_disable_unprepare(pcie->clk);
}

--
2.17.1


2022-01-06 16:03:49

by Jim Quinlan

[permalink] [raw]
Subject: [PATCH v11 2/7] dt-bindings: PCI: Correct brcmstb interrupts, interrupt-map.

The "pcie" and "msi" interrupts were given the same interrupt when they are
actually different. Interrupt-map only had the INTA entry; the INTB, INTC,
and INTD entries are added.

Signed-off-by: Jim Quinlan <[email protected]>
Acked-by: Florian Fainelli <[email protected]>
Acked-by: Rob Herring <[email protected]>
---
Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
index 1fe102743f82..22f2ef446f18 100644
--- a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
+++ b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
@@ -143,11 +143,15 @@ examples:
#address-cells = <3>;
#size-cells = <2>;
#interrupt-cells = <1>;
- interrupts = <GIC_SPI 148 IRQ_TYPE_LEVEL_HIGH>,
+ interrupts = <GIC_SPI 147 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 148 IRQ_TYPE_LEVEL_HIGH>;
interrupt-names = "pcie", "msi";
interrupt-map-mask = <0x0 0x0 0x0 0x7>;
- interrupt-map = <0 0 0 1 &gicv2 GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-map = <0 0 0 1 &gicv2 GIC_SPI 143 IRQ_TYPE_LEVEL_HIGH
+ 0 0 0 2 &gicv2 GIC_SPI 144 IRQ_TYPE_LEVEL_HIGH
+ 0 0 0 3 &gicv2 GIC_SPI 145 IRQ_TYPE_LEVEL_HIGH
+ 0 0 0 4 &gicv2 GIC_SPI 146 IRQ_TYPE_LEVEL_HIGH>;
+
msi-parent = <&pcie0>;
msi-controller;
ranges = <0x02000000 0x0 0xf8000000 0x6 0x00000000 0x0 0x04000000>;
--
2.17.1


2022-01-06 16:03:52

by Jim Quinlan

[permalink] [raw]
Subject: [PATCH v11 3/7] dt-bindings: PCI: Add bindings for Brcmstb EP voltage regulators

Add bindings for Brcmstb EP voltage regulators. A new mechanism is to be
added to the Linux PCI subsystem that will allocate and turn on/off
regulators. These are standard regulators -- vpcie12v, vpcie3v3, and
vpcie3v3aux -- placed in the DT in the bridge node under the host bridge
device.

The use of a regulator property in the pcie EP subnode such as
"vpcie12v-supply" depends on a pending pullreq to the pci-bus.yaml
file at

https://github.com/devicetree-org/dt-schema/pull/63

Reviewed-by: Rob Herring <[email protected]>
Signed-off-by: Jim Quinlan <[email protected]>
---
.../bindings/pci/brcm,stb-pcie.yaml | 19 +++++++++++++++++++
1 file changed, 19 insertions(+)

diff --git a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
index 22f2ef446f18..3e3c8929c97c 100644
--- a/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
+++ b/Documentation/devicetree/bindings/pci/brcm,stb-pcie.yaml
@@ -159,5 +159,24 @@ examples:
<0x42000000 0x1 0x80000000 0x3 0x00000000 0x0 0x80000000>;
brcm,enable-ssc;
brcm,scb-sizes = <0x0000000080000000 0x0000000080000000>;
+
+ /* PCIe bridge, Root Port */
+ pci@0,0 {
+ #address-cells = <3>;
+ #size-cells = <2>;
+ reg = <0x0 0x0 0x0 0x0 0x0>;
+ compatible = "pciclass,0604";
+ device_type = "pci";
+ vpcie3v3-supply = <&vreg7>;
+ ranges;
+
+ /* PCIe endpoint */
+ pci-ep@0,0 {
+ assigned-addresses =
+ <0x82010000 0x0 0xf8000000 0x6 0x00000000 0x0 0x2000>;
+ reg = <0x0 0x0 0x0 0x0 0x0>;
+ compatible = "pci14e4,1688";
+ };
+ };
};
};
--
2.17.1


2022-01-06 16:03:55

by Jim Quinlan

[permalink] [raw]
Subject: [PATCH v11 4/7] PCI: brcmstb: Split brcm_pcie_setup() into two funcs

We need to take some code in brcm_pcie_setup() and put it in a new function
brcm_pcie_linkup(). In future commits the brcm_pcie_linkup() function will
be called indirectly by pci_host_probe() as opposed to the host driver
invoking it directly.

Some code that was executed after the PCIe linkup is now placed so that it
executes prior to linkup, since this code has to run prior to the
invocation of pci_host_probe().

Signed-off-by: Jim Quinlan <[email protected]>
---
drivers/pci/controller/pcie-brcmstb.c | 65 ++++++++++++++++-----------
1 file changed, 39 insertions(+), 26 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 9ed79ddb6a83..5f373227aad6 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -863,16 +863,9 @@ static inline int brcm_pcie_get_rc_bar2_size_and_offset(struct brcm_pcie *pcie,

static int brcm_pcie_setup(struct brcm_pcie *pcie)
{
- struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
u64 rc_bar2_offset, rc_bar2_size;
void __iomem *base = pcie->base;
- struct device *dev = pcie->dev;
- struct resource_entry *entry;
- bool ssc_good = false;
- struct resource *res;
- int num_out_wins = 0;
- u16 nlw, cls, lnksta;
- int i, ret, memc;
+ int ret, memc;
u32 tmp, burst, aspm_support;

/* Reset the bridge */
@@ -957,6 +950,40 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
if (pcie->gen)
brcm_pcie_set_gen(pcie, pcie->gen);

+ /* Don't advertise L0s capability if 'aspm-no-l0s' */
+ aspm_support = PCIE_LINK_STATE_L1;
+ if (!of_property_read_bool(pcie->np, "aspm-no-l0s"))
+ aspm_support |= PCIE_LINK_STATE_L0S;
+ tmp = readl(base + PCIE_RC_CFG_PRIV1_LINK_CAPABILITY);
+ u32p_replace_bits(&tmp, aspm_support,
+ PCIE_RC_CFG_PRIV1_LINK_CAPABILITY_ASPM_SUPPORT_MASK);
+ writel(tmp, base + PCIE_RC_CFG_PRIV1_LINK_CAPABILITY);
+
+ /*
+ * For config space accesses on the RC, show the right class for
+ * a PCIe-PCIe bridge (the default setting is to be EP mode).
+ */
+ tmp = readl(base + PCIE_RC_CFG_PRIV1_ID_VAL3);
+ u32p_replace_bits(&tmp, 0x060400,
+ PCIE_RC_CFG_PRIV1_ID_VAL3_CLASS_CODE_MASK);
+ writel(tmp, base + PCIE_RC_CFG_PRIV1_ID_VAL3);
+
+ return 0;
+}
+
+static int brcm_pcie_linkup(struct brcm_pcie *pcie)
+{
+ struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
+ struct device *dev = pcie->dev;
+ void __iomem *base = pcie->base;
+ struct resource_entry *entry;
+ struct resource *res;
+ int num_out_wins = 0;
+ u16 nlw, cls, lnksta;
+ bool ssc_good = false;
+ u32 tmp;
+ int ret, i;
+
/* Unassert the fundamental reset */
pcie->perst_set(pcie, 0);

@@ -994,24 +1021,6 @@ static int brcm_pcie_setup(struct brcm_pcie *pcie)
num_out_wins++;
}

- /* Don't advertise L0s capability if 'aspm-no-l0s' */
- aspm_support = PCIE_LINK_STATE_L1;
- if (!of_property_read_bool(pcie->np, "aspm-no-l0s"))
- aspm_support |= PCIE_LINK_STATE_L0S;
- tmp = readl(base + PCIE_RC_CFG_PRIV1_LINK_CAPABILITY);
- u32p_replace_bits(&tmp, aspm_support,
- PCIE_RC_CFG_PRIV1_LINK_CAPABILITY_ASPM_SUPPORT_MASK);
- writel(tmp, base + PCIE_RC_CFG_PRIV1_LINK_CAPABILITY);
-
- /*
- * For config space accesses on the RC, show the right class for
- * a PCIe-PCIe bridge (the default setting is to be EP mode).
- */
- tmp = readl(base + PCIE_RC_CFG_PRIV1_ID_VAL3);
- u32p_replace_bits(&tmp, 0x060400,
- PCIE_RC_CFG_PRIV1_ID_VAL3_CLASS_CODE_MASK);
- writel(tmp, base + PCIE_RC_CFG_PRIV1_ID_VAL3);
-
if (pcie->ssc) {
ret = brcm_pcie_set_ssc(pcie);
if (ret == 0)
@@ -1200,6 +1209,10 @@ static int brcm_pcie_resume(struct device *dev)
if (ret)
goto err_reset;

+ ret = brcm_pcie_linkup(pcie);
+ if (ret)
+ goto err_reset;
+
if (pcie->msi)
brcm_msi_set_regs(pcie->msi);

--
2.17.1


2022-01-06 16:03:59

by Jim Quinlan

[permalink] [raw]
Subject: [PATCH v11 5/7] PCI: brcmstb: Add mechanism to turn on subdev regulators

Add a mechanism to identify standard PCIe regulators in the DT, allocate
them, and turn them on before the rest of the bus is scanned during
pci_host_probe().

The allocated structure that contains the regulators is stored in the port
driver dev.driver_data field. Here is a point-by-point of how and
when this mechanism is activated:

If:
-- PCIe RC driver sets pci_ops {add,remove)_bus to
pci_subdev_regulators_{add,remove}_bus during its probe.
-- There is a DT node "RB" under the host bridge DT node.
-- During the RC driver's pci_host_probe() the add_bus callback
is invoked where (bus->parent && pci_is_root_bus(bus->parent)
is true

Then:
-- A struct subdev_regulators structure will be allocated and
assigned to bus->dev.driver_data.
-- regulator_bulk_{get,enable} will be invoked on &bus->dev
and the former will search for and process any
vpcie{12v,3v3,3v3aux}-supply properties that reside in node "RB".
-- The regulators will be turned off/on for any unbind/bind operations.
-- The regulators will be turned off/on for any suspend/resumes, but
only if the RC driver handles this on its own. This will appear
in a later commit for the pcie-brcmstb.c driver.

The unabridged reason for doing this is as follows. We would like the
Broadcom STB PCIe root complex driver (and others) to be able to turn
off/on regulators[1] that provide power to endpoint[2] devices. Typically,
the drivers of these endpoint devices are stock Linux drivers that are not
aware that these regulator(s) exist and must be turned on for the driver to
be probed. The simple solution of course is to turn these regulators on at
boot and keep them on. However, this solution does not satisfy at least
three of our usage modes:

1. For example, one customer uses multiple PCIe controllers, but wants the
ability to, by script invoking and unbind, turn any or all of them by and
their subdevices off to save power, e.g. when in battery mode.

2. Another example is when a watchdog script discovers that an endpoint
device is in an unresponsive state and would like to unbind, power toggle,
and re-bind just the PCIe endpoint and controller.

3. Of course we also want power turned off during suspend mode. However,
some endpoint devices may be able to "wake" during suspend and we need to
recognise this case and veto the nominal act of turning off its regulator.
Such is the case with Wake-on-LAN and Wake-on-WLAN support where PCIe
end-point device needs to be kept powered on in order to receive network
packets and wake-up the system.

In all of these cases it is advantageous for the PCIe controller to govern
the turning off/on the regulators needed by the endpoint device. The first
two cases can be done by simply unbinding and binding the PCIe controller,
if the controller has control of these regulators.

[1] These regulators typically govern the actual power supply to the
endpoint chip. Sometimes they may be the official PCIe socket
power -- such as 3.3v or aux-3.3v. Sometimes they are truly
the regulator(s) that supply power to the EP chip.

[2] The 99% configuration of our boards is a single endpoint device
attached to the PCIe controller. I use the term endpoint but it could
possibly mean a switch as well.

Signed-off-by: Jim Quinlan <[email protected]>
---
drivers/pci/controller/pcie-brcmstb.c | 76 +++++++++++++++++++++++++++
1 file changed, 76 insertions(+)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 5f373227aad6..7d46eccf17ff 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -24,6 +24,7 @@
#include <linux/pci.h>
#include <linux/pci-ecam.h>
#include <linux/printk.h>
+#include <linux/regulator/consumer.h>
#include <linux/reset.h>
#include <linux/sizes.h>
#include <linux/slab.h>
@@ -257,6 +258,14 @@ static const struct pcie_cfg_data bcm2711_cfg = {
.bridge_sw_init_set = brcm_pcie_bridge_sw_init_set_generic,
};

+struct subdev_regulators {
+ unsigned int num_supplies;
+ struct regulator_bulk_data supplies[];
+};
+
+static int pci_subdev_regulators_add_bus(struct pci_bus *bus);
+static void pci_subdev_regulators_remove_bus(struct pci_bus *bus);
+
struct brcm_msi {
struct device *dev;
void __iomem *base;
@@ -406,6 +415,71 @@ static int brcm_pcie_set_ssc(struct brcm_pcie *pcie)
return ssc && pll ? 0 : -EIO;
}

+static void *alloc_subdev_regulators(struct device *dev)
+{
+ static const char * const supplies[] = {
+ "vpcie3v3",
+ "vpcie3v3aux",
+ "vpcie12v",
+ };
+ const size_t size = sizeof(struct subdev_regulators)
+ + sizeof(struct regulator_bulk_data) * ARRAY_SIZE(supplies);
+ struct subdev_regulators *sr;
+ int i;
+
+ sr = devm_kzalloc(dev, size, GFP_KERNEL);
+ if (sr) {
+ sr->num_supplies = ARRAY_SIZE(supplies);
+ for (i = 0; i < ARRAY_SIZE(supplies); i++)
+ sr->supplies[i].supply = supplies[i];
+ }
+
+ return sr;
+}
+
+static int pci_subdev_regulators_add_bus(struct pci_bus *bus)
+{
+ struct device *dev = &bus->dev;
+ struct subdev_regulators *sr;
+ int ret;
+
+ if (!dev->of_node || !bus->parent || !pci_is_root_bus(bus->parent))
+ return 0;
+
+ if (dev->driver_data)
+ dev_err(dev, "dev.driver_data unexpectedly non-NULL\n");
+
+ sr = alloc_subdev_regulators(dev);
+ if (!sr)
+ return -ENOMEM;
+
+ dev->driver_data = sr;
+ ret = regulator_bulk_get(dev, sr->num_supplies, sr->supplies);
+ if (ret)
+ return ret;
+
+ ret = regulator_bulk_enable(sr->num_supplies, sr->supplies);
+ if (ret) {
+ dev_err(dev, "failed to enable regulators for downstream device\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static void pci_subdev_regulators_remove_bus(struct pci_bus *bus)
+{
+ struct device *dev = &bus->dev;
+ struct subdev_regulators *sr = dev->driver_data;
+
+ if (!sr || !bus->parent || !pci_is_root_bus(bus->parent))
+ return;
+
+ if (regulator_bulk_disable(sr->num_supplies, sr->supplies))
+ dev_err(dev, "failed to disable regulators for downstream device\n");
+ dev->driver_data = NULL;
+}
+
/* Limits operation to a specific generation (1, 2, or 3) */
static void brcm_pcie_set_gen(struct brcm_pcie *pcie, int gen)
{
@@ -722,6 +796,8 @@ static struct pci_ops brcm_pcie_ops = {
.map_bus = brcm_pcie_map_conf,
.read = pci_generic_config_read,
.write = pci_generic_config_write,
+ .add_bus = pci_subdev_regulators_add_bus,
+ .remove_bus = pci_subdev_regulators_remove_bus,
};

static inline void brcm_pcie_bridge_sw_init_set_generic(struct brcm_pcie *pcie, u32 val)
--
2.17.1


2022-01-06 16:04:03

by Jim Quinlan

[permalink] [raw]
Subject: [PATCH v11 6/7] PCI: brcmstb: Add control of subdevice voltage regulators

This Broadcom STB PCIe RC driver has one port and connects directly to one
device, be it a switch or an endpoint. We want to be able to leverage
the recently added mechansim that allocates and turns on/off subdevice
regulators.

All that needs to be done is to put the regulator DT nodes in the bridge
below host and to set the pci_ops methods add_bus and remove_bus.

Note that the pci_subdev_regulators_add_bus() method is wrapped for two
reasons:

1. To acheive linkup after the voltage regulators are turned on.

2. If, in the case of an unsuccessful linkup, to redirect
any PCIe accesses to subdevices, e.g. the scan for
DEV/ID. This redirection is needed because the Broadcom
PCIe HW wil issue a CPU abort if such an access is made when
there is no linkup.

Signed-off-by: Jim Quinlan <[email protected]>
---
drivers/pci/controller/pcie-brcmstb.c | 82 +++++++++++++++++++++++++--
1 file changed, 77 insertions(+), 5 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 7d46eccf17ff..3c1b2fefe7e1 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -192,6 +192,8 @@ static inline void brcm_pcie_bridge_sw_init_set_generic(struct brcm_pcie *pcie,
static inline void brcm_pcie_perst_set_4908(struct brcm_pcie *pcie, u32 val);
static inline void brcm_pcie_perst_set_7278(struct brcm_pcie *pcie, u32 val);
static inline void brcm_pcie_perst_set_generic(struct brcm_pcie *pcie, u32 val);
+static int brcm_pcie_linkup(struct brcm_pcie *pcie);
+static int brcm_pcie_add_bus(struct pci_bus *bus);

enum {
RGR1_SW_INIT_1,
@@ -304,6 +306,8 @@ struct brcm_pcie {
u32 hw_rev;
void (*perst_set)(struct brcm_pcie *pcie, u32 val);
void (*bridge_sw_init_set)(struct brcm_pcie *pcie, u32 val);
+ bool refusal_mode;
+ struct subdev_regulators *sr;
};

/*
@@ -467,6 +471,34 @@ static int pci_subdev_regulators_add_bus(struct pci_bus *bus)
return 0;
}

+static int brcm_pcie_add_bus(struct pci_bus *bus)
+{
+ struct device *dev = &bus->dev;
+ struct brcm_pcie *pcie = (struct brcm_pcie *) bus->sysdata;
+ int ret;
+
+ if (!dev->of_node || !bus->parent || !pci_is_root_bus(bus->parent))
+ return 0;
+
+ ret = pci_subdev_regulators_add_bus(bus);
+ if (ret)
+ return ret;
+
+ /* Grab the regulators for suspend/resume */
+ pcie->sr = bus->dev.driver_data;
+
+ /*
+ * If we have failed linkup there is no point to return an error as
+ * currently it will cause a WARNING() from pci_alloc_child_bus().
+ * We return 0 and turn on the "refusal_mode" so that any further
+ * accesses to the pci_dev just get 0xffffffff
+ */
+ if (brcm_pcie_linkup(pcie) != 0)
+ pcie->refusal_mode = true;
+
+ return 0;
+}
+
static void pci_subdev_regulators_remove_bus(struct pci_bus *bus)
{
struct device *dev = &bus->dev;
@@ -785,6 +817,18 @@ static void __iomem *brcm_pcie_map_conf(struct pci_bus *bus, unsigned int devfn,
/* Accesses to the RC go right to the RC registers if slot==0 */
if (pci_is_root_bus(bus))
return PCI_SLOT(devfn) ? NULL : base + where;
+ if (pcie->refusal_mode) {
+ /*
+ * At this point we do not have link. There will be a CPU
+ * abort -- a quirk with this controller --if Linux tries
+ * to read any config-space registers besides those
+ * targeting the host bridge. To prevent this we hijack
+ * the address to point to a safe access that will return
+ * 0xffffffff.
+ */
+ writel(0xffffffff, base + PCIE_MISC_RC_BAR2_CONFIG_HI);
+ return base + PCIE_MISC_RC_BAR2_CONFIG_HI + (where & 0x3);
+ }

/* For devices, write to the config space index register */
idx = PCIE_ECAM_OFFSET(bus->number, devfn, 0);
@@ -796,7 +840,7 @@ static struct pci_ops brcm_pcie_ops = {
.map_bus = brcm_pcie_map_conf,
.read = pci_generic_config_read,
.write = pci_generic_config_write,
- .add_bus = pci_subdev_regulators_add_bus,
+ .add_bus = brcm_pcie_add_bus,
.remove_bus = pci_subdev_regulators_remove_bus,
};

@@ -1245,6 +1289,14 @@ static int brcm_pcie_suspend(struct device *dev)
return ret;
}

+ if (pcie->sr) {
+ ret = regulator_bulk_disable(pcie->sr->num_supplies, pcie->sr->supplies);
+ if (ret) {
+ dev_err(dev, "Could not turn off regulators\n");
+ reset_control_reset(pcie->rescal);
+ return ret;
+ }
+ }
clk_disable_unprepare(pcie->clk);

return 0;
@@ -1262,9 +1314,17 @@ static int brcm_pcie_resume(struct device *dev)
if (ret)
return ret;

+ if (pcie->sr) {
+ ret = regulator_bulk_enable(pcie->sr->num_supplies, pcie->sr->supplies);
+ if (ret) {
+ dev_err(dev, "Could not turn on regulators\n");
+ goto err_disable_clk;
+ }
+ }
+
ret = reset_control_reset(pcie->rescal);
if (ret)
- goto err_disable_clk;
+ goto err_regulator;

ret = brcm_phy_start(pcie);
if (ret)
@@ -1296,6 +1356,8 @@ static int brcm_pcie_resume(struct device *dev)

err_reset:
reset_control_rearm(pcie->rescal);
+err_regulator:
+ regulator_bulk_disable(pcie->sr->num_supplies, pcie->sr->supplies);
err_disable_clk:
clk_disable_unprepare(pcie->clk);
return ret;
@@ -1425,7 +1487,17 @@ static int brcm_pcie_probe(struct platform_device *pdev)

platform_set_drvdata(pdev, pcie);

- return pci_host_probe(bridge);
+ ret = pci_host_probe(bridge);
+ if (!ret && !brcm_pcie_link_up(pcie))
+ ret = -ENODEV;
+
+ if (ret) {
+ brcm_pcie_remove(pdev);
+ return ret;
+ }
+
+ return 0;
+
fail:
__brcm_pcie_remove(pcie);
return ret;
@@ -1434,8 +1506,8 @@ static int brcm_pcie_probe(struct platform_device *pdev)
MODULE_DEVICE_TABLE(of, brcm_pcie_match);

static const struct dev_pm_ops brcm_pcie_pm_ops = {
- .suspend = brcm_pcie_suspend,
- .resume = brcm_pcie_resume,
+ .suspend_noirq = brcm_pcie_suspend,
+ .resume_noirq = brcm_pcie_resume,
};

static struct platform_driver brcm_pcie_driver = {
--
2.17.1


2022-01-06 16:04:05

by Jim Quinlan

[permalink] [raw]
Subject: [PATCH v11 7/7] PCI: brcmstb: Do not turn off WOL regulators on suspend

If any downstream device can be awoken do not turn off
the regulators as the device will need them on.

Signed-off-by: Jim Quinlan <[email protected]>
---
drivers/pci/controller/pcie-brcmstb.c | 53 ++++++++++++++++++++++-----
1 file changed, 44 insertions(+), 9 deletions(-)

diff --git a/drivers/pci/controller/pcie-brcmstb.c b/drivers/pci/controller/pcie-brcmstb.c
index 3c1b2fefe7e1..8a3321314b74 100644
--- a/drivers/pci/controller/pcie-brcmstb.c
+++ b/drivers/pci/controller/pcie-brcmstb.c
@@ -308,6 +308,7 @@ struct brcm_pcie {
void (*bridge_sw_init_set)(struct brcm_pcie *pcie, u32 val);
bool refusal_mode;
struct subdev_regulators *sr;
+ bool ep_wakeup_capable;
};

/*
@@ -1269,9 +1270,21 @@ static void brcm_pcie_turn_off(struct brcm_pcie *pcie)
pcie->bridge_sw_init_set(pcie, 1);
}

+static int pci_dev_may_wakeup(struct pci_dev *dev, void *data)
+{
+ bool *ret = data;
+
+ if (device_may_wakeup(&dev->dev)) {
+ *ret = true;
+ dev_info(&dev->dev, "disable cancelled for wake-up device\n");
+ }
+ return (int) *ret;
+}
+
static int brcm_pcie_suspend(struct device *dev)
{
struct brcm_pcie *pcie = dev_get_drvdata(dev);
+ struct pci_host_bridge *bridge = pci_host_bridge_from_priv(pcie);
int ret;

brcm_pcie_turn_off(pcie);
@@ -1290,11 +1303,22 @@ static int brcm_pcie_suspend(struct device *dev)
}

if (pcie->sr) {
- ret = regulator_bulk_disable(pcie->sr->num_supplies, pcie->sr->supplies);
- if (ret) {
- dev_err(dev, "Could not turn off regulators\n");
- reset_control_reset(pcie->rescal);
- return ret;
+ /*
+ * Now turn off the regulators, but if at least one
+ * downstream device is enabled as a wake-up source, do not
+ * turn off regulators.
+ */
+ pcie->ep_wakeup_capable = false;
+ pci_walk_bus(bridge->bus, pci_dev_may_wakeup,
+ &pcie->ep_wakeup_capable);
+ if (!pcie->ep_wakeup_capable) {
+ ret = regulator_bulk_disable(pcie->sr->num_supplies,
+ pcie->sr->supplies);
+ if (ret) {
+ dev_err(dev, "Could not turn off regulators\n");
+ reset_control_reset(pcie->rescal);
+ return ret;
+ }
}
}
clk_disable_unprepare(pcie->clk);
@@ -1315,10 +1339,21 @@ static int brcm_pcie_resume(struct device *dev)
return ret;

if (pcie->sr) {
- ret = regulator_bulk_enable(pcie->sr->num_supplies, pcie->sr->supplies);
- if (ret) {
- dev_err(dev, "Could not turn on regulators\n");
- goto err_disable_clk;
+ if (pcie->ep_wakeup_capable) {
+ /*
+ * We are resuming from a suspend. In the suspend we
+ * did not disable the power supplies, so there is
+ * no need to enable them (and falsely increase their
+ * usage count).
+ */
+ pcie->ep_wakeup_capable = false;
+ } else {
+ ret = regulator_bulk_enable(pcie->sr->num_supplies,
+ pcie->sr->supplies);
+ if (ret) {
+ dev_err(dev, "Could not turn on regulators\n");
+ goto err_disable_clk;
+ }
}
}

--
2.17.1


2022-01-06 16:09:43

by Florian Fainelli

[permalink] [raw]
Subject: Re: [PATCH v11 3/7] dt-bindings: PCI: Add bindings for Brcmstb EP voltage regulators

On 1/6/22 8:03 AM, Jim Quinlan wrote:
> Add bindings for Brcmstb EP voltage regulators. A new mechanism is to be
> added to the Linux PCI subsystem that will allocate and turn on/off
> regulators. These are standard regulators -- vpcie12v, vpcie3v3, and
> vpcie3v3aux -- placed in the DT in the bridge node under the host bridge
> device.
>
> The use of a regulator property in the pcie EP subnode such as
> "vpcie12v-supply" depends on a pending pullreq to the pci-bus.yaml
> file at
>
> https://github.com/devicetree-org/dt-schema/pull/63
>
> Reviewed-by: Rob Herring <[email protected]>
> Signed-off-by: Jim Quinlan <[email protected]>

Acked-by: Florian Fainelli <[email protected]>
--
Florian

2022-01-07 10:04:23

by Lorenzo Pieralisi

[permalink] [raw]
Subject: Re: [PATCH v11 0/7] PCI: brcmstb: root port turns on sub-device power

On Thu, 6 Jan 2022 11:03:23 -0500, Jim Quinlan wrote:
> v11 -- Only one commit has changed from v10: "PCI: brcmstb: Add mechanism
> to turn on subdev regulators". Lorenzo requested that this commit's
> code be moved from pci/pci.h and pci/bus.c to
> pci/controller/pcie-brcmstb.c as a stopgap measure until we figure
> out a better way of offering this feature for all. This will most
> likely involve Pali's upcoming changes to refactor PCI host probe
> code.
>
> [...]

Applied to pci/brcmstb, thanks!

[1/7] PCI: brcmstb: Fix function return value handling
https://git.kernel.org/lpieralisi/pci/c/ec16b36f89
[2/7] dt-bindings: PCI: Correct brcmstb interrupts, interrupt-map.
https://git.kernel.org/lpieralisi/pci/c/7d0ad0c8cc
[3/7] dt-bindings: PCI: Add bindings for Brcmstb EP voltage regulators
https://git.kernel.org/lpieralisi/pci/c/9a617ffe0d
[4/7] PCI: brcmstb: Split brcm_pcie_setup() into two funcs
https://git.kernel.org/lpieralisi/pci/c/2d8bb00647
[5/7] PCI: brcmstb: Add mechanism to turn on subdev regulators
https://git.kernel.org/lpieralisi/pci/c/c2c8468565
[6/7] PCI: brcmstb: Add control of subdevice voltage regulators
https://git.kernel.org/lpieralisi/pci/c/31d8fedc28
[7/7] PCI: brcmstb: Do not turn off WOL regulators on suspend
https://git.kernel.org/lpieralisi/pci/c/d25f7bee89

Thanks,
Lorenzo