While the Apple PCIe driver works correctly when directly booted
from the firmware, it fails to initialise when the kernel is booted
from a bootloader using PCIe such as u-boot.
That's beacuse we're missing a proper reset of the port (we only
clear the reset, but never assert it).
The PCIe spec requirements are two-fold:
- #PERST must be asserted before setting up the clocks, and
stay asserted for at least 100us (Tperst-clk).
- Once #PERST is deasserted, the OS must wait for at least 100ms
"from the end of a Conventional Reset" before we can start talking
to the devices
Implementing this results in a booting system.
Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")
Signed-off-by: Marc Zyngier <[email protected]>
Cc: Alyssa Rosenzweig <[email protected]>
Cc: Lorenzo Pieralisi <[email protected]>
Cc: Bjorn Helgaas <[email protected]>
Cc: Pali Rohár <[email protected]>
---
drivers/pci/controller/pcie-apple.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c
index 1bf4d75b61be..957960a733c4 100644
--- a/drivers/pci/controller/pcie-apple.c
+++ b/drivers/pci/controller/pcie-apple.c
@@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
+ /* Engage #PERST before setting up the clock */
+ gpiod_set_value(reset, 0);
+
ret = apple_pcie_setup_refclk(pcie, port);
if (ret < 0)
return ret;
+ /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
+ usleep_range(100, 200);
+
+ /* Deassert #PERST */
rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
gpiod_set_value(reset, 1);
+ /* Wait for 100ms after #PERST deassertion (PCIe r2.0, 6.6.1) */
+ msleep(100);
+
ret = readl_relaxed_poll_timeout(port->base + PORT_STATUS, stat,
stat & PORT_STATUS_READY, 100, 250000);
if (ret < 0) {
--
2.30.2
On Monday 22 November 2021 10:41:56 Marc Zyngier wrote:
> While the Apple PCIe driver works correctly when directly booted
> from the firmware, it fails to initialise when the kernel is booted
> from a bootloader using PCIe such as u-boot.
>
> That's beacuse we're missing a proper reset of the port (we only
> clear the reset, but never assert it).
>
> The PCIe spec requirements are two-fold:
>
> - #PERST must be asserted before setting up the clocks, and
> stay asserted for at least 100us (Tperst-clk).
>
> - Once #PERST is deasserted, the OS must wait for at least 100ms
> "from the end of a Conventional Reset" before we can start talking
> to the devices
>
> Implementing this results in a booting system.
>
> Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")
> Signed-off-by: Marc Zyngier <[email protected]>
> Cc: Alyssa Rosenzweig <[email protected]>
> Cc: Lorenzo Pieralisi <[email protected]>
> Cc: Bjorn Helgaas <[email protected]>
> Cc: Pali Rohár <[email protected]>
Looks good, but see comment below.
Acked-by: Pali Rohár <[email protected]>
> ---
> drivers/pci/controller/pcie-apple.c | 10 ++++++++++
> 1 file changed, 10 insertions(+)
>
> diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c
> index 1bf4d75b61be..957960a733c4 100644
> --- a/drivers/pci/controller/pcie-apple.c
> +++ b/drivers/pci/controller/pcie-apple.c
> @@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
>
> rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
>
> + /* Engage #PERST before setting up the clock */
> + gpiod_set_value(reset, 0);
> +
> ret = apple_pcie_setup_refclk(pcie, port);
> if (ret < 0)
> return ret;
>
> + /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
> + usleep_range(100, 200);
> +
> + /* Deassert #PERST */
> rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
> gpiod_set_value(reset, 1);
+ Luca
Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting
means to really set value to 1.
But there was a discussion that de-asserting should be done by call:
gpiod_set_value(reset, 0);
https://lore.kernel.org/linux-pci/[email protected]/
Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0)
for de-asserting, like in other drivers?
>
> + /* Wait for 100ms after #PERST deassertion (PCIe r2.0, 6.6.1) */
> + msleep(100);
> +
> ret = readl_relaxed_poll_timeout(port->base + PORT_STATUS, stat,
> stat & PORT_STATUS_READY, 100, 250000);
> if (ret < 0) {
> --
> 2.30.2
>
On Mon, 22 Nov 2021 12:03:47 +0000,
Pali Rohár <[email protected]> wrote:
>
> On Monday 22 November 2021 10:41:56 Marc Zyngier wrote:
> > While the Apple PCIe driver works correctly when directly booted
> > from the firmware, it fails to initialise when the kernel is booted
> > from a bootloader using PCIe such as u-boot.
> >
> > That's beacuse we're missing a proper reset of the port (we only
> > clear the reset, but never assert it).
> >
> > The PCIe spec requirements are two-fold:
> >
> > - #PERST must be asserted before setting up the clocks, and
> > stay asserted for at least 100us (Tperst-clk).
> >
> > - Once #PERST is deasserted, the OS must wait for at least 100ms
> > "from the end of a Conventional Reset" before we can start talking
> > to the devices
> >
> > Implementing this results in a booting system.
> >
> > Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")
> > Signed-off-by: Marc Zyngier <[email protected]>
> > Cc: Alyssa Rosenzweig <[email protected]>
> > Cc: Lorenzo Pieralisi <[email protected]>
> > Cc: Bjorn Helgaas <[email protected]>
> > Cc: Pali Rohár <[email protected]>
>
> Looks good, but see comment below.
>
> Acked-by: Pali Rohár <[email protected]>
Thanks for that.
>
> > ---
> > drivers/pci/controller/pcie-apple.c | 10 ++++++++++
> > 1 file changed, 10 insertions(+)
> >
> > diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c
> > index 1bf4d75b61be..957960a733c4 100644
> > --- a/drivers/pci/controller/pcie-apple.c
> > +++ b/drivers/pci/controller/pcie-apple.c
> > @@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
> >
> > rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
> >
> > + /* Engage #PERST before setting up the clock */
> > + gpiod_set_value(reset, 0);
> > +
> > ret = apple_pcie_setup_refclk(pcie, port);
> > if (ret < 0)
> > return ret;
> >
> > + /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
> > + usleep_range(100, 200);
> > +
> > + /* Deassert #PERST */
> > rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
> > gpiod_set_value(reset, 1);
>
> + Luca
>
> Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting
> means to really set value to 1.
>
> But there was a discussion that de-asserting should be done by call:
> gpiod_set_value(reset, 0);
>
> https://lore.kernel.org/linux-pci/[email protected]/
>
> Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0)
> for de-asserting, like in other drivers?
I guess it depends whether you care about the assertion or the signal
itself. I think we may have a bug in the way the GPIOs are handled at
the moment, as it makes no difference whether I register the GPIO are
active high or active low...
I guess that will be yet another thing to debug, but in the meantime
we have a reliable reset.
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
Hi Marc,
On 22/11/21 15:43, Marc Zyngier wrote:
> On Mon, 22 Nov 2021 12:03:47 +0000,
> Pali Rohár <[email protected]> wrote:
>>
>> On Monday 22 November 2021 10:41:56 Marc Zyngier wrote:
>>> While the Apple PCIe driver works correctly when directly booted
>>> from the firmware, it fails to initialise when the kernel is booted
>>> from a bootloader using PCIe such as u-boot.
>>>
>>> That's beacuse we're missing a proper reset of the port (we only
>>> clear the reset, but never assert it).
>>>
>>> The PCIe spec requirements are two-fold:
>>>
>>> - #PERST must be asserted before setting up the clocks, and
>>> stay asserted for at least 100us (Tperst-clk).
>>>
>>> - Once #PERST is deasserted, the OS must wait for at least 100ms
>>> "from the end of a Conventional Reset" before we can start talking
>>> to the devices
>>>
>>> Implementing this results in a booting system.
>>>
>>> Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")
>>> Signed-off-by: Marc Zyngier <[email protected]>
>>> Cc: Alyssa Rosenzweig <[email protected]>
>>> Cc: Lorenzo Pieralisi <[email protected]>
>>> Cc: Bjorn Helgaas <[email protected]>
>>> Cc: Pali Rohár <[email protected]>
>>
>> Looks good, but see comment below.
>>
>> Acked-by: Pali Rohár <[email protected]>
>
> Thanks for that.
>
>>
>>> ---
>>> drivers/pci/controller/pcie-apple.c | 10 ++++++++++
>>> 1 file changed, 10 insertions(+)
>>>
>>> diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c
>>> index 1bf4d75b61be..957960a733c4 100644
>>> --- a/drivers/pci/controller/pcie-apple.c
>>> +++ b/drivers/pci/controller/pcie-apple.c
>>> @@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
>>>
>>> rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
>>>
>>> + /* Engage #PERST before setting up the clock */
>>> + gpiod_set_value(reset, 0);
>>> +
>>> ret = apple_pcie_setup_refclk(pcie, port);
>>> if (ret < 0)
>>> return ret;
>>>
>>> + /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
>>> + usleep_range(100, 200);
>>> +
>>> + /* Deassert #PERST */
>>> rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
>>> gpiod_set_value(reset, 1);
>>
>> + Luca
>>
>> Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting
>> means to really set value to 1.
>>
>> But there was a discussion that de-asserting should be done by call:
>> gpiod_set_value(reset, 0);
>>
>> https://lore.kernel.org/linux-pci/[email protected]/
>>
>> Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0)
>> for de-asserting, like in other drivers?
I agree it should be done right from the beginning since this is a new
driver. Fixing it later is a painful process.
> I guess it depends whether you care about the assertion or the signal
> itself. I think we may have a bug in the way the GPIOs are handled at
> the moment, as it makes no difference whether I register the GPIO are
> active high or active low...
>
> I guess that will be yet another thing to debug, but in the meantime
> we have a reliable reset.
Strange, in my case the "active low" pin polarity is correctly picked up
from device tree by the gpiolib code, thus using gpio_set_value(gpiod,
1) asserts the pin as it should, resulting in an electrically low pin.
--
Luca
> Date: Mon, 22 Nov 2021 14:43:37 +0000
> From: Marc Zyngier <[email protected]>
>
> On Mon, 22 Nov 2021 12:03:47 +0000,
> Pali Rohár <[email protected]> wrote:
> >
> > On Monday 22 November 2021 10:41:56 Marc Zyngier wrote:
> > > While the Apple PCIe driver works correctly when directly booted
> > > from the firmware, it fails to initialise when the kernel is booted
> > > from a bootloader using PCIe such as u-boot.
> > >
> > > That's beacuse we're missing a proper reset of the port (we only
> > > clear the reset, but never assert it).
> > >
> > > The PCIe spec requirements are two-fold:
> > >
> > > - #PERST must be asserted before setting up the clocks, and
> > > stay asserted for at least 100us (Tperst-clk).
> > >
> > > - Once #PERST is deasserted, the OS must wait for at least 100ms
> > > "from the end of a Conventional Reset" before we can start talking
> > > to the devices
> > >
> > > Implementing this results in a booting system.
> > >
> > > Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")
> > > Signed-off-by: Marc Zyngier <[email protected]>
> > > Cc: Alyssa Rosenzweig <[email protected]>
> > > Cc: Lorenzo Pieralisi <[email protected]>
> > > Cc: Bjorn Helgaas <[email protected]>
> > > Cc: Pali Rohár <[email protected]>
> >
> > Looks good, but see comment below.
> >
> > Acked-by: Pali Rohár <[email protected]>
>
> Thanks for that.
>
> >
> > > ---
> > > drivers/pci/controller/pcie-apple.c | 10 ++++++++++
> > > 1 file changed, 10 insertions(+)
> > >
> > > diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c
> > > index 1bf4d75b61be..957960a733c4 100644
> > > --- a/drivers/pci/controller/pcie-apple.c
> > > +++ b/drivers/pci/controller/pcie-apple.c
> > > @@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
> > >
> > > rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
> > >
> > > + /* Engage #PERST before setting up the clock */
> > > + gpiod_set_value(reset, 0);
> > > +
> > > ret = apple_pcie_setup_refclk(pcie, port);
> > > if (ret < 0)
> > > return ret;
> > >
> > > + /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
> > > + usleep_range(100, 200);
> > > +
> > > + /* Deassert #PERST */
> > > rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
> > > gpiod_set_value(reset, 1);
> >
> > + Luca
> >
> > Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting
> > means to really set value to 1.
> >
> > But there was a discussion that de-asserting should be done by call:
> > gpiod_set_value(reset, 0);
> >
> > https://lore.kernel.org/linux-pci/[email protected]/
> >
> > Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0)
> > for de-asserting, like in other drivers?
>
> I guess it depends whether you care about the assertion or the signal
> itself. I think we may have a bug in the way the GPIOs are handled at
> the moment, as it makes no difference whether I register the GPIO are
> active high or active low...
That's unfortunate. But maybe that's an opportunity to fix the
devicetree to use GPIO_ACTIVE_LOW for these GPIOs?
Luca,
On Mon, 22 Nov 2021 21:32:15 +0000,
Luca Ceresoli <[email protected]> wrote:
>
> >> Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting
> >> means to really set value to 1.
> >>
> >> But there was a discussion that de-asserting should be done by call:
> >> gpiod_set_value(reset, 0);
> >>
> >> https://lore.kernel.org/linux-pci/[email protected]/
> >>
> >> Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0)
> >> for de-asserting, like in other drivers?
>
> I agree it should be done right from the beginning since this is a new
> driver. Fixing it later is a painful process.
No more painful than anything else. At this stage, using a positive or
negative polarity is immaterial, as there is no core infrastructure
making any use of this behaviour (every single driver must reinvent
its own square wheel). If such an infrastructure existed, that'd
indeed be a requirement. For now, this is merely a convention.
> > I guess it depends whether you care about the assertion or the signal
> > itself. I think we may have a bug in the way the GPIOs are handled at
> > the moment, as it makes no difference whether I register the GPIO are
> > active high or active low...
> >
> > I guess that will be yet another thing to debug, but in the meantime
> > we have a reliable reset.
>
> Strange, in my case the "active low" pin polarity is correctly picked up
> from device tree by the gpiolib code, thus using gpio_set_value(gpiod,
> 1) asserts the pin as it should, resulting in an electrically low pin.
As I said, this looks like a bug, probably in the M1 DT. I'll try to
look into it when I get the time.
M.
--
Without deviation from the norm, progress is not possible.
On Mon, 22 Nov 2021 21:50:48 +0000,
Mark Kettenis <[email protected]> wrote:
>
> > Date: Mon, 22 Nov 2021 14:43:37 +0000
> > From: Marc Zyngier <[email protected]>
> >
> > On Mon, 22 Nov 2021 12:03:47 +0000,
> > Pali Rohár <[email protected]> wrote:
> > >
> > > On Monday 22 November 2021 10:41:56 Marc Zyngier wrote:
> > > > While the Apple PCIe driver works correctly when directly booted
> > > > from the firmware, it fails to initialise when the kernel is booted
> > > > from a bootloader using PCIe such as u-boot.
> > > >
> > > > That's beacuse we're missing a proper reset of the port (we only
> > > > clear the reset, but never assert it).
> > > >
> > > > The PCIe spec requirements are two-fold:
> > > >
> > > > - #PERST must be asserted before setting up the clocks, and
> > > > stay asserted for at least 100us (Tperst-clk).
> > > >
> > > > - Once #PERST is deasserted, the OS must wait for at least 100ms
> > > > "from the end of a Conventional Reset" before we can start talking
> > > > to the devices
> > > >
> > > > Implementing this results in a booting system.
> > > >
> > > > Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")
> > > > Signed-off-by: Marc Zyngier <[email protected]>
> > > > Cc: Alyssa Rosenzweig <[email protected]>
> > > > Cc: Lorenzo Pieralisi <[email protected]>
> > > > Cc: Bjorn Helgaas <[email protected]>
> > > > Cc: Pali Rohár <[email protected]>
> > >
> > > Looks good, but see comment below.
> > >
> > > Acked-by: Pali Rohár <[email protected]>
> >
> > Thanks for that.
> >
> > >
> > > > ---
> > > > drivers/pci/controller/pcie-apple.c | 10 ++++++++++
> > > > 1 file changed, 10 insertions(+)
> > > >
> > > > diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c
> > > > index 1bf4d75b61be..957960a733c4 100644
> > > > --- a/drivers/pci/controller/pcie-apple.c
> > > > +++ b/drivers/pci/controller/pcie-apple.c
> > > > @@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
> > > >
> > > > rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
> > > >
> > > > + /* Engage #PERST before setting up the clock */
> > > > + gpiod_set_value(reset, 0);
> > > > +
> > > > ret = apple_pcie_setup_refclk(pcie, port);
> > > > if (ret < 0)
> > > > return ret;
> > > >
> > > > + /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
> > > > + usleep_range(100, 200);
> > > > +
> > > > + /* Deassert #PERST */
> > > > rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
> > > > gpiod_set_value(reset, 1);
> > >
> > > + Luca
> > >
> > > Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting
> > > means to really set value to 1.
> > >
> > > But there was a discussion that de-asserting should be done by call:
> > > gpiod_set_value(reset, 0);
> > >
> > > https://lore.kernel.org/linux-pci/[email protected]/
> > >
> > > Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0)
> > > for de-asserting, like in other drivers?
> >
> > I guess it depends whether you care about the assertion or the signal
> > itself. I think we may have a bug in the way the GPIOs are handled at
> > the moment, as it makes no difference whether I register the GPIO are
> > active high or active low...
>
> That's unfortunate. But maybe that's an opportunity to fix the
> devicetree to use GPIO_ACTIVE_LOW for these GPIOs?
Indeed. The following hack does the right thing, and I can then
reverse the polarity of the reset in the Linux driver. Of course, it
breaks u-boot at the same time (and I suspect OpenBSD would be equally
affected).
So if we are going down that road, we may need a flag day where all
the moving parts change. I don't really mind not being able to boot
older kernels, but this goes beyond Linux at this point.
Thanks,
M.
diff --git a/arch/arm64/boot/dts/apple/t8103.dtsi b/arch/arm64/boot/dts/apple/t8103.dtsi
index d2e9afde3729..cad1ab920304 100644
--- a/arch/arm64/boot/dts/apple/t8103.dtsi
+++ b/arch/arm64/boot/dts/apple/t8103.dtsi
@@ -10,6 +10,7 @@
#include <dt-bindings/interrupt-controller/apple-aic.h>
#include <dt-bindings/interrupt-controller/irq.h>
#include <dt-bindings/pinctrl/apple.h>
+#include <dt-bindings/gpio/gpio.h>
/ {
compatible = "apple,t8103", "apple,arm-platform";
@@ -293,7 +294,7 @@ pcie0: pcie@690000000 {
port00: pci@0,0 {
device_type = "pci";
reg = <0x0 0x0 0x0 0x0 0x0>;
- reset-gpios = <&pinctrl_ap 152 0>;
+ reset-gpios = <&pinctrl_ap 152 GPIO_ACTIVE_LOW>;
max-link-speed = <2>;
#address-cells = <3>;
@@ -313,7 +314,7 @@ port00: pci@0,0 {
port01: pci@1,0 {
device_type = "pci";
reg = <0x800 0x0 0x0 0x0 0x0>;
- reset-gpios = <&pinctrl_ap 153 0>;
+ reset-gpios = <&pinctrl_ap 153 GPIO_ACTIVE_LOW>;
max-link-speed = <2>;
#address-cells = <3>;
@@ -333,7 +334,7 @@ port01: pci@1,0 {
port02: pci@2,0 {
device_type = "pci";
reg = <0x1000 0x0 0x0 0x0 0x0>;
- reset-gpios = <&pinctrl_ap 33 0>;
+ reset-gpios = <&pinctrl_ap 33 GPIO_ACTIVE_LOW>;
max-link-speed = <1>;
#address-cells = <3>;
--
Without deviation from the norm, progress is not possible.
On Tuesday 23 November 2021 12:24:10 Marc Zyngier wrote:
> On Mon, 22 Nov 2021 21:50:48 +0000,
> Mark Kettenis <[email protected]> wrote:
> >
> > > Date: Mon, 22 Nov 2021 14:43:37 +0000
> > > From: Marc Zyngier <[email protected]>
> > >
> > > On Mon, 22 Nov 2021 12:03:47 +0000,
> > > Pali Rohár <[email protected]> wrote:
> > > >
> > > > On Monday 22 November 2021 10:41:56 Marc Zyngier wrote:
> > > > > While the Apple PCIe driver works correctly when directly booted
> > > > > from the firmware, it fails to initialise when the kernel is booted
> > > > > from a bootloader using PCIe such as u-boot.
> > > > >
> > > > > That's beacuse we're missing a proper reset of the port (we only
> > > > > clear the reset, but never assert it).
> > > > >
> > > > > The PCIe spec requirements are two-fold:
> > > > >
> > > > > - #PERST must be asserted before setting up the clocks, and
> > > > > stay asserted for at least 100us (Tperst-clk).
> > > > >
> > > > > - Once #PERST is deasserted, the OS must wait for at least 100ms
> > > > > "from the end of a Conventional Reset" before we can start talking
> > > > > to the devices
> > > > >
> > > > > Implementing this results in a booting system.
> > > > >
> > > > > Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")
> > > > > Signed-off-by: Marc Zyngier <[email protected]>
> > > > > Cc: Alyssa Rosenzweig <[email protected]>
> > > > > Cc: Lorenzo Pieralisi <[email protected]>
> > > > > Cc: Bjorn Helgaas <[email protected]>
> > > > > Cc: Pali Rohár <[email protected]>
> > > >
> > > > Looks good, but see comment below.
> > > >
> > > > Acked-by: Pali Rohár <[email protected]>
> > >
> > > Thanks for that.
> > >
> > > >
> > > > > ---
> > > > > drivers/pci/controller/pcie-apple.c | 10 ++++++++++
> > > > > 1 file changed, 10 insertions(+)
> > > > >
> > > > > diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c
> > > > > index 1bf4d75b61be..957960a733c4 100644
> > > > > --- a/drivers/pci/controller/pcie-apple.c
> > > > > +++ b/drivers/pci/controller/pcie-apple.c
> > > > > @@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
> > > > >
> > > > > rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
> > > > >
> > > > > + /* Engage #PERST before setting up the clock */
> > > > > + gpiod_set_value(reset, 0);
> > > > > +
> > > > > ret = apple_pcie_setup_refclk(pcie, port);
> > > > > if (ret < 0)
> > > > > return ret;
> > > > >
> > > > > + /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
> > > > > + usleep_range(100, 200);
> > > > > +
> > > > > + /* Deassert #PERST */
> > > > > rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
> > > > > gpiod_set_value(reset, 1);
> > > >
> > > > + Luca
> > > >
> > > > Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting
> > > > means to really set value to 1.
> > > >
> > > > But there was a discussion that de-asserting should be done by call:
> > > > gpiod_set_value(reset, 0);
> > > >
> > > > https://lore.kernel.org/linux-pci/[email protected]/
> > > >
> > > > Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0)
> > > > for de-asserting, like in other drivers?
> > >
> > > I guess it depends whether you care about the assertion or the signal
> > > itself. I think we may have a bug in the way the GPIOs are handled at
> > > the moment, as it makes no difference whether I register the GPIO are
> > > active high or active low...
> >
> > That's unfortunate. But maybe that's an opportunity to fix the
> > devicetree to use GPIO_ACTIVE_LOW for these GPIOs?
>
> Indeed. The following hack does the right thing, and I can then
> reverse the polarity of the reset in the Linux driver. Of course, it
> breaks u-boot at the same time (and I suspect OpenBSD would be equally
> affected).
U-Boot has its own copy of DTS files, right? So it can be fixed in
U-Boot driver together with U-Boot DTS file at the same time.
> So if we are going down that road, we may need a flag day where all
> the moving parts change. I don't really mind not being able to boot
> older kernels, but this goes beyond Linux at this point.
>
> Thanks,
>
> M.
>
> diff --git a/arch/arm64/boot/dts/apple/t8103.dtsi b/arch/arm64/boot/dts/apple/t8103.dtsi
> index d2e9afde3729..cad1ab920304 100644
> --- a/arch/arm64/boot/dts/apple/t8103.dtsi
> +++ b/arch/arm64/boot/dts/apple/t8103.dtsi
> @@ -10,6 +10,7 @@
> #include <dt-bindings/interrupt-controller/apple-aic.h>
> #include <dt-bindings/interrupt-controller/irq.h>
> #include <dt-bindings/pinctrl/apple.h>
> +#include <dt-bindings/gpio/gpio.h>
>
> / {
> compatible = "apple,t8103", "apple,arm-platform";
> @@ -293,7 +294,7 @@ pcie0: pcie@690000000 {
> port00: pci@0,0 {
> device_type = "pci";
> reg = <0x0 0x0 0x0 0x0 0x0>;
> - reset-gpios = <&pinctrl_ap 152 0>;
> + reset-gpios = <&pinctrl_ap 152 GPIO_ACTIVE_LOW>;
> max-link-speed = <2>;
>
> #address-cells = <3>;
> @@ -313,7 +314,7 @@ port00: pci@0,0 {
> port01: pci@1,0 {
> device_type = "pci";
> reg = <0x800 0x0 0x0 0x0 0x0>;
> - reset-gpios = <&pinctrl_ap 153 0>;
> + reset-gpios = <&pinctrl_ap 153 GPIO_ACTIVE_LOW>;
> max-link-speed = <2>;
>
> #address-cells = <3>;
> @@ -333,7 +334,7 @@ port01: pci@1,0 {
> port02: pci@2,0 {
> device_type = "pci";
> reg = <0x1000 0x0 0x0 0x0 0x0>;
> - reset-gpios = <&pinctrl_ap 33 0>;
> + reset-gpios = <&pinctrl_ap 33 GPIO_ACTIVE_LOW>;
> max-link-speed = <1>;
>
> #address-cells = <3>;
>
> --
> Without deviation from the norm, progress is not possible.
> Date: Tue, 23 Nov 2021 08:48:50 +0000
> From: Marc Zyngier <[email protected]>
>
> Luca,
>
> On Mon, 22 Nov 2021 21:32:15 +0000,
> Luca Ceresoli <[email protected]> wrote:
> >
> > >> Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting
> > >> means to really set value to 1.
> > >>
> > >> But there was a discussion that de-asserting should be done by call:
> > >> gpiod_set_value(reset, 0);
> > >>
> > >> https://lore.kernel.org/linux-pci/[email protected]/
> > >>
> > >> Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0)
> > >> for de-asserting, like in other drivers?
> >
> > I agree it should be done right from the beginning since this is a new
> > driver. Fixing it later is a painful process.
>
> No more painful than anything else. At this stage, using a positive or
> negative polarity is immaterial, as there is no core infrastructure
> making any use of this behaviour (every single driver must reinvent
> its own square wheel). If such an infrastructure existed, that'd
> indeed be a requirement. For now, this is merely a convention.
>
> > > I guess it depends whether you care about the assertion or the signal
> > > itself. I think we may have a bug in the way the GPIOs are handled at
> > > the moment, as it makes no difference whether I register the GPIO are
> > > active high or active low...
> > >
> > > I guess that will be yet another thing to debug, but in the meantime
> > > we have a reliable reset.
> >
> > Strange, in my case the "active low" pin polarity is correctly picked up
> > from device tree by the gpiolib code, thus using gpio_set_value(gpiod,
> > 1) asserts the pin as it should, resulting in an electrically low pin.
>
> As I said, this looks like a bug, probably in the M1 DT. I'll try to
> look into it when I get the time.
So the diff below is what the changes look like for U-Boot. The
U-Boot Apple PCIe driver has not been submitted upstream yet, so
making this change is no problem.
diff --git a/arch/arm/dts/t8103-j274.dts b/arch/arm/dts/t8103-j274.dts
index aef1ae29b6..3777337033 100644
--- a/arch/arm/dts/t8103-j274.dts
+++ b/arch/arm/dts/t8103-j274.dts
@@ -65,7 +65,7 @@
device_type = "pci";
reg = <0x0 0x0 0x0 0x0 0x0>;
pwren-gpios = <&smc 13 0>;
- reset-gpios = <&pinctrl_ap 152 0>;
+ reset-gpios = <&pinctrl_ap 152 GPIO_ACTIVE_LOW>;
max-link-speed = <2>;
#address-cells = <3>;
@@ -76,7 +76,7 @@
pci1: pci@1,0 {
device_type = "pci";
reg = <0x800 0x0 0x0 0x0 0x0>;
- reset-gpios = <&pinctrl_ap 153 0>;
+ reset-gpios = <&pinctrl_ap 153 GPIO_ACTIVE_LOW>;
max-link-speed = <2>;
#address-cells = <3>;
@@ -87,7 +87,7 @@
pci2: pci@2,0 {
device_type = "pci";
reg = <0x1000 0x0 0x0 0x0 0x0>;
- reset-gpios = <&pinctrl_ap 33 0>;
+ reset-gpios = <&pinctrl_ap 33 GPIO_ACTIVE_LOW>;
max-link-speed = <1>;
#address-cells = <3>;
diff --git a/drivers/pci/pcie_apple.c b/drivers/pci/pcie_apple.c
index bef6043adb..89eec70d81 100644
--- a/drivers/pci/pcie_apple.c
+++ b/drivers/pci/pcie_apple.c
@@ -210,7 +210,7 @@ static int apple_pcie_setup_port(struct apple_pcie_priv *pcie, unsigned idx)
return 0;
dm_gpio_set_dir_flags(&pcie->perstn[idx], GPIOD_IS_OUT);
- dm_gpio_set_value(&pcie->perstn[idx], 0);
+ dm_gpio_set_value(&pcie->perstn[idx], 1);
rmwl(0, PORT_APPCLK_EN, pcie->base_port[idx] + PORT_APPCLK);
@@ -221,7 +221,7 @@ static int apple_pcie_setup_port(struct apple_pcie_priv *pcie, unsigned idx)
apple_pcie_port_pwren(pcie, idx);
rmwl(0, PORT_PERST_OFF, pcie->base_port[idx] + PORT_PERST);
- dm_gpio_set_value(&pcie->perstn[idx], 1);
+ dm_gpio_set_value(&pcie->perstn[idx], 0);
res = readl_poll_timeout(pcie->base_port[idx] + PORT_STATUS, stat, (stat & PORT_STATUS_READY), 100, 250000);
if (res < 0) {
> Date: Tue, 23 Nov 2021 12:24:10 +0000
> From: Marc Zyngier <[email protected]>
>
> On Mon, 22 Nov 2021 21:50:48 +0000,
> Mark Kettenis <[email protected]> wrote:
> >
> > > Date: Mon, 22 Nov 2021 14:43:37 +0000
> > > From: Marc Zyngier <[email protected]>
> > >
> > > On Mon, 22 Nov 2021 12:03:47 +0000,
> > > Pali Rohár <[email protected]> wrote:
> > > >
> > > > On Monday 22 November 2021 10:41:56 Marc Zyngier wrote:
> > > > > While the Apple PCIe driver works correctly when directly booted
> > > > > from the firmware, it fails to initialise when the kernel is booted
> > > > > from a bootloader using PCIe such as u-boot.
> > > > >
> > > > > That's beacuse we're missing a proper reset of the port (we only
> > > > > clear the reset, but never assert it).
> > > > >
> > > > > The PCIe spec requirements are two-fold:
> > > > >
> > > > > - #PERST must be asserted before setting up the clocks, and
> > > > > stay asserted for at least 100us (Tperst-clk).
> > > > >
> > > > > - Once #PERST is deasserted, the OS must wait for at least 100ms
> > > > > "from the end of a Conventional Reset" before we can start talking
> > > > > to the devices
> > > > >
> > > > > Implementing this results in a booting system.
> > > > >
> > > > > Fixes: 1e33888fbe44 ("PCI: apple: Add initial hardware bring-up")
> > > > > Signed-off-by: Marc Zyngier <[email protected]>
> > > > > Cc: Alyssa Rosenzweig <[email protected]>
> > > > > Cc: Lorenzo Pieralisi <[email protected]>
> > > > > Cc: Bjorn Helgaas <[email protected]>
> > > > > Cc: Pali Rohár <[email protected]>
> > > >
> > > > Looks good, but see comment below.
> > > >
> > > > Acked-by: Pali Rohár <[email protected]>
> > >
> > > Thanks for that.
> > >
> > > >
> > > > > ---
> > > > > drivers/pci/controller/pcie-apple.c | 10 ++++++++++
> > > > > 1 file changed, 10 insertions(+)
> > > > >
> > > > > diff --git a/drivers/pci/controller/pcie-apple.c b/drivers/pci/controller/pcie-apple.c
> > > > > index 1bf4d75b61be..957960a733c4 100644
> > > > > --- a/drivers/pci/controller/pcie-apple.c
> > > > > +++ b/drivers/pci/controller/pcie-apple.c
> > > > > @@ -539,13 +539,23 @@ static int apple_pcie_setup_port(struct apple_pcie *pcie,
> > > > >
> > > > > rmw_set(PORT_APPCLK_EN, port->base + PORT_APPCLK);
> > > > >
> > > > > + /* Engage #PERST before setting up the clock */
> > > > > + gpiod_set_value(reset, 0);
> > > > > +
> > > > > ret = apple_pcie_setup_refclk(pcie, port);
> > > > > if (ret < 0)
> > > > > return ret;
> > > > >
> > > > > + /* The minimal Tperst-clk value is 100us (PCIe CMS r2.0, 2.6.2) */
> > > > > + usleep_range(100, 200);
> > > > > +
> > > > > + /* Deassert #PERST */
> > > > > rmw_set(PORT_PERST_OFF, port->base + PORT_PERST);
> > > > > gpiod_set_value(reset, 1);
> > > >
> > > > + Luca
> > > >
> > > > Just one comment. PERST# (PCIe Reset) is active-low signal. De-asserting
> > > > means to really set value to 1.
> > > >
> > > > But there was a discussion that de-asserting should be done by call:
> > > > gpiod_set_value(reset, 0);
> > > >
> > > > https://lore.kernel.org/linux-pci/[email protected]/
> > > >
> > > > Could we make this new pcie-apple.c driver to use gpiod_set_value(reset, 0)
> > > > for de-asserting, like in other drivers?
> > >
> > > I guess it depends whether you care about the assertion or the signal
> > > itself. I think we may have a bug in the way the GPIOs are handled at
> > > the moment, as it makes no difference whether I register the GPIO are
> > > active high or active low...
> >
> > That's unfortunate. But maybe that's an opportunity to fix the
> > devicetree to use GPIO_ACTIVE_LOW for these GPIOs?
>
> Indeed. The following hack does the right thing, and I can then
> reverse the polarity of the reset in the Linux driver. Of course, it
> breaks u-boot at the same time (and I suspect OpenBSD would be equally
> affected).
As I said in my other reply (that clearly crossed this message) the
U-Boot PCIe driver has not been upstreamed yet. OpenBSD (and also
NetBSD) rely on the configuration done by U-Boot and don't mess with
the reset GPIO at this moment.
> So if we are going down that road, we may need a flag day where all
> the moving parts change. I don't really mind not being able to boot
> older kernels, but this goes beyond Linux at this point.
So I don't think this is needed. The earlier we fix this the better!
> diff --git a/arch/arm64/boot/dts/apple/t8103.dtsi b/arch/arm64/boot/dts/apple/t8103.dtsi
> index d2e9afde3729..cad1ab920304 100644
> --- a/arch/arm64/boot/dts/apple/t8103.dtsi
> +++ b/arch/arm64/boot/dts/apple/t8103.dtsi
> @@ -10,6 +10,7 @@
> #include <dt-bindings/interrupt-controller/apple-aic.h>
> #include <dt-bindings/interrupt-controller/irq.h>
> #include <dt-bindings/pinctrl/apple.h>
> +#include <dt-bindings/gpio/gpio.h>
>
> / {
> compatible = "apple,t8103", "apple,arm-platform";
> @@ -293,7 +294,7 @@ pcie0: pcie@690000000 {
> port00: pci@0,0 {
> device_type = "pci";
> reg = <0x0 0x0 0x0 0x0 0x0>;
> - reset-gpios = <&pinctrl_ap 152 0>;
> + reset-gpios = <&pinctrl_ap 152 GPIO_ACTIVE_LOW>;
> max-link-speed = <2>;
>
> #address-cells = <3>;
> @@ -313,7 +314,7 @@ port00: pci@0,0 {
> port01: pci@1,0 {
> device_type = "pci";
> reg = <0x800 0x0 0x0 0x0 0x0>;
> - reset-gpios = <&pinctrl_ap 153 0>;
> + reset-gpios = <&pinctrl_ap 153 GPIO_ACTIVE_LOW>;
> max-link-speed = <2>;
>
> #address-cells = <3>;
> @@ -333,7 +334,7 @@ port01: pci@1,0 {
> port02: pci@2,0 {
> device_type = "pci";
> reg = <0x1000 0x0 0x0 0x0 0x0>;
> - reset-gpios = <&pinctrl_ap 33 0>;
> + reset-gpios = <&pinctrl_ap 33 GPIO_ACTIVE_LOW>;
> max-link-speed = <1>;
>
> #address-cells = <3>;
>
> --
> Without deviation from the norm, progress is not possible.
>
> _______________________________________________
> linux-arm-kernel mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
On Tue, 23 Nov 2021 12:27:19 +0000,
Pali Rohár <[email protected]> wrote:
>
> U-Boot has its own copy of DTS files, right? So it can be fixed in
> U-Boot driver together with U-Boot DTS file at the same time.
No, there is only a single DTS, solely provided by either u-boot or
the earlier firmware. The version in the kernel tree is purely for
documentation purposes.
M.
--
Without deviation from the norm, progress is not possible.