The Ethernet TX performance has been historically bad on Meson8b and
Meson8m2 SoCs because high packet loss was seen. I found out that this
was related (yet again) to the RGMII TX delay configuration.
In the process of discussing the big picture (and not just a single
patch) [0] with Andrew I discovered that the IP block behind the
dwmac-meson8b driver actually seems to support the configuration of the
RGMII RX delay (at least on the Meson8b SoC generation).
Since I sent the last RFC I got additional documentation from Jianxin
(many thanks!). Also I have discovered some more interesting details:
- Meson8b Odroid-C1 requires an RX delay (by either the PHY or the MAC)
Based on the vendor u-boot code (not upstream) I assume that it will
be the same for all Meson8b and Meson8m2 boards
- Khadas VIM2 seems to have the RX delay built into the PCB trace
length. When I enable the RX delay on the PHY or MAC I can't get any
data through. I expect that we will have the same situation on all
GXBB, GXM, AXG, G12A, G12B and SM1 boards
Changes since RFC v1 at [1]:
- add support for the timing adjustment clock input (dt-bindings and
in the driver) thanks to the input from the unnamed Ethernet engineer
at Amlogic. This is the missing link between the fclk_div2 clock and
the Ethernet controller on Meson8b (no traffic would flow if that
clock was disabled)
- add support fot the amlogic,rx-delay-ns property. The only supported
values so far are 0ns and 2ns. The registers seem to allow more
precise timing adjustments, but I could not make that work so far.
- add more register documentation (for the new RX delay bits) and
unified the placement of existing register documentation. Again,
thanks to Jianxin and the unnamed Ethernet engineer at Amlogic
- DO NOT MERGE: .dts patches to show the conversion of the Meson8b
and Meson8m2 boards to "rgmii-id". I didn't have time for all arm64
patches yet, but these will switch to phy-mode = "rgmii-txid" with
amlogic,rx-delay-ns = <0> (because the delay seems to be provided by
the PCB trace length).
[0] https://patchwork.kernel.org/patch/11309891/
[1] https://patchwork.kernel.org/cover/11310719/
Martin Blumenstingl (11):
dt-bindings: net: meson-dwmac: Add the amlogic,rx-delay-ns property
dt-bindings: net: dwmac-meson: Document the "timing-adjustment" clock
net: stmmac: dwmac-meson8b: use FIELD_PREP instead of open-coding it
net: stmmac: dwmac-meson8b: Move the documentation for the TX delay
net: stmmac: dwmac-meson8b: Add the PRG_ETH0_ADJ_* bits
net: stmmac: dwmac-meson8b: Fetch the "timing-adjustment" clock
net: stmmac: dwmac-meson8b: Make the clock enabling code re-usable
net: stmmac: dwmac-meson8b: add support for the RX delay configuration
arm64: dts: amlogic: Add the Ethernet "timing-adjustment" clock
ARM: dts: meson: Add the Ethernet "timing-adjustment" clock
ARM: dts: meson: Switch existing boards with RGMII PHY to "rgmii-id"
.../bindings/net/amlogic,meson-dwmac.yaml | 23 ++-
arch/arm/boot/dts/meson8b-odroidc1.dts | 3 +-
arch/arm/boot/dts/meson8b.dtsi | 5 +-
arch/arm/boot/dts/meson8m2-mxiii-plus.dts | 4 +-
arch/arm/boot/dts/meson8m2.dtsi | 5 +-
arch/arm64/boot/dts/amlogic/meson-axg.dtsi | 6 +-
.../boot/dts/amlogic/meson-g12-common.dtsi | 6 +-
arch/arm64/boot/dts/amlogic/meson-gxbb.dtsi | 5 +-
arch/arm64/boot/dts/amlogic/meson-gxl.dtsi | 5 +-
.../ethernet/stmicro/stmmac/dwmac-meson8b.c | 140 ++++++++++++++----
10 files changed, 150 insertions(+), 52 deletions(-)
--
2.26.2
The PRG_ETHERNET registers have a built-in timing adjustment circuit
which can provide the RX delay in RGMII mode. This is driven by an
external (to this IP, but internal to the SoC) clock input. Fetch this
clock as optional (even though it's there on all supported SoCs) since
we just learned about it and existing .dtbs don't specify it.
Signed-off-by: Martin Blumenstingl <[email protected]>
---
drivers/net/ethernet/stmicro/stmmac/dwmac-meson8b.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/drivers/net/ethernet/stmicro/stmmac/dwmac-meson8b.c b/drivers/net/ethernet/stmicro/stmmac/dwmac-meson8b.c
index 70075628c58e..41f3ef6bea66 100644
--- a/drivers/net/ethernet/stmicro/stmmac/dwmac-meson8b.c
+++ b/drivers/net/ethernet/stmicro/stmmac/dwmac-meson8b.c
@@ -85,6 +85,7 @@ struct meson8b_dwmac {
phy_interface_t phy_mode;
struct clk *rgmii_tx_clk;
u32 tx_delay_ns;
+ struct clk *timing_adj_clk;
};
struct meson8b_dwmac_clk_configs {
@@ -380,6 +381,13 @@ static int meson8b_dwmac_probe(struct platform_device *pdev)
&dwmac->tx_delay_ns))
dwmac->tx_delay_ns = 2;
+ dwmac->timing_adj_clk = devm_clk_get_optional(dwmac->dev,
+ "timing-adjustment");
+ if (IS_ERR(dwmac->timing_adj_clk)) {
+ ret = PTR_ERR(dwmac->timing_adj_clk);
+ goto err_remove_config_dt;
+ }
+
ret = meson8b_init_rgmii_tx_clk(dwmac);
if (ret)
goto err_remove_config_dt;
--
2.26.2
The PRG_ETHERNET registers on Meson8b and newer SoCs can add an RX
delay. Add a property with the known supported values so it can be
configured according to the board layout.
Signed-off-by: Martin Blumenstingl <[email protected]>
---
.../bindings/net/amlogic,meson-dwmac.yaml | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml b/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
index ae91aa9d8616..8d851f59d9f2 100644
--- a/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
+++ b/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
@@ -67,6 +67,19 @@ allOf:
PHY and MAC are adding a delay).
Any configuration is ignored when the phy-mode is set to "rmii".
+ amlogic,rx-delay-ns:
+ $ref: /schemas/types.yaml#definitions/uint32
+ enum:
+ - 0
+ - 2
+ description:
+ The internal RGMII RX clock delay (provided by this IP block) in
+ nanoseconds. When phy-mode is set to "rgmii" then the RX delay
+ should be explicitly configured. When not configured a fallback of
+ 0ns is used. When the phy-mode is set to either "rgmii-id" or
+ "rgmii-rxid" the RX clock delay is already provided by the PHY.
+ Any configuration is ignored when the phy-mode is set to "rmii".
+
properties:
compatible:
additionalItems: true
--
2.26.2
> - Khadas VIM2 seems to have the RX delay built into the PCB trace
> length. When I enable the RX delay on the PHY or MAC I can't get any
> data through. I expect that we will have the same situation on all
> GXBB, GXM, AXG, G12A, G12B and SM1 boards
Hi Martin
Can you actually see this on the PCB? The other possibility is that
the bootloader is configuring something, which is not getting
overridden when linux starts up.
Andrew
Hi Andrew,
On Wed, Apr 29, 2020 at 11:29 PM Andrew Lunn <[email protected]> wrote:
>
> > - Khadas VIM2 seems to have the RX delay built into the PCB trace
> > length. When I enable the RX delay on the PHY or MAC I can't get any
> > data through. I expect that we will have the same situation on all
> > GXBB, GXM, AXG, G12A, G12B and SM1 boards
>
> Hi Martin
>
> Can you actually see this on the PCB? The other possibility is that
> the bootloader is configuring something, which is not getting
> overridden when linux starts up.
at least it doesn't jump straight into my eye.
I checked in u-boot and Linux, and for both the RX delay is disabled
in the PHY as well as in the MAC.
The schematics of the Khadas VIM2 also show the the RX delay in the
PHY is turned off by pin-strapping, see page 7 on the right: [0]
It's the same for the Khadas VIM3 schematics, also on page 7: [1]
There are also high resolution images of the Khadas VIM3 online so you
can look at it yourself (I couldn't find any for the Khadas VIM2 which
is what I have): [2]
I agree that we need to get an answer to the RX delay question on the
arm64 SoCs.
If there's no way to find out from the existing resources then I can
contact Khadas and ask them about the PCB trace length on VIM2, VIM3
and VIM3L (these are the ones with RGMII PHYs).
For the older SoCs the RX delay has to be provided by either the MAC
or the PHY and right now we're not configuring it.
We cannot simply enable the RX delay at the PHY level because the
bootloader enables it in the MAC (so we have to turn it off there).
So it would be great if you could still review this series.
Martin
[0] https://dl.khadas.com/Hardware/VIM2/Schematic/VIM2_V12_Sch.pdf
[1] https://dl.khadas.com/Hardware/VIM3/Schematic/VIM3_V12_Sch.pdf
[2] https://forum.khadas.com/t/khadas-vim3-is-launching-on-24-june/4103
On Wed, Apr 29, 2020 at 10:16:34PM +0200, Martin Blumenstingl wrote:
> The PRG_ETHERNET registers on Meson8b and newer SoCs can add an RX
> delay. Add a property with the known supported values so it can be
> configured according to the board layout.
>
> Signed-off-by: Martin Blumenstingl <[email protected]>
Reviewed-by: Andrew Lunn <[email protected]>
Andrew
On Wed, Apr 29, 2020 at 10:16:39PM +0200, Martin Blumenstingl wrote:
> The PRG_ETHERNET registers have a built-in timing adjustment circuit
> which can provide the RX delay in RGMII mode. This is driven by an
> external (to this IP, but internal to the SoC) clock input. Fetch this
> clock as optional (even though it's there on all supported SoCs) since
> we just learned about it and existing .dtbs don't specify it.
>
> Signed-off-by: Martin Blumenstingl <[email protected]>
Reviewed-by: Andrew Lunn <[email protected]>
Andrew
On Wed, Apr 29, 2020 at 10:16:34PM +0200, Martin Blumenstingl wrote:
> The PRG_ETHERNET registers on Meson8b and newer SoCs can add an RX
> delay. Add a property with the known supported values so it can be
> configured according to the board layout.
>
> Signed-off-by: Martin Blumenstingl <[email protected]>
> ---
> .../bindings/net/amlogic,meson-dwmac.yaml | 13 +++++++++++++
> 1 file changed, 13 insertions(+)
>
> diff --git a/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml b/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
> index ae91aa9d8616..8d851f59d9f2 100644
> --- a/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
> +++ b/Documentation/devicetree/bindings/net/amlogic,meson-dwmac.yaml
> @@ -67,6 +67,19 @@ allOf:
> PHY and MAC are adding a delay).
> Any configuration is ignored when the phy-mode is set to "rmii".
>
> + amlogic,rx-delay-ns:
> + $ref: /schemas/types.yaml#definitions/uint32
Don't need to define the type when in standard units.
> + enum:
> + - 0
> + - 2
> + description:
> + The internal RGMII RX clock delay (provided by this IP block) in
> + nanoseconds. When phy-mode is set to "rgmii" then the RX delay
> + should be explicitly configured. When not configured a fallback of
> + 0ns is used. When the phy-mode is set to either "rgmii-id" or
'default: 0' expresses this.
> + "rgmii-rxid" the RX clock delay is already provided by the PHY.
> + Any configuration is ignored when the phy-mode is set to "rmii".
> +
> properties:
> compatible:
> additionalItems: true
> --
> 2.26.2
>