Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20191225005655.1502037-1-martin.blumenstingl@googlemail.com>
 <20191225005655.1502037-2-martin.blumenstingl@googlemail.com> <20191225150845.GA16671@lunn.ch>
In-Reply-To: <20191225150845.GA16671@lunn.ch>
From:   Martin Blumenstingl <martin.blumenstingl@googlemail.com>
Date:   Wed, 25 Dec 2019 16:33:44 +0100
Message-ID: <CAFBinCA4X1e5_5nBiHmNiB40uJyr9Nm1b2VkF9NqM+wb7-1xmw@mail.gmail.com>
Subject: Re: [PATCH 1/3] net: stmmac: dwmac-meson8b: Fix the RGMII TX delay on
 Meson8b/8m2 SoCs
To:     Andrew Lunn <andrew@lunn.ch>
Cc:     linux-amlogic@lists.infradead.org, netdev@vger.kernel.org,
        davem@davemloft.net, khilman@baylibre.com,
        linus.luessing@c0d3.blue, balbes-150@yandex.ru,
        linux-kernel@vger.kernel.org, linux-arm-kernel@lists.infradead.org,
        ingrassia@epigenesys.com, jbrunet@baylibre.com
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hi Andrew,

thank you as always for taking a close look at my patches :-)

On Wed, Dec 25, 2019 at 4:08 PM Andrew Lunn <andrew@lunn.ch> wrote:
>
> On Wed, Dec 25, 2019 at 01:56:53AM +0100, Martin Blumenstingl wrote:
> > GXBB and newer SoCs use the fixed FCLK_DIV2 (1GHz) clock as input for
> > the m250_sel clock. Meson8b and Meson8m2 use MPLL2 instead, whose rate
> > can be adjusted at runtime.
> >
> > So far we have been running MPLL2 with ~250MHz (and the internal
> > m250_div with value 1), which worked enough that we could transfer data
> > with an TX delay of 4ns. Unfortunately there is high packet loss with
> > an RGMII PHY when transferring data (receiving data works fine though).
> > Odroid-C1's u-boot is running with a TX delay of only 2ns as well as
> > the internal m250_div set to 2 - no lost (TX) packets can be observed
> > with that setting in u-boot.
> >
> > Manual testing has shown that the TX packet loss goes away when using
> > the following settings in Linux:
> > - MPLL2 clock set to ~500MHz
> > - m250_div set to 2
> > - TX delay set to 2ns
>
> Hi Martin
>
> The delay will depend on the PHY, the value of phy-mode, and the PCB
> layout.
>
> https://ethernetfmc.com/rgmii-interface-timing-considerations/
>
> RGMII requires a delay of 2ns between the data and the clock
> signal. There are at least three ways this can happen.
>
> 1) The MAC adds the delay
>
> 2) The PCB adds the delay by making the clock line longer than the
> data line.
>
> 3) The PHY adds the delay.
>
> In linux you configure this using the phy-mode in DT.
>
>       # RX and TX delays are added by the MAC when required
>       - rgmii
>
>       # RGMII with internal RX and TX delays provided by the PHY,
>       # the MAC should not add the RX or TX delays in this case
>       - rgmii-id
>
>       # RGMII with internal RX delay provided by the PHY, the MAC
>       # should not add an RX delay in this case
>       - rgmii-rxid
>
>       # RGMII with internal TX delay provided by the PHY, the MAC
>       # should not add an TX delay in this case
>       - rgmii-txid
>
> So ideally, you want the MAC to add no delay at all, and then use the
> correct phy-mode so the PHY adds the correct delay. This gives you the
> most flexibility in terms of PHY and PCB design. This does however
> require that the PHY implements the delay, which not all do.
these boards (with RGMII PHY) that I am aware of are using an RTL8211F
PHY which implements a 2ns PHY TX delay
however, the 3.10 vendor kernel also supports Micrel RGMII (and RMII)
PHYs where I don't know if they implement a (configurable) TX delay.

> Looking at patches 2 and 3, the phy-mode is set to rgmii. What you
> might actually need to do is set this to rgmii-txid, or maybe
> rgmii-id, once you have the MAC not inserting any delay.
please let us split this discussion:
1) I believe that this patch is still correct and required whenever
the MAC *has to* generate the TX delay (one use-case could be the
Micrel PHYs I mentioned above)
2) the correct phy-mode and where the TX delay is being generated. I
have tried "rgmii-txid" on my own Odroid-C1 and it's working fine
there. however, it's the only board with RGMII PHY that I have from
this generation of SoCs (and other testers are typically rare for this
platform, because it's an older SoC). so my idea was to use the same
settings as the 3.10 vendor kernel because these seem to be the "known
working" ones.

what do you think about 2)? my main concern is that this *could* break
Ethernet on other people's boards.
on the other hand I have no idea how likely that actually is.


Martin