Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Ioana Ciornei <ioana.ciornei@nxp.com>
To:     Geert Uytterhoeven <geert@linux-m68k.org>
CC:     Andrew Lunn <andrew@lunn.ch>,
        Heiner Kallweit <hkallweit1@gmail.com>,
        "David S . Miller" <davem@davemloft.net>,
        Jakub Kicinski <kuba@kernel.org>,
        Russell King <linux@armlinux.org.uk>,
        Wolfram Sang <wsa+renesas@sang-engineering.com>,
        "netdev@vger.kernel.org" <netdev@vger.kernel.org>,
        "linux-renesas-soc@vger.kernel.org" 
        <linux-renesas-soc@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] [RFC] net: phy: Fix reboot crash if CONFIG_IP_PNP is not
 set
Thread-Topic: [PATCH] [RFC] net: phy: Fix reboot crash if CONFIG_IP_PNP is not
 set
Thread-Index: AQHW4pSHwxyh2hM/e0O6/3UMlPC3maoXjZOAgAAE5QCAAB7HAA==
Date:   Mon, 4 Jan 2021 17:01:13 +0000
Message-ID: <20210104170112.hn6t3kojhifyuaf6@skbuf>
References: <20210104122415.1263541-1-geert+renesas@glider.be>
 <20210104145331.tlwjwbzey5i4vgvp@skbuf>
 <CAMuHMdUVsSuAur1wWkjs7FW5N-36XV9iXA6wmvst59eKoUFDHQ@mail.gmail.com>
In-Reply-To: <CAMuHMdUVsSuAur1wWkjs7FW5N-36XV9iXA6wmvst59eKoUFDHQ@mail.gmail.com>
Accept-Language: en-US
Content-Language: en-US
Content-Type: text/plain; charset="us-ascii"
Content-ID: <E90DEA3B79733A4C9BDAC12066F7321B@eurprd04.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-Network-Message-Id: 009e2b53-c62e-47f6-7d72-08d8b0d25809
X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Jan 2021 17:01:13.8533
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 686ea1d3-bc2b-4c6f-a92c-d99c5c301635
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: HmsTkG8yBweq4ilFscQ/PKWYvft1AvgTHLGImb9H/hth2dFAlaVW1MrJzfFjzC9Fm9oIqDTX71wAi1++s10nBQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: VI1PR0402MB3471
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Mon, Jan 04, 2021 at 04:11:02PM +0100, Geert Uytterhoeven wrote:
> Hi Ioana,
>=20
> On Mon, Jan 4, 2021 at 3:53 PM Ioana Ciornei <ioana.ciornei@nxp.com> wrot=
e:
> > On Mon, Jan 04, 2021 at 01:24:15PM +0100, Geert Uytterhoeven wrote:
> > > Wolfram reports that his R-Car H2-based Lager board can no longer be
> > > rebooted in v5.11-rc1, as it crashes with an imprecise external abort=
.
> > > The issue can be reproduced on other boards (e.g. Koelsch with R-Car
> > > M2-W) too, if CONFIG_IP_PNP is disabled:
> >
> > What kind of PHYs are used on these boards?
>=20
> Micrel KSZ8041RNLI
>=20
> > >     Unhandled fault: imprecise external abort (0x1406) at 0x00000000
> > >     pgd =3D (ptrval)
> > >     [00000000] *pgd=3D422b6835, *pte=3D00000000, *ppte=3D00000000
> > >     Internal error: : 1406 [#1] ARM
> > >     Modules linked in:
> > >     CPU: 0 PID: 1105 Comm: init Tainted: G        W         5.10.0-rc=
1-00402-ge2f016cf7751 #1048
> > >     Hardware name: Generic R-Car Gen2 (Flattened Device Tree)
> > >     PC is at sh_mdio_ctrl+0x44/0x60
> > >     LR is at sh_mmd_ctrl+0x20/0x24
> > >     ...
> > >     Backtrace:
> > >     [<c0451f30>] (sh_mdio_ctrl) from [<c0451fd4>] (sh_mmd_ctrl+0x20/0=
x24)
> > >      r7:0000001f r6:00000020 r5:00000002 r4:c22a1dc4
> > >     [<c0451fb4>] (sh_mmd_ctrl) from [<c044fc18>] (mdiobb_cmd+0x38/0xa=
8)
> > >     [<c044fbe0>] (mdiobb_cmd) from [<c044feb8>] (mdiobb_read+0x58/0xd=
c)
> > >      r9:c229f844 r8:c0c329dc r7:c221e000 r6:00000001 r5:c22a1dc4 r4:0=
0000001
> > >     [<c044fe60>] (mdiobb_read) from [<c044c854>] (__mdiobus_read+0x74=
/0xe0)
> > >      r7:0000001f r6:00000001 r5:c221e000 r4:c221e000
> > >     [<c044c7e0>] (__mdiobus_read) from [<c044c9d8>] (mdiobus_read+0x4=
0/0x54)
> > >      r7:0000001f r6:00000001 r5:c221e000 r4:c221e458
> > >     [<c044c998>] (mdiobus_read) from [<c044d678>] (phy_read+0x1c/0x20=
)
> > >      r7:ffffe000 r6:c221e470 r5:00000200 r4:c229f800
> > >     [<c044d65c>] (phy_read) from [<c044d94c>] (kszphy_config_intr+0x4=
4/0x80)
> > >     [<c044d908>] (kszphy_config_intr) from [<c044694c>] (phy_disable_=
interrupts+0x44/0x50)
> > >      r5:c229f800 r4:c229f800
> > >     [<c0446908>] (phy_disable_interrupts) from [<c0449370>] (phy_shut=
down+0x18/0x1c)
> > >      r5:c229f800 r4:c229f804
> > >     [<c0449358>] (phy_shutdown) from [<c040066c>] (device_shutdown+0x=
168/0x1f8)
> > >     [<c0400504>] (device_shutdown) from [<c013de44>] (kernel_restart_=
prepare+0x3c/0x48)
> > >      r9:c22d2000 r8:c0100264 r7:c0b0d034 r6:00000000 r5:4321fedc r4:0=
0000000
> > >     [<c013de08>] (kernel_restart_prepare) from [<c013dee0>] (kernel_r=
estart+0x1c/0x60)
> > >     [<c013dec4>] (kernel_restart) from [<c013e1d8>] (__do_sys_reboot+=
0x168/0x208)
> > >      r5:4321fedc r4:01234567
> > >     [<c013e070>] (__do_sys_reboot) from [<c013e2e8>] (sys_reboot+0x18=
/0x1c)
> > >      r7:00000058 r6:00000000 r5:00000000 r4:00000000
> > >     [<c013e2d0>] (sys_reboot) from [<c0100060>] (ret_fast_syscall+0x0=
/0x54)
> > >
> > > Calling phy_disable_interrupts() unconditionally means that the PHY
> > > registers may be accessed while the device is suspended, causing
> > > undefined behavior, which may crash the system.
> > >
> > > Fix this by calling phy_disable_interrupts() only when the PHY has be=
en
> > > started.
> > >
> > > Reported-by: Wolfram Sang <wsa+renesas@sang-engineering.com>
> > > Fixes: e2f016cf775129c0 ("net: phy: add a shutdown procedure")
> > > Signed-off-by: Geert Uytterhoeven <geert+renesas@glider.be>
> > > ---
> > > Marked RFC as I do not know if this change breaks the use case fixed =
by
> > > the faulty commit.
> >
> > I haven't tested it yet but most probably this change would partially
> > revert the behavior to how things were before adding the shutdown
> > procedure.
> >
> > And this is because the interrupts are enabled at phy_connect and not a=
t
> > phy_start so we would want to disable any PHY interrupts even though th=
e
> > PHY has not been started yet.
>=20
> Makes sense.
>=20
> > > Alternatively, the device may have to be started
> > > explicitly first.
> >
> > Have you actually tried this out and it worked?
>=20
> No, I haven't tested restarting the device first.
> I would like to avoid starting the device during shutdown, unless it is
> absolutely necessary.

I was talking about starting the PHY device but in light of the new
information, this would lead to the exact same crash since it's just
another PHY register access.

Now I understand that you were referring to the sh_eth device itself.

>=20
> > I am asking this because I would much rather expect this to be a proble=
m
> > with how the sh_eth driver behaves if the netdevice did not connect to
> > the PHY (this is done in .open() alongside the phy_start()) and it
> > suddently has to interract with it through the mdiobb_ops callbacks.
> >
> > Also, I just re-tested this use case in which I do not start the
> > interface and just issue a reboot, and it behaves as expected.
>=20
> It depends on the hardware: the sh_eth device is powered down when its
> module clock is stopped. When powered down, any access to the sh_eth
> registers or to the PHY connected to it will cause a crash.
>=20
> On most other hardware, you can access the PHY regardless, and no crash
> will happen.

Ok, so this does not have anything to do with interrupts explicitly but
rather with the fact that any PHY access will cause a crash when the
sh_eth device is powered down.

If the device is powered-down before the actual .ndo_open() how is the
probe actually setting up the device? Or is the device returned to the
powered-down state after the probe and only powered-up at a subsequent
.ndo_open()?

Instead of the phy_is_started() call we could check if we had previously
enabled the interrupts on the PHY but this would mean that a basic
assumption of the PHY library is violated in that a registered PHY
device cannot access its regiters because the MDIO controller just
decided so.

Can't the MDIO bitbang driver callbacks just check if the device is
powered-down and if it is just power it back up temporarily?=