Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20201020072532.949137-1-narmstrong@baylibre.com>
 <20201020072532.949137-2-narmstrong@baylibre.com> <7hsga8kb8z.fsf@baylibre.com>
 <CAF2Aj3g6c8FEZb3e1by6sd8LpKLaeN5hsKrrQkZUvh8hosiW9A@mail.gmail.com>
 <87r1hwwier.wl-maz@kernel.org> <7h7diwgjup.fsf@baylibre.com> <87im0m277h.wl-maz@kernel.org>
In-Reply-To: <87im0m277h.wl-maz@kernel.org>
From:   Saravana Kannan <saravanak@google.com>
Date:   Tue, 3 Aug 2021 18:36:45 -0700
Message-ID: <CAGETcx9OukoWM_qprMse9aXdzCE=GFUgFEkfhhNjg44YYsOQLw@mail.gmail.com>
Subject: Re: [PATCH 1/2] irqchip: irq-meson-gpio: make it possible to build as
 a module
To:     Marc Zyngier <maz@kernel.org>
Cc:     Kevin Hilman <khilman@baylibre.com>,
        Lee Jones <lee.jones@linaro.org>,
        Neil Armstrong <narmstrong@baylibre.com>,
        Jerome Brunet <jbrunet@baylibre.com>,
        linux-amlogic@lists.infradead.org,
        linux-arm-kernel <linux-arm-kernel@lists.infradead.org>,
        open list <linux-kernel@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Tue, Aug 3, 2021 at 2:44 AM Marc Zyngier <maz@kernel.org> wrote:
>
> On Mon, 14 Jun 2021 23:30:22 +0100,
> Kevin Hilman <khilman@baylibre.com> wrote:
> >
> > Marc Zyngier <maz@kernel.org> writes:
> >
> > > On Fri, 21 May 2021 10:47:48 +0100,
> > > Lee Jones <lee.jones@linaro.org> wrote:
> > >>
> > >> [1  <text/plain; UTF-8 (quoted-printable)>]
> > >> On Tue, 20 Oct 2020 at 19:23, Kevin Hilman <khilman@baylibre.com> wrote:
> > >>
> > >> > Neil Armstrong <narmstrong@baylibre.com> writes:
> > >> >
> > >> > > In order to reduce the kernel Image size on multi-platform distributions,
> > >> > > make it possible to build the Amlogic GPIO IRQ controller as a module
> > >> > > by switching it to a platform driver.
> > >> > >
> > >> > > Signed-off-by: Neil Armstrong <narmstrong@baylibre.com>
> > >> >
> > >> > Reviewed-by: Kevin Hilman <khilman@baylibre.com>
> > >> > Tested-by: Kevin Hilman <khilman@baylibre.com>
> > >> >
> > >> > Tested as a module on meson-sm1-khadas-vim3l where the wired networking
> > >> > uses GPIO IRQs.
> > >> >
> > >>
> > >> Good morning Neil, Kevin,
> > >>
> > >> What happened to this set in the end?  I still don't see it in Mainline.
> > >
> > > Last time I tried this patch, it broke my test setup in non-obvious
> > > ways. Has someone checked that the issue I reported back then has been
> > > resolved now that fw_devlink is more usable?
> >
> > OK, after much anticipation (and much delay due to me forgetting about
> > this), I just gave this series a spin again on top of v5.13-rc6, and it
> > seems to work fine with `fw_devlink=on`
> >
> > I started with your config[1] and accepting all the defaults of any new
> > configs.  IOW, I ran: yes '' | make oldconfig after copying your config
> > to .config.
> >
> > With that it seems to be working fine for me.
> >
> > Right after boot (and before network probes) I see module loaded, but no
> > users yet in /proc/interrupts:
> >
> > / # uname -a
> > Linux buildroot 5.13.0-rc6-00002-g679c8e852942 #5 SMP PREEMPT Mon Jun 14 15:08:40 PDT 2021 aarch64 GNU/Linux
> > / # lsmod |grep gpio
> > irq_meson_gpio         20480  0
> > leds_gpio              16384  0
> > / # cat /proc/interrupts
> >            CPU0       CPU1       CPU2       CPU3
> >   9:          0          0          0          0     GICv2  25 Level     vgic
> >  11:          0          0          0          0     GICv2  30 Level     kvm guest ptimer
> >  12:          0          0          0          0     GICv2  27 Level     kvm guest vtimer
> >  13:       1416        916        534       1421     GICv2  26 Level     arch_timer
> >  15:          5          0          0          0     GICv2  89 Edge      dw_hdmi_top_irq, ff600000.hdmi-tx
> >  22:         38          0          0          0     GICv2 225 Edge      ttyAML0
> >  23:         20          0          0          0     GICv2 227 Edge      ff805000.i2c
> >  25:          2          0          0          0     GICv2 232 Edge      ff809000.adc
> >  28:        322          0          0          0     GICv2  35 Edge      meson
> >  31:          0          0          0          0     GICv2 222 Edge      ffe05000.sd
> >  32:        787          0          0          0     GICv2 223 Edge      ffe07000.mmc
> >  34:          0          0          0          0     GICv2 194 Level     panfrost-job
> >  35:          0          0          0          0     GICv2 193 Level     panfrost-mmu
> >  36:          3          0          0          0     GICv2 192 Level     panfrost-gpu
> >  39:          0          0          0          0     GICv2  63 Level     ff400000.usb, ff400000.usb
> >  40:         32          0          0          0     GICv2  62 Level     xhci-hcd:usb1
> > IPI0:       425        544        664        925       Rescheduling interrupts
> > IPI1:        86        166        269        136       Function call interrupts
> > IPI2:         0          0          0          0       CPU stop interrupts
> > IPI3:         0          0          0          0       CPU stop (for crash dump) interrupts
> > IPI4:         0          0          0          0       Timer broadcast interrupts
> > IPI5:         0          0          0          0       IRQ work interrupts
> > IPI6:         0          0          0          0       CPU wake-up interrupts
> > Err:          0
> >
> > So then I init the network interface and PHY works, DHCP works etc.
> >
> > / # udhcpc
> > udhcpc: started, v1.31.1
> > [  102.250449] meson8b-dwmac ff3f0000.ethernet eth0: PHY [0.0:00] driver [RTL8211F Gigabit Ethernet] (irq=37)
> > [  102.256413] meson8b-dwmac ff3f0000.ethernet eth0: Register MEM_TYPE_PAGE_POOL RxQ-0
> > [  102.269433] meson8b-dwmac ff3f0000.ethernet eth0: No Safety Features support found
> > [  102.271357] meson8b-dwmac ff3f0000.ethernet eth0: PTP not supported by HW
> > [  102.278493] meson8b-dwmac ff3f0000.ethernet eth0: configuring for phy/rgmii link mode
> > udhcpc: sending discover
> > [  104.743301] meson8b-dwmac ff3f0000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx
> > [  104.746470] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
> > udhcpc: sending discover
> > udhcpc: sending select for 192.168.0.122
> > udhcpc: lease of 192.168.0.122 obtained, lease time 600
> > deleting routers
> > adding dns 192.168.0.254
> > adding dns 192.168.0.254
> > / # cat /proc/interrupts
> >            CPU0       CPU1       CPU2       CPU3
> >   9:          0          0          0          0     GICv2  25 Level     vgic
> >  11:          0          0          0          0     GICv2  30 Level     kvm guest ptimer
> >  12:          0          0          0          0     GICv2  27 Level     kvm guest vtimer
> >  13:       1575       1018        604       1588     GICv2  26 Level     arch_timer
> >  14:          8          0          0          0     GICv2  40 Level     eth0
> >  15:          5          0          0          0     GICv2  89 Edge      dw_hdmi_top_irq, ff600000.hdmi-tx
> >  22:        132          0          0          0     GICv2 225 Edge      ttyAML0
> >  23:         20          0          0          0     GICv2 227 Edge      ff805000.i2c
> >  25:          2          0          0          0     GICv2 232 Edge      ff809000.adc
> >  28:        322          0          0          0     GICv2  35 Edge      meson
> >  31:          0          0          0          0     GICv2 222 Edge      ffe05000.sd
> >  32:        787          0          0          0     GICv2 223 Edge      ffe07000.mmc
> >  34:          0          0          0          0     GICv2 194 Level     panfrost-job
> >  35:          0          0          0          0     GICv2 193 Level     panfrost-mmu
> >  36:          3          0          0          0     GICv2 192 Level     panfrost-gpu
> >  37:          2          0          0          0  meson-gpio-irqchip  26 Level     0.0:00
> >  39:          0          0          0          0     GICv2  63 Level     ff400000.usb, ff400000.usb
> >  40:         32          0          0          0     GICv2  62 Level     xhci-hcd:usb1
> > IPI0:       476        567        720        956       Rescheduling interrupts
> > IPI1:        93        166        270        137       Function call interrupts
> > IPI2:         0          0          0          0       CPU stop interrupts
> > IPI3:         0          0          0          0       CPU stop (for crash dump) interrupts
> > IPI4:         0          0          0          0       Timer broadcast interrupts
> > IPI5:         0          0          0          0       IRQ work interrupts
> > IPI6:         0          0          0          0       CPU wake-up interrupts
> > Err:          0
> > / #
>
> This thing keeps failing on my end. It only works if I force the
> irqchip module to be present before the MDIO module is loaded. Here's
> an example:
>
> root@tiger-roach:~# modprobe mdio_mux_meson_g12a
> [  125.871544] libphy: mdio_mux: probed
> [  125.882575] g12a-mdio_mux ff64c000.mdio-multiplexer: Error: Failed to register MDIO bus for child /soc/bus@ff600000/mdio-multiplexer@4c000/mdio@0
> [  125.892630] libphy: mdio_mux: probed
>
> Trying to bring up the Ethernet interface will fail. Note that there
> was no attempt to load the irqchip driver.
>
> root@tiger-roach:~# modprobe -r mdio_mux_meson_g12a
> root@tiger-roach:~# modprobe irq-meson-gpio
> [  144.983344] meson-gpio-intc ffd0f080.interrupt-controller: 100 to 8 gpio interrupt mux initialized
> root@tiger-roach:~# modprobe mdio_mux_meson_g12a
> [  150.376464] libphy: mdio_mux: probed
> [  150.391039] libphy: mdio_mux: probed
>
> And it now works.
>
> Is it a MDIO issue? a fw_devlink issue? No idea. But I'd really like
> to see this addressed before taking this patch, as everything works
> just fine as long as the irqchip is built in (which on its own could
> well pure luck).
>
> Saravana, could you please have a look from a fw_devlink perspective?

Sigh... I spent several hours looking at this and wrote up an analysis
and then realized I might be looking at the wrong DT files.

Marc, can you point me to the board file in upstream that corresponds
to the platform in which you see this issue? I'm not asking for [1],
but the actual final .dts (not .dtsi) file that corresponds to the
platform/board/system.

Based on your error messages, it's failing for mdio@0 which
corresponds to ext_mdio. But none of the board dts files in upstream
have a compatible property for "ext_mdio". Which means fw_devlink
_should_ propagate the gpio_intc IRQ dependency all the way up to
eth_phy.

Also, in the failing case, can you run:
ls -ld supplier:*

in the /sys/devices/....<something>/ folder that corresponds to the
"eth_phy: mdio-multiplexer@4c000" DT node and tell me what it shows?

Thanks,
Saravana

[1] - arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi