Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.37 as permitted sender) client-ip=23.128.96.37;
Date:   Wed, 04 Oct 2023 16:32:47 +0100
Message-ID: <86cyxuo128.wl-maz@kernel.org>
From:   Marc Zyngier <maz@kernel.org>
To:     Anup Patel <apatel@ventanamicro.com>
Cc:     Dmitry Dunaev <dunaev@tecon.ru>, dunaich@mail.ru,
        Thomas Gleixner <tglx@linutronix.de>,
        Paul Walmsley <paul.walmsley@sifive.com>,
        Palmer Dabbelt <palmer@dabbelt.com>,
        Albert Ou <aou@eecs.berkeley.edu>,
        linux-kernel@vger.kernel.org, linux-riscv@lists.infradead.org
Subject: Re: [PATCH] irqchip/riscv-intc: Mark INTC nodes for secondary CPUs as initialized.
In-Reply-To: <CAK9=C2UUQ+ChueJVs+AnY0oCTb1zSv7qOCTijSchwxiMspX74w@mail.gmail.com>
References: <20230926102801.1591126-1-dunaev@tecon.ru>
        <CAK9=C2UiBNQtv0Q2yJMKqc5pMX_jam+ZmfRz3Rme0ZYuqN68HA@mail.gmail.com>
        <86il7mofmm.wl-maz@kernel.org>
        <CAK9=C2UUQ+ChueJVs+AnY0oCTb1zSv7qOCTijSchwxiMspX74w@mail.gmail.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.1
 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Wed, 04 Oct 2023 15:59:33 +0100,
Anup Patel <apatel@ventanamicro.com> wrote:
>=20
> On Wed, Oct 4, 2023 at 3:48=E2=80=AFPM Marc Zyngier <maz@kernel.org> wrot=
e:
> >
> > On Tue, 26 Sep 2023 11:36:31 +0100,
> > Anup Patel <apatel@ventanamicro.com> wrote:
> > >
> > > On Tue, Sep 26, 2023 at 3:59=E2=80=AFPM Dmitry Dunaev <dunaev@tecon.r=
u> wrote:
> > > >
> > > > The current Linux driver irq-riscv-intc initialize IRQ domain only =
once,
> > > > when init function called on primary hart. In other cases no IRQ do=
main is
> > > > created and no operation on interrupt-controller node is performed.
> > > > This is cause of that no common Linux driver can use per-cpu interr=
upts
> > > > mapped to several CPUs because fwnode of secondary cores INTC is not
> > > > marked as initialized. This device is always will be marked as defe=
rred.
> > > > For example the system with devicetree
> > > >
> > > >     cpu0: cpu@0 {
> > > >         cpu0_intc: interrupt-controller {
> > > >             interrupt-controller;
> > > >             compatible =3D riscv,cpu-intc;
> > > >         };
> > > >     };
> > > >
> > > >     cpu1: cpu@1 {
> > > >         cpu1_intc: interrupt-controller {
> > > >             interrupt-controller;
> > > >             compatible =3D riscv,cpu-intc;
> > > >         };
> > > >     };
> > > >
> > > >     buserr {
> > > >         compatible =3D riscv,buserr;
> > > >         interrupts-extended =3D <&cpu0_intc 16 &cpu1_intc 16>;
> > > >     };
> > > >
> > > > will always report 'buserr' node as deferred without calling any
> > > > bus probe function.
> > > >
> > > > This patch will mark all secondary nodes passed to irq-riscv-intc
> > > > driver init function as initialized to be able to act as correct
> > > > IRQ phandle node.
> > > >
> > > > Signed-off-by: Dmitry Dunaev <dunaev@tecon.ru>
> > > > ---
> > > >  drivers/irqchip/irq-riscv-intc.c | 8 ++++++--
> > > >  1 file changed, 6 insertions(+), 2 deletions(-)
> > > >
> > > > diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq=
-riscv-intc.c
> > > > index 4adeee1bc391..c01a4e8d4983 100644
> > > > --- a/drivers/irqchip/irq-riscv-intc.c
> > > > +++ b/drivers/irqchip/irq-riscv-intc.c
> > > > @@ -155,8 +155,10 @@ static int __init riscv_intc_init(struct devic=
e_node *node,
> > > >          * for each INTC DT node. We only need to do INTC initializ=
ation
> > > >          * for the INTC DT node belonging to boot CPU (or boot HART=
).
> > > >          */
> > > > -       if (riscv_hartid_to_cpuid(hartid) !=3D smp_processor_id())
> > > > +       if (riscv_hartid_to_cpuid(hartid) !=3D smp_processor_id()) {
> > > > +               fwnode_dev_initialized(of_node_to_fwnode(node), tru=
e);
> > >
> > > There is already a patch on LKML to address this.
> > > https://www.spinics.net/lists/kernel/msg4929886.html
> >
> > If this is a fix, why is it buried in a huge series and not brought
> > forward as an independent fix that needs to be picked early?
>=20
> Dmitry saw this issue in a totally different context which is not
> reproducible with existing DTS files in kernel sources.

I hope you're not suggesting that only the DTs that are present in the
kernel tree are valid. Because as far as I'm concern, the DTs in the
kernel tree are only some *examples*, and not a reference.

I fully expect the vast majority of DTs to live *outside* of the
kernel tree, provided by the firmware, and never upstreamed. Would you
expect every PC vendor to upstream their ACPI tables?

> This issue only manifests when some platform driver DT node
> points to the per-HART INTC nodes. For example, RISC-V
> irqchip device DT nodes point to per-HART INTC nodes.

Is this configuration legal or not as per the DT binding? I don't see
anything that suggests it isn't legal, and having per-CPU interrupts
isn't exactly a new thing.

> Currently, all RISC-V irqchip drivers (INTC and PLIC) are probed
> early (not as platform drivers) so we don't see this issue with
> existing irqchip drivers.

You don't, but Dimitry does. Who wins?

	M.

--=20
Without deviation from the norm, progress is not possible.