Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <20230719113542.2293295-1-apatel@ventanamicro.com>
 <20230719113542.2293295-4-apatel@ventanamicro.com> <CAGETcx9zAF2ipO0s-6-zjyvn1JWt7OUS9G=cQ6OwyOPuqh-pBA@mail.gmail.com>
In-Reply-To: <CAGETcx9zAF2ipO0s-6-zjyvn1JWt7OUS9G=cQ6OwyOPuqh-pBA@mail.gmail.com>
From:   Anup Patel <apatel@ventanamicro.com>
Date:   Thu, 20 Jul 2023 10:51:13 +0530
Message-ID: <CAK9=C2WZYn_TFZVY2a-DMaGzTesHUotDA2YJCjM6dYmcyyZ1og@mail.gmail.com>
Subject: Re: [PATCH v6 03/14] drivers: irqchip/riscv-intc: Mark all INTC nodes
 as initialized
To:     Saravana Kannan <saravanak@google.com>
Cc:     Palmer Dabbelt <palmer@dabbelt.com>,
        Paul Walmsley <paul.walmsley@sifive.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Marc Zyngier <maz@kernel.org>,
        Rob Herring <robh+dt@kernel.org>,
        Krzysztof Kozlowski <krzysztof.kozlowski+dt@linaro.org>,
        Frank Rowand <frowand.list@gmail.com>,
        Conor Dooley <conor+dt@kernel.org>,
        Atish Patra <atishp@atishpatra.org>,
        Andrew Jones <ajones@ventanamicro.com>,
        Sunil V L <sunilvl@ventanamicro.com>,
        Anup Patel <anup@brainfault.org>,
        linux-riscv@lists.infradead.org, linux-kernel@vger.kernel.org,
        devicetree@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Precedence: bulk

On Thu, Jul 20, 2023 at 3:45=E2=80=AFAM Saravana Kannan <saravanak@google.c=
om> wrote:
>
> On Wed, Jul 19, 2023 at 4:36=E2=80=AFAM Anup Patel <apatel@ventanamicro.c=
om> wrote:
> >
> > The RISC-V INTC local interrupts are per-HART (or per-CPU) so
> > we create INTC IRQ domain only for the INTC node belonging to
> > the boot HART. This means only the boot HART INTC node will be
> > marked as initialized and other INTC nodes won't be marked which
> > results downstream interrupt controllers (such as IMSIC and APLIC
> > direct-mode) not being probed due to missing device suppliers.
> >
> > To address this issue, we mark all INTC node for which we don't
> > create IRQ domain as initialized.
> >
> > Signed-off-by: Anup Patel <apatel@ventanamicro.com>
> > ---
> >  drivers/irqchip/irq-riscv-intc.c | 10 +++++++++-
> >  1 file changed, 9 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/irqchip/irq-riscv-intc.c b/drivers/irqchip/irq-ris=
cv-intc.c
> > index 65f4a2afb381..4e2704bc25fb 100644
> > --- a/drivers/irqchip/irq-riscv-intc.c
> > +++ b/drivers/irqchip/irq-riscv-intc.c
> > @@ -155,8 +155,16 @@ static int __init riscv_intc_init(struct device_no=
de *node,
> >          * for each INTC DT node. We only need to do INTC initializatio=
n
> >          * for the INTC DT node belonging to boot CPU (or boot HART).
> >          */
> > -       if (riscv_hartid_to_cpuid(hartid) !=3D smp_processor_id())
> > +       if (riscv_hartid_to_cpuid(hartid) !=3D smp_processor_id()) {
> > +               /*
> > +                * The INTC nodes of each CPU are suppliers for downstr=
eam
> > +                * interrupt controllers (such as IMSIC and APLIC direc=
t-mode)
> > +                * so we should mark an INTC node as initialized if we =
are
> > +                * not creating IRQ domain for it.
> > +                */
>
> I'm a bit confused by this. If those non-boot CPUs INTC doesn't have
> an IRQ domain, why are the downstream interrupt controllers listing
> these non-boot CPU INTCs as an upstream interrupt controller?

Downstream interrupt controllers (such as PLIC, APLIC direct-mode,
or IMSIC) use "interrupts-extended" DT property to associate their
MMIO resources with target CPU.

For example: If we have 4 CPUs then PLIC will typically have 2 PLIC
MMIO contexts for each CPU where one context is for M-mode and
another context is for S-mode. In this case, there are total 8 PLIC
MMIO contexts and the arrangement of these MMIO resources is
platform specific so "interrupts-extended" DT property is used by
PLIC driver to discover the arrangement of PLIC MMIO contexts
and their mapping to target CPU.

Similar to the above example, the APLIC direct-mode has per-CPU
IDC MMIO registers and "interrupts-extended" DT property is used
to discover arrangement of these IDC MMIO registers and their
mapping to the target CPU.

>
> This is more of a question of the existing behavior that this patch,
> but this patch highlights the existing oddity.
>
> -Saravana
>
> > +               fwnode_dev_initialized(of_fwnode_handle(node), true);
> >                 return 0;
> > +       }
> >
> >         return riscv_intc_init_common(of_node_to_fwnode(node));
> >  }
> > --
> > 2.34.1
> >

Regards,
Anup