Date:   Fri, 24 Dec 2021 09:51:44 +0000
Message-ID: <87wnjuz5xr.wl-maz@kernel.org>
From:   Marc Zyngier <maz@kernel.org>
To:     Huacai Chen <chenhuacai@gmail.com>
Cc:     Huacai Chen <chenhuacai@loongson.cn>,
        Thomas Gleixner <tglx@linutronix.de>,
        LKML <linux-kernel@vger.kernel.org>,
        Xuefeng Li <lixuefeng@loongson.cn>,
        Jiaxun Yang <jiaxun.yang@flygoat.com>
Subject: Re: [PATCH V8 02/10] irqchip/loongson-pch-pic: Add ACPI init support
In-Reply-To: <CAAhV-H6VFaRtiyhgq-vDQPUbiAtf9L1M6ZNyeApTu11O9JCXnw@mail.gmail.com>
References: <20211216125157.631992-1-chenhuacai@loongson.cn>
        <20211216125356.632067-1-chenhuacai@loongson.cn>
        <20211216125356.632067-2-chenhuacai@loongson.cn>
        <87pmpwwpw5.wl-maz@kernel.org>
        <CAAhV-H75SwqWiRjey_9MiRQtY-_Wjm7Tppx31XM8EfLDb_YUhQ@mail.gmail.com>
        <87czlrwk2k.wl-maz@kernel.org>
        <CAAhV-H6VFaRtiyhgq-vDQPUbiAtf9L1M6ZNyeApTu11O9JCXnw@mail.gmail.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1
 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk

On Fri, 24 Dec 2021 07:31:53 +0000,
Huacai Chen <chenhuacai@gmail.com> wrote:
> 
> Hi, Marc,
> 
> On Mon, Dec 20, 2021 at 8:13 PM Marc Zyngier <maz@kernel.org> wrote:
> >
> > On Fri, 17 Dec 2021 04:45:24 +0000,
> > Huacai Chen <chenhuacai@gmail.com> wrote:
> > >
> > > Hi, Marc,
> > >
> > > On Thu, Dec 16, 2021 at 11:06 PM Marc Zyngier <maz@kernel.org> wrote:
> > > >
> > > > On Thu, 16 Dec 2021 12:53:48 +0000,
> > > > Huacai Chen <chenhuacai@loongson.cn> wrote:
> > > > >
> > > > > We are preparing to add new Loongson (based on LoongArch, not compatible
> > > > > with old MIPS-based Loongson) support. LoongArch use ACPI other than DT
> > > > > as its boot protocol, so add ACPI init support.
> > > > >
> > > > > PCH-PIC/PCH-MSI stands for "Interrupt Controller" that described in
> > > > > Section 5 of "Loongson 7A1000 Bridge User Manual". For more information
> > > > > please refer Documentation/loongarch/irq-chip-model.rst.
> > > > >
> > > > > Signed-off-by: Huacai Chen <chenhuacai@loongson.cn>
> > > > > ---
> > > > >  drivers/irqchip/irq-loongson-pch-pic.c | 108 ++++++++++++++++++-------
> > > > >  1 file changed, 81 insertions(+), 27 deletions(-)
> > > >
> > > > [...]
> > > >
> > > > >
> > > > > +#ifdef CONFIG_ACPI
> > > > > +
> > > > > +struct irq_domain *pch_pic_acpi_init(struct irq_domain *parent,
> > > > > +                                     struct acpi_madt_bio_pic *acpi_pchpic)
> > > >
> > > > Who is calling this? This works the opposite way from what the arm64
> > > > irqchips are doing. Why? I have the ugly feeling that this is called
> > > > from the arch code, bypassing the existing infrastructure...
> > > Yes, this is called from the arch code and a bit ugly, but I can't
> > > find a better way to do this.
> > >
> > > Is the "existing infrastructure" declare the irqchip init function
> > > with  IRQCHIP_ACPI_DECLARE and the arch code only need to call
> > > irqchip_init()? Then we have a problem: our irqchips have a 4 level
> > > hierachy and the parent should be initialized before its children. In
> > > FDT world this is not a problem, because of_irq_init() will sort
> > > irqchip drivers to ensure the right order. But in ACPI world,
> > > acpi_probe_device_table just call init functions in the linking order.
> > > If we want to control the order, it seems we can only sort the drivers
> > > in drivers/irq/Makefile. But I don't think this is a good idea...
> > >
> > > If there are better solutions, please let me know. Thanks.
> >
> > We have the exact same thing on the arm64 side, and we don't need of
> > this to be arch specific:
> >
> > - The MADT table describes the root interrupt controller, and it is
> >   probed via IRQCHIP_ACPI_DECLARE().
> >
> > - Each children controller is declared in ACPI as a *device*, and is
> >   both an interrupt producer and an interrupt consumer. Normal probe
> >   deferral rules apply. See irq-mbigen.c for an example of how this is
> >   done.
> Thank you for your suggestions, I have tried but failed. It seems
> there are some differences between irq-mbigen.c and our irqchips.
> Because our irqchips are mandatory while mbigen is optional.

The fact that this is optional has nothing to do with it. On a system
that requires mbigen to boot (both mass storage and networking are
hanging off it), there is a guarantee that the probe order will
respect the resource dependency.

And if that's not enough, -EPROBE_DEFER is your friend, always.

> If we declare our irqchips as devices, they are initialized in the
> initcall phase, which is too late for pci devices.

This suggests that your PCIe driver is either not enforcing the
dependencies on the interrupt controller (bad), or that the core code
is not made aware of the dependencies (equally bad).

In any case, this needs sorting, because a new architecture should be
able to boot without resorting to handcrafted dependencies that will
inevitably result in a larger pile of hacks over time. It is much
easier to solve it before day-1.

	M.

-- 
Without deviation from the norm, progress is not possible.