Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Message-ID: <9ceab104f5189664648c219fb9d1e0d70d6bda9f.camel@svanheule.net>
Subject: Re: [PATCH v1 3/6] gpio: realtek-otto: Support per-cpu interrupts
From:   Sander Vanheule <sander@svanheule.net>
To:     Marc Zyngier <maz@kernel.org>,
        Linus Walleij <linus.walleij@linaro.org>
Cc:     linux-gpio@vger.kernel.org, devicetree@vger.kernel.org,
        Bartosz Golaszewski <brgl@bgdev.pl>,
        Rob Herring <robh+dt@kernel.org>,
        Krzysztof Kozlowski <krzk+dt@kernel.org>,
        Bert Vermeulen <bert@biot.com>, linux-kernel@vger.kernel.org
Date:   Fri, 22 Apr 2022 22:40:39 +0200
In-Reply-To: <b8b62753ad5235e065b4cb0856a7a7c33438dfbb.camel@svanheule.net>
References: <cover.1649533972.git.sander@svanheule.net>
         <8d4e0848f233c2c1b98aa141741c61d95cd3843f.1649533972.git.sander@svanheule.net>
         <CACRpkdbSdDAKiFAsHBosdVDpBhWW-Keoq+t8GJ5LsyWjOZwp_g@mail.gmail.com>
         <87h76mahsl.wl-maz@kernel.org>
         <b8b62753ad5235e065b4cb0856a7a7c33438dfbb.camel@svanheule.net>
Content-Type: text/plain; charset="UTF-8"
User-Agent: Evolution 3.42.4 (3.42.4-2.fc35) 
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

On Fri, 2022-04-22 at 09:04 +0200, Sander Vanheule wrote:
> Hi Linus, Marc,
> 
> On Thu, 2022-04-21 at 10:48 +0100, Marc Zyngier wrote:
> > On Thu, 21 Apr 2022 00:04:16 +0100,
> > Linus Walleij <linus.walleij@linaro.org> wrote:
> > > 
> > > On Sat, Apr 9, 2022 at 9:56 PM Sander Vanheule <sander@svanheule.net> wrote:
> > > 
> > > > On SoCs with multiple cores, it is possible that the GPIO interrupt
> > > > controller supports assigning specific pins to one or more cores.
> > > > 
> > > > IRQ balancing can be performed on a line-by-line basis if the parent
> > > > interrupt is routed to all available cores, which is the default upon
> > > > initialisation.
> > > > 
> > > > Signed-off-by: Sander Vanheule <sander@svanheule.net>
> > > 
> > > That sounds complicated.
> > > 
> > > Sounds like something the IRQ maintainer (Marc Z) should
> > > have a quick look at.
> > 
> > This is pretty odd indeed. There seem to be a direct mapping between
> > the GPIOs and the CPU it interrupts (or at least that's what the code
> > seem to express). However, I don't see a direct relation between the
> > CPUs and the chained interrupt. It isn't even clear if this interrupt
> > itself is per-CPU.
> > 
> > So this begs a few questions:
> > 
> > - is the affinity actually affecting the target CPU? or is it
> >   affecting the target mux?
> > 
> > - how is the affinity of the mux interrupt actually enforced?
> 
> There are three interrupt controllers at play here:
>    1. MIPS CPU interrupt controller: drivers/irqchip/irq-mips-cpu.c
>       One interrupt controller per VPE, so in this case there are two. Provides
>       per-CPU interrupts.
>    2. SoC interrupt controller: drivers/irqchip/irq-realtek-rtl.c
>       Also one interrupt controller per VPE. I suppose these will also be per-
>       CPU, although this isn't implemented in the driver yet, and I don't think
>       I yet fully understand how should work in the kernel.
>    3. GPIO interrupt controller: drivers/gpio/gpio-realtek-otto.c
>       One interrupt controller for the entire GPIO bank, with optional
>       configurable affinity (this patch) for the different VPEs.
> 
> For the RTL839x series of SoCs, this results in the following:

Sorry for the messed up formattng, let me try that again.

GPIO LINES        SOC IRQ              MIPS
    +--------+    LINE +-----------+   HW IRQ +--------+
--->| GPIO   |         | SOC IRQ   |   LINES  | IRQ    |
--->| BANK   |-----o-->| VPE0 CTRL |=========>| VPE0   |
 .  |        |     |   +-----------+          +--------+
 .  +--------+     | 
 .                 |
                   |   +-----------+          +--------+
                   \-->| SOC IRQ   |          | IRQ    |
                       | VPE1 CTRL |=========>| VPE1   |
                       +-----------+          +--------+


> For RTL930x, where GPIO IRQ affinity is configurable:

GPIO LINES        SOC IRQ              MIPS
    +--------+    LINE +-----------+   HW IRQ +--------+
--->| GPIO   |-------->| SOC IRQ   |   LINES  | IRQ    |
--->| BANK   |         | VPE0 CTRL |=========>| VPE0   |
 .  |        |-----\   +-----------+          +--------+
 .  +--------+     | 
 .                 |
                   |   +-----------+          +--------+
                   \-->| SOC IRQ   |          | IRQ    |
                       | VPE1 CTRL |=========>| VPE1   |
                       +-----------+          +--------+

> The interrupt for the GPIO controller can be muxed to any of the MIPS HW
> interrupts on any (or all) of the VPEs, and these muxes (SoC IRQ controllers)
> can be configured independently per CPU. The SoC IRQ line index is fixed, and
> consistent for both VPEs.
> Only in the second diagram can individual GPIO interrupts be muxed to any of the
> VPEs, but there is still only one IRQ line per VPE for all selected GPIO lines.
> 
> I hopes this helps to clarify the situation. We don't have any real
> documentation, so this is basically derived from registers descriptions in SDK
> headers and testing the interrupt behaviour.
> 
> Best,
> Sander