Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Fri, 05 Nov 2021 11:35:46 +0000
Message-ID: <87zgqi96nh.wl-maz@kernel.org>
From:   Marc Zyngier <maz@kernel.org>
To:     Brad Larson <brad@pensando.io>
Cc:     Mark Rutland <mark.rutland@arm.com>,
        Linux ARM <linux-arm-kernel@lists.infradead.org>,
        Arnd Bergmann <arnd@arndb.de>,
        Linus Walleij <linus.walleij@linaro.org>,
        Bartosz Golaszewski <bgolaszewski@baylibre.com>,
        Mark Brown <broonie@kernel.org>,
        Serge Semin <fancer.lancer@gmail.com>,
        Adrian Hunter <adrian.hunter@intel.com>,
        Ulf Hansson <ulf.hansson@linaro.org>,
        Olof Johansson <olof@lixom.net>,
        "open list:GPIO SUBSYSTEM" <linux-gpio@vger.kernel.org>,
        linux-spi <linux-spi@vger.kernel.org>,
        linux-mmc <linux-mmc@vger.kernel.org>,
        "open list:OPEN FIRMWARE AND FLATTENED DEVICE TREE BINDINGS" 
        <devicetree@vger.kernel.org>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v3 11/11] arm64: dts: Add Pensando Elba SoC support
In-Reply-To: <CAK9rFnxH=DtC9vEJJCXkGEWjvg2E8vHPUcRy9rJpo34Furtmvw@mail.gmail.com>
References: <20211025015156.33133-1-brad@pensando.io>
        <20211025015156.33133-12-brad@pensando.io>
        <20211025091731.GA2001@C02TD0UTHF1T.local>
        <a20805de16e1196c2ed46dd949473c9a@kernel.org>
        <CAK9rFnxH=DtC9vEJJCXkGEWjvg2E8vHPUcRy9rJpo34Furtmvw@mail.gmail.com>
User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue)
 FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1
 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO)
MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue")
Content-Type: text/plain; charset=US-ASCII
Precedence: bulk

Brad,

On Fri, 05 Nov 2021 00:02:04 +0000,
Brad Larson <brad@pensando.io> wrote:
> 
> Hi Marc,
> 
> On Mon, Oct 25, 2021 at 4:15 AM Marc Zyngier <maz@kernel.org> wrote:
> >
> > On 2021-10-25 10:17, Mark Rutland wrote:
> > > Hi,
> > >
> > > On Sun, Oct 24, 2021 at 06:51:56PM -0700, Brad Larson wrote:
> > >> Add Pensando common and Elba SoC specific device nodes
> > >>
> > >> Signed-off-by: Brad Larson <brad@pensando.io>
> > >
> > > [...]
> > >> +            gic: interrupt-controller@800000 {
> > >> +                    compatible = "arm,gic-v3";
> > >> +                    #interrupt-cells = <3>;
> > >> +                    #address-cells = <2>;
> > >> +                    #size-cells = <2>;
> > >> +                    ranges;
> > >> +                    interrupt-controller;
> > >> +                    reg = <0x0 0x800000 0x0 0x200000>,      /* GICD */
> > >> +                          <0x0 0xa00000 0x0 0x200000>;      /* GICR */
> >
> > This is missing the GICv2 compat regions that the CPUs implement.
> 
> Is this what is described as optional in the GIC architecture specification
> where a GICv3 system can run restricted GICv2 code?

Yup, that. It is actually implemented by the CPU.

> Can you point me in the right direction in the spec and example dts
> node if needed.

The Cortex-A72 TRM has everything you need [1]. And since you used the
Synquacer as the model for this, you will see that it has the missing
regions. Alternatively, rk3399.dtsi is a good example.


> > >> +                    interrupts = <GIC_PPI 9 IRQ_TYPE_LEVEL_HIGH>;
> > >> +
> > >> +                    gic_its: msi-controller@820000 {
> > >> +                            compatible = "arm,gic-v3-its";
> > >> +                            msi-controller;
> > >> +                            #msi-cells = <1>;
> > >> +                            reg = <0x0 0x820000 0x0 0x10000>;
> > >> +                            socionext,synquacer-pre-its =
> > >> +                                                    <0xc00000 0x1000000>;
> > >> +                    };
> > >> +            };
> > >
> > > Is there any shared lineage with Synquacer? The commit message didn't
> > > describe this quirk.
> >
> > Funny, it looks like there is a sudden outburst of stupid copy/paste
> > among HW designers. TI did the exact same thing recently.
> >
> > This totally negates all the advantages of having an ITS and makes
> > sure that you have all the overhead. Facepalm...
> 
> Some background may help explain.  To generate an LPI a peripheral must
> write to the GITS_TRANSLATER (a specific address). For the ITS to know
> which translations apply to the generated interrupts, it must know which
> peripheral performed the write. The ID of the peripheral is known as its
> DeviceID, which is often carried along with the write as an AXI sideband
> signal.

Yes, I happen to be vaguely familiar with the GIC architecture.

> The Elba SoC doesn't carry the DeviceID, so we have to conjure one up
> between the peripheral and the ITS.  Instead of telling a peripheral to target
> the GITS_TRANSLATER directly, we instead direct it to a specific offset
> within a pre-ITS address range (our own IP block).  For writes that land in
> that memory range, we derive the DeviceID from (offset >> 2).  The pre-ITS
> block then sends (DeviceID, data) to the GITS_TRANSLATER.
> 
> The hardware designer came up with the Pre-ITS mechanism in Feb 2018.
> When we looked at the upstream kernel later (we developed on 4.14)
> we found that not only did it support something similar, it supported the
> exact scheme we are using.

And this scheme is totally wrong. It breaks interrupt isolation.

Instead of having a single doorbell and getting the ITS to segregate
between devices itself, you end-up with multiple ones, allowing a
rogue device to impersonate another one by targeting another doorbell.
You can't even use an SMMU to preserve some isolation, because all the
doorbells are in the *same page*. Unmitigated disaster.

At this stage, why did you bother having an ITS at all? You get none
of the security features. Only the excess area, memory allocation,
additional latency and complexity. All you get is a larger INTID
space.

This only shows that the hardware designer didn't understand the ITS
at all. Which seems a common pattern, unfortunately.

	M.

[1] https://developer.arm.com/documentation/100095/0003/Generic-Interrupt-Controller-CPU-Interface/GIC-functional-description/GIC-memory-map?lang=en#way1382452674438__CHDEBJAJ

-- 
Without deviation from the norm, progress is not possible.