Subject: Re: [PATCH] ARM64: dts: rockchip: add core dtsi file for RK3399 SoCs
To: Marc Zyngier <marc.zyngier@arm.com>
References: <1461122150-9042-1-git-send-email-jay.xu@rock-chips.com>
 <1461211092-26331-1-git-send-email-jay.xu@rock-chips.com>
 <20160421101930.GG6879@leverpostej> <5718AFB8.5070004@rock-chips.com>
 <20160421123018.096d4a75@arm.com>
Cc: Mark Rutland <mark.rutland@arm.com>, devicetree@vger.kernel.org,
        davidriley@chromium.org, heiko@sntech.de, pawel.moll@arm.com,
        ijc+devicetree@hellion.org.uk, catalin.marinas@arm.com,
        will.deacon@arm.com, dianders@chromium.org, smbarber@chromium.org,
        linux-rockchip@lists.infradead.org, robh+dt@kernel.org,
        galak@codeaurora.org, jwerner@chromium.org,
        linux-kernel@vger.kernel.org, Jianqun Xu <jay.xu@rock-chips.com>,
        linux-arm-kernel@lists.infradead.org
From: "Huang, Tao" <huangtao@rock-chips.com>
Message-ID: <571DE803.3010902@rock-chips.com>
Date: Mon, 25 Apr 2016 17:48:51 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:38.0) Gecko/20100101
 Thunderbird/38.6.0
MIME-Version: 1.0
In-Reply-To: <20160421123018.096d4a75@arm.com>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 4493
Lines: 133

Hi, Marc:
On 2016年04月21日 19:30, Marc Zyngier wrote:
> On Thu, 21 Apr 2016 18:47:20 +0800
> "Huang, Tao" <huangtao@rock-chips.com> wrote:
>
>> Hi, Mark:
>> On 2016年04月21日 18:19, Mark Rutland wrote:
>>> On Thu, Apr 21, 2016 at 11:58:12AM +0800, Jianqun Xu wrote:
>>>> +		cpu_l0: cpu@0 {
>>>> +			device_type = "cpu";
>>>> +			compatible = "arm,cortex-a53", "arm,armv8";
>>>> +			reg = <0x0 0x0>;
>>>> +			enable-method = "psci";
>>>> +			#cooling-cells = <2>; /* min followed by max */
>>>> +			clocks = <&cru ARMCLKL>;
>>>> +		};
>>>> +		cpu_b0: cpu@100 {
>>>> +			device_type = "cpu";
>>>> +			compatible = "arm,cortex-a72", "arm,armv8";
>>>> +			reg = <0x0 0x100>;
>>>> +			enable-method = "psci";
>>>> +			#cooling-cells = <2>; /* min followed by max */
>>>> +			clocks = <&cru ARMCLKB>;
>>>> +		};
>>>> +
>>>> +	arm-pmu {
>>>> +		compatible = "arm,armv8-pmuv3";
>>>> +		interrupts = <GIC_PPI 7 IRQ_TYPE_LEVEL_LOW>;
>>>> +	};
>>> This is wrong, and must go. There should be a separate node for the PMU
>>> of each microarchitecture, with the appropriate compatible string to
>>> represent that (see the juno dts).
>> You are right. The first version we wrote is:
>>     pmu_a53 {
>>         compatible = "arm,cortex-a53-pmu";
>>         interrupts = <GIC_PPI 7 IRQ_TYPE_LEVEL_LOW>;
>>         interrupt-affinity = <&cpu_l0>,
>>                      <&cpu_l1>,
>>                      <&cpu_l2>,
>>                      <&cpu_l3>;
>>     };
>>
>>     pmu_a72 {
>>         compatible = "arm,cortex-a72-pmu";
>>         interrupts = <GIC_PPI 7 IRQ_TYPE_LEVEL_LOW>;
>>         interrupt-affinity = <&cpu_b0>,
>>                      <&cpu_b1>;
>>     };
>> but unfortunately, the arm pmu driver do not support PPI in two cluster
>> well,
>> so we have to replace with this implementation.
>>> In this case things are messier as the same PPI number is being used
>>> across clusters. Marc (Cc'd) has been working on PPI partitions, which
>>> should allow us to support that.
>> Great! So what we can do right now? Wait this feature, and delete
>> arm-pmu node?
> I'd rather you have a look at the patches, test them with your HW,
> and comment on what doesn't work!
>
> You can find the patches over there:
>
> https://lkml.org/lkml/2016/4/11/182
>
> and on the following branch:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/maz/arm-platforms.git
> irq/percpu-partition

I tested these patches. Because our kernel is based on v4.4, so I back
port most changes about
include/linux/irqdomain.h
kernel/irq/irqdomain.c
drivers/irqchip/irq-gic-v3.c
and change rk3399.dtsi base on your arm,gic-v3.txt:

     gic: interrupt-controller@fee00000 {
         compatible = "arm,gic-v3";
-        #interrupt-cells = <3>;
+        #interrupt-cells = <4>;
         #address-cells = <2>;
         #size-cells = <2>;
...
+
+        ppi-partitions {
+            part0: interrupt-partition-0 {
+                affinity = <&cpu_l0 &cpu_l1 &cpu_l2 &cpu_l3>;
+            };
+
+            part1: interrupt-partition-1 {
+                affinity = <&cpu_b0 &cpu_b1>;
+            };
+        };

and change every interrupts from three cells to four cells, such as
     saradc: saradc@ff100000 {
         compatible = "rockchip,rk3399-saradc";
         reg = <0x0 0xff100000 0x0 0x100>;
-        interrupts = <GIC_SPI 62 IRQ_TYPE_LEVEL_HIGH>;
+        interrupts = <GIC_SPI 62 IRQ_TYPE_LEVEL_HIGH 0>;
         #io-channel-cells = <1>;
         clocks = <&cru SCLK_SARADC>, <&cru PCLK_SARADC>;
         clock-names = "saradc", "apb_pclk";

and pmu define as:
    pmu_a53 {
        compatible = "arm,cortex-a53-pmu";
        interrupts = <GIC_PPI 7 IRQ_TYPE_LEVEL_LOW &part0>;
        interrupt-affinity = <&cpu_l0>,
                     <&cpu_l1>,
                     <&cpu_l2>,
                     <&cpu_l3>;
    };

    pmu_a72 {
        compatible = "arm,cortex-a72-pmu", "arm,cortex-a57-pmu";
        interrupts = <GIC_PPI 7 IRQ_TYPE_LEVEL_LOW &part1>;
        interrupt-affinity = <&cpu_b0>,
                     <&cpu_b1>;
    };

It can boot. And I test with Android simpleperf stat and perf top, it works!
So these patches work on RK3399.

But as I mentioned, we must change every interrupt in dts, do you think
this is acceptable?
>
> Of course, you'll have to hack a bit in the PMU code to make it
> understand per-PMU affinity together with percpu interrupts, but it
> wouldn't be fun if there was nothing to do...
I don't change drivers/perf/arm_pmu.c, it just work.

Thanks,
Huang Tao