Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [PATCH] devfreq: exynos-bus: workaround dev_pm_opp_set_rate()
 errors on Exynos5422/5800 SoCs
From:   Chanwoo Choi <cw00.choi@samsung.com>
To:     Kamil Konieczny <k.konieczny@samsung.com>,
        Marek Szyprowski <m.szyprowski@samsung.com>,
        k.konieczny@partner.samsung.com
Cc:     Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>,
        MyungJoo Ham <myungjoo.ham@samsung.com>,
        Kyungmin Park <kyungmin.park@samsung.com>,
        Kukjin Kim <kgene@kernel.org>,
        Krzysztof Kozlowski <krzk@kernel.org>,
        linux-pm@vger.kernel.org, linux-samsung-soc@vger.kernel.org,
        linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org
Organization: Samsung Electronics
Message-ID: <635904ed-93e1-944b-9317-8c9a19844223@samsung.com>
Date:   Thu, 14 Nov 2019 16:38:40 +0900
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
        Thunderbird/60.9.0
MIME-Version: 1.0
In-Reply-To: <9e0a4aa6-46a6-3ca6-42db-11ed55b468dd@samsung.com>
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset="utf-8"
CMS-TYPE: 101P
DLP-Filter: Pass
References: <CGME20191008134950eucas1p15cfef5800efc10d5b18ec5eb37dde60b@eucas1p1.samsung.com>
        <20191008134923.30123-1-k.konieczny@partner.samsung.com>
        <4f14d3af-e455-d05b-fc03-cba58e001f41@samsung.com>
        <0ce56e65-d989-18f8-af84-2fbd74ba20aa@samsung.com>
        <d742e7be-ca79-ae9e-6cc2-dc1fae08d252@samsung.com>
        <dd5bc937-e776-f717-1cf3-ee0e17621304@samsung.com>
        <9e0a4aa6-46a6-3ca6-42db-11ed55b468dd@samsung.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hi Kamil,

On 11/14/19 3:07 PM, Chanwoo Choi wrote:
> Hi Kamil,
> 
> On 11/14/19 12:12 AM, Kamil Konieczny wrote:
>> Hi Chanwoo,
>>
>> On 14.10.2019 08:46, Chanwoo Choi wrote:
>>> Hi Marek,
>>>
>>> On 19. 10. 11. 오후 8:33, Marek Szyprowski wrote:
>>>> Hi Chanwoo,
>>>>
>>>> On 10.10.2019 04:50, Chanwoo Choi wrote:
>>>>> On 2019년 10월 08일 22:49, k.konieczny@partner.samsung.com wrote:
>>>>>> Commit 4294a779bd8d ("PM / devfreq: exynos-bus: Convert to use
>>>>>> dev_pm_opp_set_rate()") introduced errors:
>>>>>> exynos-bus: new bus device registered: soc:bus_wcore ( 84000 KHz ~ 400000 KHz)
>>>>>> exynos-bus: new bus device registered: soc:bus_noc ( 67000 KHz ~ 100000 KHz)
>>>>>> exynos-bus: new bus device registered: soc:bus_fsys_apb (100000 KHz ~ 200000 KHz)
>>>>>> ...
>>>>>> exynos-bus soc:bus_wcore: dev_pm_opp_set_rate: failed to find current OPP for freq 532000000 (-34)
>>>>>> exynos-bus soc:bus_noc: dev_pm_opp_set_rate: failed to find current OPP for freq 111000000 (-34)
>>>>>> exynos-bus soc:bus_fsys_apb: dev_pm_opp_set_rate: failed to find current OPP for freq 222000000 (-34)
>>>>>>
>>>>>> They are caused by incorrect PLL assigned to clock source, which results
>>>>>> in clock rate outside of OPP range. Add workaround for this in
>>>>>> exynos_bus_parse_of() by adjusting clock rate to those present in OPP.
>>>>> If the clock caused this issue, you can set the initial clock on DeviceTree
>>>>> with assigned-clock-* properties. Because the probe time of clock driver
>>>>> is early than the any device drivers.
>>>>>
>>>>> It is not proper to fix the clock issue on other device driver.
>>>>> I think you can fix it by using the supported clock properties.
>>>>
>>>> This issue is about something completely different. The OPPs defined in 
>>>> DT cannot be applied, because it is not possible to derive the needed 
>>>> clock rate from the bootloader-configured clock topology (mainly due to 
>>>> lack of common divisor values for some of the parent clocks). Some time 
>>>> ago Lukasz tried initially to redefine this clock topology using 
>>>> assigned-clock-rates/parents properties (see 
>>>> https://protect2.fireeye.com/url?k=4b80c0304459bc8e.4b814b7f-f87f1e1aee1a85c0&u=https://lkml.org/lkml/2019/7/15/276), but it has limitations and some 
>>>> such changes has to be done in bootloader. Until this is resolved, 
>>>> devfreq simply cannot set some of the defined OPPs.
>>>
>>> As you mentioned, the wrong setting in bootloader cause the this issue.
>>> So, this patch change the rate on exynos-bus.c in order to fix
>>> the issue with workaround style. 
>>>
>>> But, also, it can be fixed by initializing the clock rate on DT
>>> although it is not fundamental solution as you mentioned.
>>>
>>> If above two method are workaround way, I think that set the clock
>>> rate in DT is proper. The role of 'assigned-clock-*' properties
>>> is for this case in order to set the initial frequency on probe time.
>>
>> I can add 'assigned-clock-*' to DT, but the issue is caused in opp points,
>> so the warning from exynos-bus will still be there.
>>
>> Before this fix, devfreq will issue warning and then change clock to max
>> frequency within opp range. This fix mask warning, and as Marek and
>> Lukasz Luba wrotes, the proper fix will be to make changes in u-boot
>> (and connect proper PLLs to IPs).
> 
> PLL could be changed by clock device driver in the linux kernel.
> If you don't add the supported frequency into PLL frequency table 
> of clock device driver, will fail to change the wanted frequency
> on the linux kernel. It means that it is not fixed by only touching
> the bootloader. 
> 
> As you commented, the wrong opp points which are specified on dt
> cause this issue. Usually, have to initialize the clock rate on dt
> by using 'assigned-clocks-*' property and then use the clock
> with the preferable clock rate. I think that we have to fix
> the fundamental problem. 
> 
> Without bootloader problem, you can fix it by initializing
> the clock on dt with 'assigned-clocks-*' property.
> 
> As I knew that it is correct way and I always tried to do this method
> for resolving the similar clock issue.
> 
> Lastly, I think that my opinion is more simple and correct.
> It could give the more correct information to linux kernel user
> which refer to the device tree file.
> 
> 1. Your suggestion 
> 	a. Add opp-table with unsupported frequency on dt
> 	b. Try to change the clock rate on exynos-bus.c by using unsupported frequency from opp-table
> 	c. If failed, retry to change the clock rate on exynos-bus.c
> 
> 2. My opinion
> 	a. Initialize the PLL or any clock by using assigned-clock-* property on dt
> 	   and add opp-table with supported frequency on dt
> 	b. Try to change the clock rate on exynos-bus.c by using supported frequency from opp-table
> 

Just I tried to add 'assigned-clock-rates' property to initialize
the clock rate of some bus node as following on odroid-xu3 board:

diff --git a/arch/arm/boot/dts/exynos5422-odroid-core.dtsi b/arch/arm/boot/dts/exynos5422-odroid-core.dtsi
index 829147e320e0..9a237af5436a 100644
--- a/arch/arm/boot/dts/exynos5422-odroid-core.dtsi
+++ b/arch/arm/boot/dts/exynos5422-odroid-core.dtsi
@@ -42,6 +42,8 @@
 };
 
 &bus_wcore {
+       assigned-clocks = <&clock CLK_DOUT_ACLK400_WCORE>;
+       assigned-clock-rates = <400000000>;
        devfreq-events = <&nocp_mem0_0>, <&nocp_mem0_1>,
                        <&nocp_mem1_0>, <&nocp_mem1_1>;
        vdd-supply = <&buck3_reg>;
@@ -50,11 +52,15 @@
 };
 
 &bus_noc {
+       assigned-clocks = <&clock CLK_DOUT_ACLK100_NOC>;
+       assigned-clock-rates = <100000000>;
        devfreq = <&bus_wcore>;
        status = "okay";
 };
 
 &bus_fsys_apb {
+       assigned-clocks = <&clock CLK_DOUT_PCLK200_FSYS>;
+       assigned-clock-rates = <200000000>;
        devfreq = <&bus_wcore>;
        status = "okay";
 };
@@ -120,6 +126,8 @@
 };
 
 &bus_mscl {
+       assigned-clocks = <&clock CLK_DOUT_ACLK400_MSCL>;
+       assigned-clock-rates = <400000000>;
        devfreq = <&bus_wcore>;
        status = "okay";
 };


In result,
[Before on v5.4-rc6, failed to set the rate by dev_pm_opp_set_rate()]
[    4.855811] exynos-bus: new bus device registered: soc:bus_wcore ( 84000 KHz ~ 400000 KHz)
[    4.863374] exynos-bus: new bus device registered: soc:bus_noc ( 67000 KHz ~ 100000 KHz)
[    4.871240] exynos-bus: new bus device registered: soc:bus_fsys_apb (100000 KHz ~ 200000 KHz)
[    4.879509] exynos-bus: new bus device registered: soc:bus_fsys (100000 KHz ~ 200000 KHz)
[    4.887957] exynos-bus: new bus device registered: soc:bus_fsys2 ( 75000 KHz ~ 150000 KHz)
[    4.896361] exynos-bus: new bus device registered: soc:bus_mfc ( 96000 KHz ~ 333000 KHz)
[    4.904330] exynos-bus: new bus device registered: soc:bus_gen ( 89000 KHz ~ 267000 KHz)
[    4.911802] exynos-bus soc:bus_wcore: dev_pm_opp_set_rate: failed to find current OPP for freq 532000000 (-34)
[    4.912710] exynos-bus: new bus device registered: soc:bus_peri ( 67000 KHz ~  67000 KHz)
[    4.924655] exynos-bus soc:bus_noc: dev_pm_opp_set_rate: failed to find current OPP for freq 111000000 (-34)
[    4.932125] exynos-bus: new bus device registered: soc:bus_g2d ( 84000 KHz ~ 333000 KHz)
[    4.939607] exynos-bus soc:bus_fsys_apb: dev_pm_opp_set_rate: failed to find current OPP for freq 222000000 (-34)
[    4.949758] exynos-bus: new bus device registered: soc:bus_g2d_acp ( 67000 KHz ~ 267000 KHz)
[    4.966991] exynos-bus: new bus device registered: soc:bus_jpeg ( 75000 KHz ~ 300000 KHz)
[    4.975136] exynos-bus: new bus device registered: soc:bus_jpeg_apb ( 84000 KHz ~ 167000 KHz)
[    4.983452] exynos-bus: new bus device registered: soc:bus_disp1_fimd (120000 KHz ~ 200000 KHz)
[    4.992218] exynos-bus: new bus device registered: soc:bus_disp1 (120000 KHz ~ 300000 KHz)
[    5.000483] exynos-bus: new bus device registered: soc:bus_gscl_scaler (150000 KHz ~ 300000 KHz)
[    5.009331] exynos-bus: new bus device registered: soc:bus_mscl ( 84000 KHz ~ 400000 KHz)
[    5.020207] exynos-bus soc:bus_mscl: dev_pm_opp_set_rate: failed to find current OPP for freq 666000000 (-34)

[After applied the 'assigned-clock-*' patch on v5.4-rc6]
[    4.840571] exynos-bus: new bus device registered: soc:bus_wcore ( 84000 KHz ~ 400000 KHz)
[    4.848099] exynos-bus: new bus device registered: soc:bus_noc ( 67000 KHz ~ 100000 KHz)
[    4.856016] exynos-bus: new bus device registered: soc:bus_fsys_apb (100000 KHz ~ 200000 KHz)
[    4.864307] exynos-bus: new bus device registered: soc:bus_fsys (100000 KHz ~ 200000 KHz)
[    4.872723] exynos-bus: new bus device registered: soc:bus_fsys2 ( 75000 KHz ~ 150000 KHz)
[    4.881124] exynos-bus: new bus device registered: soc:bus_mfc ( 96000 KHz ~ 333000 KHz)
[    4.889147] exynos-bus: new bus device registered: soc:bus_gen ( 89000 KHz ~ 267000 KHz)
[    4.896867] exynos-bus: new bus device registered: soc:bus_peri ( 67000 KHz ~  67000 KHz)
[    4.907430] exynos-bus: new bus device registered: soc:bus_g2d ( 84000 KHz ~ 333000 KHz)
[    4.914797] exynos-bus: new bus device registered: soc:bus_g2d_acp ( 67000 KHz ~ 267000 KHz)
[    4.923205] exynos-bus: new bus device registered: soc:bus_jpeg ( 75000 KHz ~ 300000 KHz)
[    4.931352] exynos-bus: new bus device registered: soc:bus_jpeg_apb ( 84000 KHz ~ 167000 KHz)
[    4.939658] exynos-bus: new bus device registered: soc:bus_disp1_fimd (120000 KHz ~ 200000 KHz)
[    4.948401] exynos-bus: new bus device registered: soc:bus_disp1 (120000 KHz ~ 300000 KHz)
[    4.956650] exynos-bus: new bus device registered: soc:bus_gscl_scaler (150000 KHz ~ 300000 KHz)
[    4.965573] exynos-bus: new bus device registered: soc:bus_mscl ( 84000 KHz ~ 400000 KHz)


>>
>> Second solution would be to write down new OPP points with currently used
>> frequencies, and with max one for 532 MHz.
>>
>>> I think that the previous patch[1] of Kamil Konieczny is missing
>>> the patches which initialize the clock rate on DT file.
>>> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4294a779bd8dff6c65e7e85ffe7a1ea236e92a68
>>>
>>>>
>>>> This issue was there from the beginning, recent Kamil's patch only 
>>>> revealed it. In fact it was even worse - devfreq and common clock 
>>>> framework silently set lower clock than the given OPP defined.
>>>>
>>>>>> Fixes: 4294a779bd8d ("PM / devfreq: exynos-bus: Convert to use dev_pm_opp_set_rate()")
>>>>>> Reported-by: Krzysztof Kozlowski <krzk@kernel.org>
>>>>>> Signed-off-by: Kamil Konieczny <k.konieczny@partner.samsung.com>
>>>>>> ---
>>>>>>   drivers/devfreq/exynos-bus.c | 14 +++++++++++---
>>>>>>   1 file changed, 11 insertions(+), 3 deletions(-)
>>>>>>
>>>>>> diff --git a/drivers/devfreq/exynos-bus.c b/drivers/devfreq/exynos-bus.c
>>>>>> index c832673273a2..37bd34d5625b 100644
>>>>>> --- a/drivers/devfreq/exynos-bus.c
>>>>>> +++ b/drivers/devfreq/exynos-bus.c
>>>>>> @@ -243,7 +243,7 @@ static int exynos_bus_parse_of(struct device_node *np,
>>>>>>   {
>>>>>>   	struct device *dev = bus->dev;
>>>>>>   	struct dev_pm_opp *opp;
>>>>>> -	unsigned long rate;
>>>>>> +	unsigned long rate, opp_rate;
>>>>>>   	int ret;
>>>>>>   
>>>>>>   	/* Get the clock to provide each bus with source clock */
>>>>>> @@ -267,13 +267,21 @@ static int exynos_bus_parse_of(struct device_node *np,
>>>>>>   	}
>>>>>>   
>>>>>>   	rate = clk_get_rate(bus->clk);
>>>>>> -
>>>>>> -	opp = devfreq_recommended_opp(dev, &rate, 0);
>>>>>> +	opp_rate = rate;
>>>>>> +	opp = devfreq_recommended_opp(dev, &opp_rate, 0);
>>>>>>   	if (IS_ERR(opp)) {
>>>>>>   		dev_err(dev, "failed to find dev_pm_opp\n");
>>>>>>   		ret = PTR_ERR(opp);
>>>>>>   		goto err_opp;
>>>>>>   	}
>>>>>> +	/*
>>>>>> +	 * FIXME: U-boot leaves clock source at incorrect PLL, this results
>>>>>> +	 * in clock rate outside defined OPP rate. Work around this bug by
>>>>>> +	 * setting clock rate to recommended one.
>>>>>> +	 */
>>>>>> +	if (rate > opp_rate)
>>>>>> +		clk_set_rate(bus->clk, opp_rate);
>>>>>> +
>>>>>>   	bus->curr_freq = dev_pm_opp_get_freq(opp);
>>>>>>   	dev_pm_opp_put(opp);
>>>>>>   
>>>>>>
>>>>>
>>>> Best regards
>>>>
>>>
>>>
>>
> 
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics