Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [PATCH] devfreq: exynos-bus: workaround dev_pm_opp_set_rate()
 errors on Exynos5422/5800 SoCs
From:   Chanwoo Choi <cw00.choi@samsung.com>
To:     Kamil Konieczny <k.konieczny@samsung.com>,
        Marek Szyprowski <m.szyprowski@samsung.com>
Cc:     Bartlomiej Zolnierkiewicz <b.zolnierkie@samsung.com>,
        MyungJoo Ham <myungjoo.ham@samsung.com>,
        Kyungmin Park <kyungmin.park@samsung.com>,
        Kukjin Kim <kgene@kernel.org>,
        Krzysztof Kozlowski <krzk@kernel.org>,
        linux-pm@vger.kernel.org, linux-samsung-soc@vger.kernel.org,
        linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org
Organization: Samsung Electronics
Message-ID: <99bc2b8a-8977-8020-f3d9-faac93416cd8@samsung.com>
Date:   Thu, 5 Dec 2019 11:48:47 +0900
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:59.0) Gecko/20100101
        Thunderbird/59.0
MIME-Version: 1.0
In-Reply-To: <635904ed-93e1-944b-9317-8c9a19844223@samsung.com>
Content-Language: en-US
Content-Transfer-Encoding: 8bit
Content-Type: text/plain; charset="utf-8"
CMS-TYPE: 101P
DLP-Filter: Pass
References: <CGME20191008134950eucas1p15cfef5800efc10d5b18ec5eb37dde60b@eucas1p1.samsung.com>
        <20191008134923.30123-1-k.konieczny@partner.samsung.com>
        <4f14d3af-e455-d05b-fc03-cba58e001f41@samsung.com>
        <0ce56e65-d989-18f8-af84-2fbd74ba20aa@samsung.com>
        <d742e7be-ca79-ae9e-6cc2-dc1fae08d252@samsung.com>
        <dd5bc937-e776-f717-1cf3-ee0e17621304@samsung.com>
        <9e0a4aa6-46a6-3ca6-42db-11ed55b468dd@samsung.com>
        <635904ed-93e1-944b-9317-8c9a19844223@samsung.com>
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hi Kamil,

On 11/14/19 4:38 PM, Chanwoo Choi wrote:
> Hi Kamil,
> 
> On 11/14/19 3:07 PM, Chanwoo Choi wrote:
>> Hi Kamil,
>>
>> On 11/14/19 12:12 AM, Kamil Konieczny wrote:
>>> Hi Chanwoo,
>>>
>>> On 14.10.2019 08:46, Chanwoo Choi wrote:
>>>> Hi Marek,
>>>>
>>>> On 19. 10. 11. 오후 8:33, Marek Szyprowski wrote:
>>>>> Hi Chanwoo,
>>>>>
>>>>> On 10.10.2019 04:50, Chanwoo Choi wrote:
>>>>>> On 2019년 10월 08일 22:49, k.konieczny@partner.samsung.com wrote:
>>>>>>> Commit 4294a779bd8d ("PM / devfreq: exynos-bus: Convert to use
>>>>>>> dev_pm_opp_set_rate()") introduced errors:
>>>>>>> exynos-bus: new bus device registered: soc:bus_wcore ( 84000 KHz ~ 400000 KHz)
>>>>>>> exynos-bus: new bus device registered: soc:bus_noc ( 67000 KHz ~ 100000 KHz)
>>>>>>> exynos-bus: new bus device registered: soc:bus_fsys_apb (100000 KHz ~ 200000 KHz)
>>>>>>> ...
>>>>>>> exynos-bus soc:bus_wcore: dev_pm_opp_set_rate: failed to find current OPP for freq 532000000 (-34)
>>>>>>> exynos-bus soc:bus_noc: dev_pm_opp_set_rate: failed to find current OPP for freq 111000000 (-34)
>>>>>>> exynos-bus soc:bus_fsys_apb: dev_pm_opp_set_rate: failed to find current OPP for freq 222000000 (-34)
>>>>>>>
>>>>>>> They are caused by incorrect PLL assigned to clock source, which results
>>>>>>> in clock rate outside of OPP range. Add workaround for this in
>>>>>>> exynos_bus_parse_of() by adjusting clock rate to those present in OPP.
>>>>>> If the clock caused this issue, you can set the initial clock on DeviceTree
>>>>>> with assigned-clock-* properties. Because the probe time of clock driver
>>>>>> is early than the any device drivers.
>>>>>>
>>>>>> It is not proper to fix the clock issue on other device driver.
>>>>>> I think you can fix it by using the supported clock properties.
>>>>>
>>>>> This issue is about something completely different. The OPPs defined in 
>>>>> DT cannot be applied, because it is not possible to derive the needed 
>>>>> clock rate from the bootloader-configured clock topology (mainly due to 
>>>>> lack of common divisor values for some of the parent clocks). Some time 
>>>>> ago Lukasz tried initially to redefine this clock topology using 
>>>>> assigned-clock-rates/parents properties (see 
>>>>> https://protect2.fireeye.com/url?k=4b80c0304459bc8e.4b814b7f-f87f1e1aee1a85c0&u=https://lkml.org/lkml/2019/7/15/276), but it has limitations and some 
>>>>> such changes has to be done in bootloader. Until this is resolved, 
>>>>> devfreq simply cannot set some of the defined OPPs.
>>>>
>>>> As you mentioned, the wrong setting in bootloader cause the this issue.
>>>> So, this patch change the rate on exynos-bus.c in order to fix
>>>> the issue with workaround style. 
>>>>
>>>> But, also, it can be fixed by initializing the clock rate on DT
>>>> although it is not fundamental solution as you mentioned.
>>>>
>>>> If above two method are workaround way, I think that set the clock
>>>> rate in DT is proper. The role of 'assigned-clock-*' properties
>>>> is for this case in order to set the initial frequency on probe time.
>>>
>>> I can add 'assigned-clock-*' to DT, but the issue is caused in opp points,
>>> so the warning from exynos-bus will still be there.
>>>
>>> Before this fix, devfreq will issue warning and then change clock to max
>>> frequency within opp range. This fix mask warning, and as Marek and
>>> Lukasz Luba wrotes, the proper fix will be to make changes in u-boot
>>> (and connect proper PLLs to IPs).
>>
>> PLL could be changed by clock device driver in the linux kernel.
>> If you don't add the supported frequency into PLL frequency table 
>> of clock device driver, will fail to change the wanted frequency
>> on the linux kernel. It means that it is not fixed by only touching
>> the bootloader. 
>>
>> As you commented, the wrong opp points which are specified on dt
>> cause this issue. Usually, have to initialize the clock rate on dt
>> by using 'assigned-clocks-*' property and then use the clock
>> with the preferable clock rate. I think that we have to fix
>> the fundamental problem. 
>>
>> Without bootloader problem, you can fix it by initializing
>> the clock on dt with 'assigned-clocks-*' property.
>>
>> As I knew that it is correct way and I always tried to do this method
>> for resolving the similar clock issue.
>>
>> Lastly, I think that my opinion is more simple and correct.
>> It could give the more correct information to linux kernel user
>> which refer to the device tree file.
>>
>> 1. Your suggestion 
>> 	a. Add opp-table with unsupported frequency on dt
>> 	b. Try to change the clock rate on exynos-bus.c by using unsupported frequency from opp-table
>> 	c. If failed, retry to change the clock rate on exynos-bus.c
>>
>> 2. My opinion
>> 	a. Initialize the PLL or any clock by using assigned-clock-* property on dt
>> 	   and add opp-table with supported frequency on dt
>> 	b. Try to change the clock rate on exynos-bus.c by using supported frequency from opp-table
>>
> 
> Just I tried to add 'assigned-clock-rates' property to initialize
> the clock rate of some bus node as following on odroid-xu3 board:
> 
> diff --git a/arch/arm/boot/dts/exynos5422-odroid-core.dtsi b/arch/arm/boot/dts/exynos5422-odroid-core.dtsi
> index 829147e320e0..9a237af5436a 100644
> --- a/arch/arm/boot/dts/exynos5422-odroid-core.dtsi
> +++ b/arch/arm/boot/dts/exynos5422-odroid-core.dtsi
> @@ -42,6 +42,8 @@
>  };
>  
>  &bus_wcore {
> +       assigned-clocks = <&clock CLK_DOUT_ACLK400_WCORE>;
> +       assigned-clock-rates = <400000000>;
>         devfreq-events = <&nocp_mem0_0>, <&nocp_mem0_1>,
>                         <&nocp_mem1_0>, <&nocp_mem1_1>;
>         vdd-supply = <&buck3_reg>;
> @@ -50,11 +52,15 @@
>  };
>  
>  &bus_noc {
> +       assigned-clocks = <&clock CLK_DOUT_ACLK100_NOC>;
> +       assigned-clock-rates = <100000000>;
>         devfreq = <&bus_wcore>;
>         status = "okay";
>  };
>  
>  &bus_fsys_apb {
> +       assigned-clocks = <&clock CLK_DOUT_PCLK200_FSYS>;
> +       assigned-clock-rates = <200000000>;
>         devfreq = <&bus_wcore>;
>         status = "okay";
>  };
> @@ -120,6 +126,8 @@
>  };
>  
>  &bus_mscl {
> +       assigned-clocks = <&clock CLK_DOUT_ACLK400_MSCL>;
> +       assigned-clock-rates = <400000000>;
>         devfreq = <&bus_wcore>;
>         status = "okay";
>  };
> 
> 
> In result,
> [Before on v5.4-rc6, failed to set the rate by dev_pm_opp_set_rate()]
> [    4.855811] exynos-bus: new bus device registered: soc:bus_wcore ( 84000 KHz ~ 400000 KHz)
> [    4.863374] exynos-bus: new bus device registered: soc:bus_noc ( 67000 KHz ~ 100000 KHz)
> [    4.871240] exynos-bus: new bus device registered: soc:bus_fsys_apb (100000 KHz ~ 200000 KHz)
> [    4.879509] exynos-bus: new bus device registered: soc:bus_fsys (100000 KHz ~ 200000 KHz)
> [    4.887957] exynos-bus: new bus device registered: soc:bus_fsys2 ( 75000 KHz ~ 150000 KHz)
> [    4.896361] exynos-bus: new bus device registered: soc:bus_mfc ( 96000 KHz ~ 333000 KHz)
> [    4.904330] exynos-bus: new bus device registered: soc:bus_gen ( 89000 KHz ~ 267000 KHz)
> [    4.911802] exynos-bus soc:bus_wcore: dev_pm_opp_set_rate: failed to find current OPP for freq 532000000 (-34)
> [    4.912710] exynos-bus: new bus device registered: soc:bus_peri ( 67000 KHz ~  67000 KHz)
> [    4.924655] exynos-bus soc:bus_noc: dev_pm_opp_set_rate: failed to find current OPP for freq 111000000 (-34)
> [    4.932125] exynos-bus: new bus device registered: soc:bus_g2d ( 84000 KHz ~ 333000 KHz)
> [    4.939607] exynos-bus soc:bus_fsys_apb: dev_pm_opp_set_rate: failed to find current OPP for freq 222000000 (-34)
> [    4.949758] exynos-bus: new bus device registered: soc:bus_g2d_acp ( 67000 KHz ~ 267000 KHz)
> [    4.966991] exynos-bus: new bus device registered: soc:bus_jpeg ( 75000 KHz ~ 300000 KHz)
> [    4.975136] exynos-bus: new bus device registered: soc:bus_jpeg_apb ( 84000 KHz ~ 167000 KHz)
> [    4.983452] exynos-bus: new bus device registered: soc:bus_disp1_fimd (120000 KHz ~ 200000 KHz)
> [    4.992218] exynos-bus: new bus device registered: soc:bus_disp1 (120000 KHz ~ 300000 KHz)
> [    5.000483] exynos-bus: new bus device registered: soc:bus_gscl_scaler (150000 KHz ~ 300000 KHz)
> [    5.009331] exynos-bus: new bus device registered: soc:bus_mscl ( 84000 KHz ~ 400000 KHz)
> [    5.020207] exynos-bus soc:bus_mscl: dev_pm_opp_set_rate: failed to find current OPP for freq 666000000 (-34)
> 
> [After applied the 'assigned-clock-*' patch on v5.4-rc6]
> [    4.840571] exynos-bus: new bus device registered: soc:bus_wcore ( 84000 KHz ~ 400000 KHz)
> [    4.848099] exynos-bus: new bus device registered: soc:bus_noc ( 67000 KHz ~ 100000 KHz)
> [    4.856016] exynos-bus: new bus device registered: soc:bus_fsys_apb (100000 KHz ~ 200000 KHz)
> [    4.864307] exynos-bus: new bus device registered: soc:bus_fsys (100000 KHz ~ 200000 KHz)
> [    4.872723] exynos-bus: new bus device registered: soc:bus_fsys2 ( 75000 KHz ~ 150000 KHz)
> [    4.881124] exynos-bus: new bus device registered: soc:bus_mfc ( 96000 KHz ~ 333000 KHz)
> [    4.889147] exynos-bus: new bus device registered: soc:bus_gen ( 89000 KHz ~ 267000 KHz)
> [    4.896867] exynos-bus: new bus device registered: soc:bus_peri ( 67000 KHz ~  67000 KHz)
> [    4.907430] exynos-bus: new bus device registered: soc:bus_g2d ( 84000 KHz ~ 333000 KHz)
> [    4.914797] exynos-bus: new bus device registered: soc:bus_g2d_acp ( 67000 KHz ~ 267000 KHz)
> [    4.923205] exynos-bus: new bus device registered: soc:bus_jpeg ( 75000 KHz ~ 300000 KHz)
> [    4.931352] exynos-bus: new bus device registered: soc:bus_jpeg_apb ( 84000 KHz ~ 167000 KHz)
> [    4.939658] exynos-bus: new bus device registered: soc:bus_disp1_fimd (120000 KHz ~ 200000 KHz)
> [    4.948401] exynos-bus: new bus device registered: soc:bus_disp1 (120000 KHz ~ 300000 KHz)
> [    4.956650] exynos-bus: new bus device registered: soc:bus_gscl_scaler (150000 KHz ~ 300000 KHz)
> [    4.965573] exynos-bus: new bus device registered: soc:bus_mscl ( 84000 KHz ~ 400000 KHz)
> 

Actually, I don't want to leave this problem on mainline.
We have to need to fix the fail of exynos-bus registration for kernel booting.
Please reply your opinion or send the fix-up patch as I commented above.

> 
>>>
>>> Second solution would be to write down new OPP points with currently used
>>> frequencies, and with max one for 532 MHz.
>>>
>>>> I think that the previous patch[1] of Kamil Konieczny is missing
>>>> the patches which initialize the clock rate on DT file.
>>>> [1] https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/?id=4294a779bd8dff6c65e7e85ffe7a1ea236e92a68
>>>>
>>>>>
>>>>> This issue was there from the beginning, recent Kamil's patch only 
>>>>> revealed it. In fact it was even worse - devfreq and common clock 
>>>>> framework silently set lower clock than the given OPP defined.
>>>>>
>>>>>>> Fixes: 4294a779bd8d ("PM / devfreq: exynos-bus: Convert to use dev_pm_opp_set_rate()")
>>>>>>> Reported-by: Krzysztof Kozlowski <krzk@kernel.org>
>>>>>>> Signed-off-by: Kamil Konieczny <k.konieczny@partner.samsung.com>
>>>>>>> ---
>>>>>>>   drivers/devfreq/exynos-bus.c | 14 +++++++++++---
>>>>>>>   1 file changed, 11 insertions(+), 3 deletions(-)
>>>>>>>
>>>>>>> diff --git a/drivers/devfreq/exynos-bus.c b/drivers/devfreq/exynos-bus.c
>>>>>>> index c832673273a2..37bd34d5625b 100644
>>>>>>> --- a/drivers/devfreq/exynos-bus.c
>>>>>>> +++ b/drivers/devfreq/exynos-bus.c
>>>>>>> @@ -243,7 +243,7 @@ static int exynos_bus_parse_of(struct device_node *np,
>>>>>>>   {
>>>>>>>   	struct device *dev = bus->dev;
>>>>>>>   	struct dev_pm_opp *opp;
>>>>>>> -	unsigned long rate;
>>>>>>> +	unsigned long rate, opp_rate;
>>>>>>>   	int ret;
>>>>>>>   
>>>>>>>   	/* Get the clock to provide each bus with source clock */
>>>>>>> @@ -267,13 +267,21 @@ static int exynos_bus_parse_of(struct device_node *np,
>>>>>>>   	}
>>>>>>>   
>>>>>>>   	rate = clk_get_rate(bus->clk);
>>>>>>> -
>>>>>>> -	opp = devfreq_recommended_opp(dev, &rate, 0);
>>>>>>> +	opp_rate = rate;
>>>>>>> +	opp = devfreq_recommended_opp(dev, &opp_rate, 0);
>>>>>>>   	if (IS_ERR(opp)) {
>>>>>>>   		dev_err(dev, "failed to find dev_pm_opp\n");
>>>>>>>   		ret = PTR_ERR(opp);
>>>>>>>   		goto err_opp;
>>>>>>>   	}
>>>>>>> +	/*
>>>>>>> +	 * FIXME: U-boot leaves clock source at incorrect PLL, this results
>>>>>>> +	 * in clock rate outside defined OPP rate. Work around this bug by
>>>>>>> +	 * setting clock rate to recommended one.
>>>>>>> +	 */
>>>>>>> +	if (rate > opp_rate)
>>>>>>> +		clk_set_rate(bus->clk, opp_rate);
>>>>>>> +
>>>>>>>   	bus->curr_freq = dev_pm_opp_get_freq(opp);
>>>>>>>   	dev_pm_opp_put(opp);
>>>>>>>   
>>>>>>>
>>>>>>
>>>>> Best regards
>>>>>
>>>>
>>>>
>>>
>>
>>
> 
> 


-- 
Best Regards,
Chanwoo Choi
Samsung Electronics