Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <71571bdc-8310-502f-77b5-954f5efbff91@amd.com>
Date:   Thu, 24 Nov 2022 16:19:14 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.13.1
Subject: Re: [PATCH] drm/amdgpu: add mb for si
Content-Language: en-US
From:   "Lazar, Lijo" <lijo.lazar@amd.com>
To:     "Quan, Evan" <Evan.Quan@amd.com>,
        =?UTF-8?B?5p2O55yf6IO9?= <lizhenneng@kylinos.cn>,
        =?UTF-8?Q?Michel_D=c3=a4nzer?= <michel.daenzer@mailbox.org>,
        "Koenig, Christian" <Christian.Koenig@amd.com>,
        "Deucher, Alexander" <Alexander.Deucher@amd.com>
Cc:     "amd-gfx@lists.freedesktop.org" <amd-gfx@lists.freedesktop.org>,
        "Pan, Xinhui" <Xinhui.Pan@amd.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org>
References: <20221118074810.380368-1-lizhenneng@kylinos.cn>
 <ecd9d251-8941-b2db-71b2-e4ac06f860a3@amd.com>
 <800a1207-8ff6-4cfa-60f3-6ff456874890@mailbox.org>
 <4adc40d1-e775-c480-bb35-23fe9c63e05e@kylinos.cn>
 <DM6PR12MB26196C35AC2F28FD50ECCAF0E40F9@DM6PR12MB2619.namprd12.prod.outlook.com>
 <4ac1b046-b3bc-6090-f03a-eb6352f52a5a@amd.com>
In-Reply-To: <4ac1b046-b3bc-6090-f03a-eb6352f52a5a@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ZHdZemw4TmNaczdlME15bUlBV1pnWExHSHFPMDUzTnNPUCtVSlMva2pQYjJS?=
 =?utf-8?B?WGJEY0lLeGx3c0ZrVzVmcVpabHhycXkwUTk3OHlMYW8wNXNJUGlUV0EwTzU3?=
 =?utf-8?B?UkFjUkczL25FWEE2ZmtFZHB5a0k4Vy9ueCtrMEU1ekZOYmdxRGZlL0hGVE4x?=
 =?utf-8?B?T0NONHFuaUorSnZXTDFPTjNCa3JvcVFXd2N3bEkwOWJDWHlERnhlckxLcUUx?=
 =?utf-8?B?eG9vN0NpdEVYc05UeHd2V2V3SE15UkJ2MFhZWk4wYmtIaStUaVZyOFJvVGYw?=
 =?utf-8?B?em8wK21Jb0tidHUyRXhnMjZMWUhvT0ludlEwWnhCRm9GTVhxOXhSZG1Gb0Qz?=
 =?utf-8?B?VlpybzVvR3NpZFQ3YTY5TDNtZUs5c0NvR3R1UGdDaW9nc1BIM1ExT1ZZN1Jx?=
 =?utf-8?B?bElva2t2SmttTXZuOGdqZjdqaG9hb01IbmNCWi9aVlgwelYxYnE0ZitTbUpr?=
 =?utf-8?B?eWtUTU9VenhNV2RpUWZRVFJDcWZQVVlLendPby9aMHUxN2FqVjBtOEZleUF5?=
 =?utf-8?B?aE9sK0o4VXY4VksrSVVnRWJQckNyeEh3Ym9lVmVKeURVbGkwanFTNDBQeEpr?=
 =?utf-8?B?L2NHdHMvN252MXVpQkxPTnR2eG1wWUpDeEZWeVlTYm8zbTUxRXNzancvWnR5?=
 =?utf-8?B?eVVjajBmRkdvd2JtMFJVZGpNSmtMdXhlNWE4MXdFMFRUdXQvTmsvY0grZ3pp?=
 =?utf-8?B?L1JMWFJpYmNhU1RlY01ZbzF3aDAyVzJxZzRmWCtGR2xMNHZacS95YXFaemt4?=
 =?utf-8?B?YXBtaStpNG5ucFFBMzQrZnhhU0g2MVN4MGtHWWNGWjdqVmJaeXNVVXp6b0hh?=
 =?utf-8?B?NDhscjdKWkxLL3ZwbXpob2lTNjFMOHg2SlR5dTluNDlVUU5zOTdidXZEREpO?=
 =?utf-8?B?WGM0eDRsa2NJektVOSsxaktEMk9aT3BCS0dnZTNkYVZuQmk2alJWMUVTNVZa?=
 =?utf-8?B?cThQRSsyR1ZDZitKcVJ2MktDRFQ5VlJIbVVMczE0OVdnODFVUUNadWNyVmpy?=
 =?utf-8?B?K3J5cDB6RFh2Q1BtNUl1VzNkaFc1Q1hSZXJHZE9sb1d5V3gxalp3azZ1UnJ6?=
 =?utf-8?B?N0oybkRjWEoxVmZKKzZLeVc1TXZqSVg0dXQ3SmN6QWtVclR3K3BCenRSYzRr?=
 =?utf-8?B?OTJtUTYzR3JnbHpEOUdMS2pHQW1teE52dWQxbGNyZ2V1NW9WTVNvWmhZTmxS?=
 =?utf-8?B?emlpdUhSSDFZQkUvbU5qVHNxUHlvRXVKRmxlL1YyOVdvVkxNcWNJaG9VSmhP?=
 =?utf-8?B?N1hRcmRBVzNTdGFzMnhudEVHeDQwdzJtUzMya2NhaEFmUmdFcVpUOUltRGJ5?=
 =?utf-8?B?UGtVV0o1amQ2TFl3bHZWVFZHK3hhVzF6czlyaFlHME5kcTVTc3M1VGl6MFFq?=
 =?utf-8?B?UEV5emczbXpOSkM5d3NqbzlwQnprNG5MT1plRzhBZ1I3U0dKTElQVEI5Unc3?=
 =?utf-8?B?YnVlMm5iTVUrMW1OOHV2VzdGMmhCM0ZDRW1sUzVoUGI2b2xxUEdFTWFydjFs?=
 =?utf-8?B?NWw0ZkJzbjFYQ2VnOVhvcHFxN0FtMVVLamJoUmQzZU8xWkRnaEtMcWxwMGZh?=
 =?utf-8?B?WTA3alVkRFE3czRaSjY0VmJVOXVXd3pNWUQwUkp4QTcrMG9JVm5VbnhqcmN4?=
 =?utf-8?B?VlpUMDJhR0E2ZkJNVjNzUEVkNnAzWm9tbnQ2Zzc5cGJMWVNxZ283ekxZUHRh?=
 =?utf-8?B?MlN1VmpUZDFsWEhnbE9kZUlSRkVReEdEcUxWWDZ1Y0FCYXJtMHdKK094VmlV?=
 =?utf-8?B?WHlPTlNyUXNIenJBK09KMzZhaEd1NjZkWVIybmtZTWlNSjdCRFBpRTFGSkQ3?=
 =?utf-8?B?blU5WHJaeGE5aFBUZUtxUXlXVGk3Njh4MkR5SGlPeUFoU1owMDMvazRHeGdS?=
 =?utf-8?B?UitGNitHZVMzc3RWS1M0NkRxWWpxRGNLcVhWZXorL0JBWnZKNkUrMzRWeHJ6?=
 =?utf-8?B?d1VLNGNFMmI0a0tLMDBldDk0UDZEVHR2L2VqVGNWZUIvNDlUR1RNaE5YaDlv?=
 =?utf-8?B?aTg2K3VuTjhjVlpoMFF2Z2YxNWhxUFUybGVQRHdVVFZJYlB0a2ZTbDZGb1Rh?=
 =?utf-8?B?N3pOL2F6bjZUWlYrYldtOE94WUdWK0JOb254Z0NocHZ1cU1KZkNUNHBEQldR?=
 =?utf-8?Q?RTQYA+SCa3hF5b1pYSB1H5HHP?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 2f51232c-8ca2-4888-6c20-08dace098e58
X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4621.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Nov 2022 10:49:26.7814
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: pb/f/Kq0z3nF16day2qEW+HmjE3tmPtXVYgkzfuhGHwAlBmQcpa7l08zmqgqDzb3
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH8PR12MB7279
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 11/24/2022 4:11 PM, Lazar, Lijo wrote:
> 
> 
> On 11/24/2022 3:34 PM, Quan, Evan wrote:
>> [AMD Official Use Only - General]
>>
>> Could the attached patch help?
>>
>> Evan
>>> -----Original Message-----
>>> From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> On Behalf Of ???
>>> Sent: Friday, November 18, 2022 5:25 PM
>>> To: Michel Dänzer <michel.daenzer@mailbox.org>; Koenig, Christian
>>> <Christian.Koenig@amd.com>; Deucher, Alexander
>>> <Alexander.Deucher@amd.com>
>>> Cc: amd-gfx@lists.freedesktop.org; Pan, Xinhui <Xinhui.Pan@amd.com>;
>>> linux-kernel@vger.kernel.org; dri-devel@lists.freedesktop.org
>>> Subject: Re: [PATCH] drm/amdgpu: add mb for si
>>>
>>>
>>> 在 2022/11/18 17:18, Michel Dänzer 写道:
>>>> On 11/18/22 09:01, Christian König wrote:
>>>>> Am 18.11.22 um 08:48 schrieb Zhenneng Li:
>>>>>> During reboot test on arm64 platform, it may failure on boot, so add
>>>>>> this mb in smc.
>>>>>>
>>>>>> The error message are as follows:
>>>>>> [    6.996395][ 7] [  T295] [drm:amdgpu_device_ip_late_init
>>>>>> [amdgpu]] *ERROR*
>>>>>>                   late_init of IP block <si_dpm> failed -22 [
>>>>>> 7.006919][ 7] [  T295] amdgpu 0000:04:00.0:
> 
> The issue is happening in late_init() which eventually does
> 
>      ret = si_thermal_enable_alert(adev, false);
> 
> Just before this, si_thermal_start_thermal_controller is called in 
> hw_init and that enables thermal alert.
> 
> Maybe the issue is with enable/disable of thermal alerts in quick 
> succession. Adding a delay inside si_thermal_start_thermal_controller 
> might help.
> 

On a second look, temperature range is already set as part of 
si_thermal_start_thermal_controller in hw_init
https://elixir.bootlin.com/linux/v6.1-rc6/source/drivers/gpu/drm/amd/pm/legacy-dpm/si_dpm.c#L6780

There is no need to set it again here -

https://elixir.bootlin.com/linux/v6.1-rc6/source/drivers/gpu/drm/amd/pm/legacy-dpm/si_dpm.c#L7635

I think it is safe to remove the call from late_init altogether. Alex/Evan?

Thanks,
Lijo

> Thanks,
> Lijo
> 
>>>>>> amdgpu_device_ip_late_init failed [    7.014224][ 7] [  T295] amdgpu
>>>>>> 0000:04:00.0: Fatal error during GPU init
>>>>> Memory barries are not supposed to be sprinkled around like this, you
>>> need to give a detailed explanation why this is necessary.
>>>>>
>>>>> Regards,
>>>>> Christian.
>>>>>
>>>>>> Signed-off-by: Zhenneng Li <lizhenneng@kylinos.cn>
>>>>>> ---
>>>>>>     drivers/gpu/drm/amd/pm/legacy-dpm/si_smc.c | 2 ++
>>>>>>     1 file changed, 2 insertions(+)
>>>>>>
>>>>>> diff --git a/drivers/gpu/drm/amd/pm/legacy-dpm/si_smc.c
>>>>>> b/drivers/gpu/drm/amd/pm/legacy-dpm/si_smc.c
>>>>>> index 8f994ffa9cd1..c7656f22278d 100644
>>>>>> --- a/drivers/gpu/drm/amd/pm/legacy-dpm/si_smc.c
>>>>>> +++ b/drivers/gpu/drm/amd/pm/legacy-dpm/si_smc.c
>>>>>> @@ -155,6 +155,8 @@ bool amdgpu_si_is_smc_running(struct
>>>>>> amdgpu_device *adev)
>>>>>>         u32 rst = RREG32_SMC(SMC_SYSCON_RESET_CNTL);
>>>>>>         u32 clk = RREG32_SMC(SMC_SYSCON_CLOCK_CNTL_0);
>>>>>>     +    mb();
>>>>>> +
>>>>>>         if (!(rst & RST_REG) && !(clk & CK_DISABLE))
>>>>>>             return true;
>>>> In particular, it makes no sense in this specific place, since it 
>>>> cannot directly
>>> affect the values of rst & clk.
>>>
>>> I thinks so too.
>>>
>>> But when I do reboot test using nine desktop machines,  there maybe 
>>> report
>>> this error on one or two machines after Hundreds of times or 
>>> Thousands of
>>> times reboot test, at the beginning, I use msleep() instead of mb(), 
>>> these
>>> two methods are all works, but I don't know what is the root case.
>>>
>>> I use this method on other verdor's oland card, this error message are
>>> reported again.
>>>
>>> What could be the root reason?
>>>
>>> test environmen:
>>>
>>> graphics card: OLAND 0x1002:0x6611 0x1642:0x1869 0x87
>>>
>>> driver: amdgpu
>>>
>>> os: ubuntu 2004
>>>
>>> platform: arm64
>>>
>>> kernel: 5.4.18
>>>
>>>>