Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <4ac1b046-b3bc-6090-f03a-eb6352f52a5a@amd.com>
Date:   Thu, 24 Nov 2022 16:11:22 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.13.1
Subject: Re: [PATCH] drm/amdgpu: add mb for si
Content-Language: en-US
To:     "Quan, Evan" <Evan.Quan@amd.com>,
        =?UTF-8?B?5p2O55yf6IO9?= <lizhenneng@kylinos.cn>,
        =?UTF-8?Q?Michel_D=c3=a4nzer?= <michel.daenzer@mailbox.org>,
        "Koenig, Christian" <Christian.Koenig@amd.com>,
        "Deucher, Alexander" <Alexander.Deucher@amd.com>
Cc:     "dri-devel@lists.freedesktop.org" <dri-devel@lists.freedesktop.org>,
        "Pan, Xinhui" <Xinhui.Pan@amd.com>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "amd-gfx@lists.freedesktop.org" <amd-gfx@lists.freedesktop.org>
References: <20221118074810.380368-1-lizhenneng@kylinos.cn>
 <ecd9d251-8941-b2db-71b2-e4ac06f860a3@amd.com>
 <800a1207-8ff6-4cfa-60f3-6ff456874890@mailbox.org>
 <4adc40d1-e775-c480-bb35-23fe9c63e05e@kylinos.cn>
 <DM6PR12MB26196C35AC2F28FD50ECCAF0E40F9@DM6PR12MB2619.namprd12.prod.outlook.com>
From:   "Lazar, Lijo" <lijo.lazar@amd.com>
In-Reply-To: <DM6PR12MB26196C35AC2F28FD50ECCAF0E40F9@DM6PR12MB2619.namprd12.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?QStzRFZNckNjdmF3dFNJbU55TUwzWitYN05zVjc0OVZsOXlObzVDNEJIOTFE?=
 =?utf-8?B?bUlsY0lDL1lwQXNpdXVmU0FGK3dHdEl3L2xlMnpPQlI1cW9RdUVwbVI4SVZV?=
 =?utf-8?B?aC9CeVZZbzh3WDJOckViQ1l1ZGlNK05xY21jZ0JOenZNRVFtMU53WUZGZlAx?=
 =?utf-8?B?Zk1rbWp4a1lqb0dJR1gwM1VtaVl1RWVydUhLd21xZ1Z5eTFZamdLdWdNczhM?=
 =?utf-8?B?SlRMWmUyZnR3M2xhZkNSekFDZlJOajNFZWdpQjN5cWNPTktXZlh0MU9JTEY2?=
 =?utf-8?B?cFBQQjBHSmRya2pab2xyQWJHMEVJd2c0MVBqajdVM3RlT0szM0w2bWFNdzRR?=
 =?utf-8?B?K0IrZmI2eE4wYi93dkY0cUtiVFNKVlA4RDh6Nlo2UWtmMU5xUERTNWdkM2p1?=
 =?utf-8?B?VkdHV0VaOWtYR3ErV1NHeG9GZTFQb3ptYlZkMXN3WEJ2MlQyTFh4emFSZTVz?=
 =?utf-8?B?UFdsa0NXdXYxR1p3NFhlTHN3LzZ3M0FsaVpwRFBJRkllU25MZTEvTTF2UnFu?=
 =?utf-8?B?T0YxOWNoRmxCWW1XRElZM3VVdmY4S2NMYlRkQmNxZi9XeHpYamRtSk5obE5a?=
 =?utf-8?B?NkVNVGoyUHlSR1BUS0pLRGNKeE92TkpNZm1NVHJhSG1ra0E0VlN0bENpYXMr?=
 =?utf-8?B?cFhkaHpocEFHSUsrMTBtZTZwUXpTNitTVTU2ejY3UmYvTmNjcFJCVG91MFk2?=
 =?utf-8?B?Vjd0MWw1R3Mvb2xGc1liNkZhdUV2TkNRR3dlUFhHUzZoUmM5RytLVS9oMWF3?=
 =?utf-8?B?TTFGaWVTcEM5dElTM0tjRHk4VFl2eDFRRzh3c1BvWC9CK2ttWkV4djZaZUdO?=
 =?utf-8?B?YXZoUUJDU2x2UktZK0dBS2EzRlBaamwrcXFRMnZTdmZpVzFxczc5WW9nYWpY?=
 =?utf-8?B?RzFmbUlNVVJ1MEVRbVVyb2NHSCtMSGFJd2l3RlQyTFZUK0xEcnpPZTlQV3dn?=
 =?utf-8?B?QjFTN3BuRENzZzdyM2tHd08yd3VGWG1VM0VEY1dhL3h5WU96cUdOOVZBV2dH?=
 =?utf-8?B?K0JBZlg2QzZWdndiV0ZyKytZMzJCUXNlMXByQjB0YkFxcnJIU3pHTzg2bHVj?=
 =?utf-8?B?dTRMZ0U0RmQ4eU1XenY2bldmR3E4b3Q4NU5BR0dhRy92cEU5VUFrOUZXcWRQ?=
 =?utf-8?B?a21LcTc0OUtXSTc4a21WNkJSdVFuSjZhcFhPck9JTjFPWGJBc1V3MDNSai9Z?=
 =?utf-8?B?N1o5NTlMWVVrSjFFT0xtcGF2RnlHOXNQeDQ2Z2ZOb1NnNkd4bmVFM0tFYjhK?=
 =?utf-8?B?c2lFbVg2bVI4bk9iRGVZSndIMEI1RytTenVvNzFUbm5WNG15bHFtTHIxWlpD?=
 =?utf-8?B?Z2E2VVJmeXNjZ0Q4TFFXSXhIdExhMklzUDY1MmpUNnVSdzM3VFJMcUNvT0FU?=
 =?utf-8?B?YjdpRnZGdnh0K1dNTXZkL2VQRjF4Tmt0QTAyTHNMZlNzcUpENW9VSkZrYTVz?=
 =?utf-8?B?SWRhVEVMQ09QanhubjMrYkVwSUNQTjNjWU5TQ2I4MktTaHpYZWx6TmRvVTlL?=
 =?utf-8?B?VExBdnpRV3pUWDJYdUNCRGd6N0lOWnpYcDF4SEVVK2kvbzgraUdXR1d5V2lW?=
 =?utf-8?B?VjQ1K0VwQ3lXYUtXTW1RcGRXUzhCbm1HNUk0Wjg2MVF6STR5L2RMY3JnMk9p?=
 =?utf-8?B?eUQ3MzU5WGpzQW9xbklFbithbVVJTHcwOW1zbFA4aWI3YlVZZXdyLzh5SlVK?=
 =?utf-8?B?U2RmQ1ZLUWZHK0EyZ2M1MHZZZnNiYWlsU0V6Ui83V3lZUmgxM0ZnR1BpTkRu?=
 =?utf-8?B?LzFEb1FFSS8vSXByU29xSXpsa0lZRmhhNHZ4RGYzTFNyOWVaQjU5em56K0xs?=
 =?utf-8?B?eTZ0WjlJUGdudUtYdE0wSVUyZ0NUYTc4OG5NMmZCUCtGdWY3QXJnRk9GcGE4?=
 =?utf-8?B?ckRTWmNvMGd4RlFWbzhzcnlGRUsxY21mSUpib2UvangrSkdKWVphSzdkUGRY?=
 =?utf-8?B?VHJlcVZQUjJWZ3h2SGNTVjhxcHhvQkEyQjUyN0QwOEJQT20rM0p2eWtSNkNZ?=
 =?utf-8?B?TVR1WkYwSmlaMkY0OXBwdEVtUGUwRUxvMWlEeW01eXlORGNDV2doejZxQzkw?=
 =?utf-8?B?c0ZZNys2QS9EWXN1TXMrZExlNmY1eDJLckRrUURJVFE1Ti9xbkpMdFpsTURH?=
 =?utf-8?Q?As3/z7nKVHmsxkzmBWg+yZMi9?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 760fca32-723a-4632-bc67-08dace0874b1
X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB4621.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Nov 2022 10:41:34.3274
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: p5GceJUq+ZmhruzIa01H707b+/ZR25o8JhGhcWfcCR5jdc8eKE1TLgfUXnjXEfOj
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB8168
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 11/24/2022 3:34 PM, Quan, Evan wrote:
> [AMD Official Use Only - General]
> 
> Could the attached patch help?
> 
> Evan
>> -----Original Message-----
>> From: amd-gfx <amd-gfx-bounces@lists.freedesktop.org> On Behalf Of ???
>> Sent: Friday, November 18, 2022 5:25 PM
>> To: Michel Dänzer <michel.daenzer@mailbox.org>; Koenig, Christian
>> <Christian.Koenig@amd.com>; Deucher, Alexander
>> <Alexander.Deucher@amd.com>
>> Cc: amd-gfx@lists.freedesktop.org; Pan, Xinhui <Xinhui.Pan@amd.com>;
>> linux-kernel@vger.kernel.org; dri-devel@lists.freedesktop.org
>> Subject: Re: [PATCH] drm/amdgpu: add mb for si
>>
>>
>> 在 2022/11/18 17:18, Michel Dänzer 写道:
>>> On 11/18/22 09:01, Christian König wrote:
>>>> Am 18.11.22 um 08:48 schrieb Zhenneng Li:
>>>>> During reboot test on arm64 platform, it may failure on boot, so add
>>>>> this mb in smc.
>>>>>
>>>>> The error message are as follows:
>>>>> [    6.996395][ 7] [  T295] [drm:amdgpu_device_ip_late_init
>>>>> [amdgpu]] *ERROR*
>>>>>                   late_init of IP block <si_dpm> failed -22 [
>>>>> 7.006919][ 7] [  T295] amdgpu 0000:04:00.0:

The issue is happening in late_init() which eventually does

	ret = si_thermal_enable_alert(adev, false);

Just before this, si_thermal_start_thermal_controller is called in 
hw_init and that enables thermal alert.

Maybe the issue is with enable/disable of thermal alerts in quick 
succession. Adding a delay inside si_thermal_start_thermal_controller 
might help.

Thanks,
Lijo

>>>>> amdgpu_device_ip_late_init failed [    7.014224][ 7] [  T295] amdgpu
>>>>> 0000:04:00.0: Fatal error during GPU init
>>>> Memory barries are not supposed to be sprinkled around like this, you
>> need to give a detailed explanation why this is necessary.
>>>>
>>>> Regards,
>>>> Christian.
>>>>
>>>>> Signed-off-by: Zhenneng Li <lizhenneng@kylinos.cn>
>>>>> ---
>>>>>     drivers/gpu/drm/amd/pm/legacy-dpm/si_smc.c | 2 ++
>>>>>     1 file changed, 2 insertions(+)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/amd/pm/legacy-dpm/si_smc.c
>>>>> b/drivers/gpu/drm/amd/pm/legacy-dpm/si_smc.c
>>>>> index 8f994ffa9cd1..c7656f22278d 100644
>>>>> --- a/drivers/gpu/drm/amd/pm/legacy-dpm/si_smc.c
>>>>> +++ b/drivers/gpu/drm/amd/pm/legacy-dpm/si_smc.c
>>>>> @@ -155,6 +155,8 @@ bool amdgpu_si_is_smc_running(struct
>>>>> amdgpu_device *adev)
>>>>>         u32 rst = RREG32_SMC(SMC_SYSCON_RESET_CNTL);
>>>>>         u32 clk = RREG32_SMC(SMC_SYSCON_CLOCK_CNTL_0);
>>>>>     +    mb();
>>>>> +
>>>>>         if (!(rst & RST_REG) && !(clk & CK_DISABLE))
>>>>>             return true;
>>> In particular, it makes no sense in this specific place, since it cannot directly
>> affect the values of rst & clk.
>>
>> I thinks so too.
>>
>> But when I do reboot test using nine desktop machines,  there maybe report
>> this error on one or two machines after Hundreds of times or Thousands of
>> times reboot test, at the beginning, I use msleep() instead of mb(), these
>> two methods are all works, but I don't know what is the root case.
>>
>> I use this method on other verdor's oland card, this error message are
>> reported again.
>>
>> What could be the root reason?
>>
>> test environmen:
>>
>> graphics card: OLAND 0x1002:0x6611 0x1642:0x1869 0x87
>>
>> driver: amdgpu
>>
>> os: ubuntu 2004
>>
>> platform: arm64
>>
>> kernel: 5.4.18
>>
>>>