Received-SPF: pass (google.com: domain of linux-kernel+bounces-153578-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Message-ID: <f2e1b969-ce9c-450c-9882-99813b7334a0@amd.com>
Date: Mon, 22 Apr 2024 16:47:36 +0200
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] drm/amdgpu: Fixup bad vram size on gmc v6 and v7
To: Alex Deucher <alexdeucher@gmail.com>
Cc: Qiang Ma <maqianga@uniontech.com>, alexander.deucher@amd.com,
 Xinhui.Pan@amd.com, airlied@gmail.com, daniel@ffwll.ch,
 srinivasan.shanmugam@amd.com, Arunpravin.PaneerSelvam@amd.com,
 le.ma@amd.com, Felix.Kuehling@amd.com, mukul.joshi@amd.com,
 amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org,
 linux-kernel@vger.kernel.org
References: <20240422052608.5297-1-maqianga@uniontech.com>
 <68f02c5c-5591-4d6f-9926-b0fc6f9f6287@amd.com>
 <D94775003178862D+20240422203329.49844e71@john-PC>
 <bde48eef-4d8a-4cfa-b824-6de88c0f87fd@amd.com>
 <CADnq5_PQ67J9ytb89-DqOgDw5V-s98TOyVjT5BGfkWMYv5sMQg@mail.gmail.com>
Content-Language: en-US
From: =?UTF-8?Q?Christian_K=C3=B6nig?= <christian.koenig@amd.com>
In-Reply-To: <CADnq5_PQ67J9ytb89-DqOgDw5V-s98TOyVjT5BGfkWMYv5sMQg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?SkVyZE5iN2ZRTHFobnV1dmM4c3hEK1ZTNVkzUktBOEhwRFVGM3Jxc1ZPdTgr?=
 =?utf-8?B?MmZ3aGxFMmZWckRFemZya08xM3FNdjdwYWQzWnI2Wmh6aFd3VEFXUW1UVjBQ?=
 =?utf-8?B?R09SL1FDOTVaWUVSMUdkQ0sxUGZsZDYxaXg1NnRYQ2E1RWRuUHp5VWJWc3BQ?=
 =?utf-8?B?ZWpLVVR6ZEhmRTJ5aVJNc2UzUlpiKzdJU01mMUNLazVwZWZrQ1RWL1BJa2Rw?=
 =?utf-8?B?RDJHVUttZTI2azhTRVBtT01sdkdEbG1YMjRUbkU1Qi9velZiYVhIb0NWd1Jy?=
 =?utf-8?B?aVBxRmI3RklCNS8xakJIRzY1dXNpRDhydC90QnI0WlczWXlqWFVqRStTeFNK?=
 =?utf-8?B?U2FGeU9qY1Y0c3M3bHUxbTh1M3VCS0dRR0xOZDl5QllmMWJFTXVKR2o1RHdh?=
 =?utf-8?B?K0N0TTJ4VlZrSGxVRTNLYmdmMTZuTlJiWjVKUWhJLzFhWE8zcW9Ybjl2YXhF?=
 =?utf-8?B?UGthY2Y2VUN5TWhBdStQV0EybHQ2YUs4a3YvVXpycmM5eEk3TTBhamZBWEov?=
 =?utf-8?B?eTgwb3JNVzE4cVJ1WU5UNms4SU5oSGVsanFyS04vTm1TWkJzaGxxMXBmNDdR?=
 =?utf-8?B?aGRFR21SbHlzVjFGMXlzYlVKeWpjRHZtcE1ra1NrVTVQR1VXektXclhwMGlr?=
 =?utf-8?B?T0ZjL2Y2c2ZzTXU4RndtM08xWHZCaWdBenB6aWNOVnZrV3NzeUtLM3pycXBs?=
 =?utf-8?B?SnFMSmd0enFCbTF5QVJDWHVYcGF4akM5QVB2WnFXZ20xKzZCZ0l3R0thNjQ4?=
 =?utf-8?B?WTFzUWZMUnZrTGd5dFlBLzZVOERxVHN5WmtQVzNRYjBGd2RlVFBzVi9CQjB1?=
 =?utf-8?B?bjF2TldxOFpFekYwOUVVOWw2OG5ZMzVUYWgxK2dUYUZjZTlpblQySDhJSXhJ?=
 =?utf-8?B?aGNhSEt5MElmSm56Y1ZhNGdzRXBldnBzZmJPTTJpaDhJRzdSR0l6R2ExYmFp?=
 =?utf-8?B?MGcwdFI5OXdPb2RLOEVHUTdsRGlZSDFSUUxxcGorZENOcUQvTXlMb2JDRTJU?=
 =?utf-8?B?eDRiR0Jpc3Q4c2gydVJWTmRKZHNObk5IeE1HMkd6MUtsL29kRnhKRytqYXlw?=
 =?utf-8?B?RG9BdThQRDZlbktmQm96dmZSbEpVQ0VQUE1PNFdFc1lRb3lOdWJNZ0lqN1JJ?=
 =?utf-8?B?MDY2TkRBUFNVdmZldUM2Tk11MEdpZWVJQVBZVFQwc1BDSGJqSE9zd0RQYXlL?=
 =?utf-8?B?ZzdvOHluekJuZXlWSE04djM4bUVTV0JqYUdDdHRLSHdpbkkyUzNsQjlkdVpY?=
 =?utf-8?B?WGRDajBXWEd2VnpUQnNaSmh1OXdpMFAxVGpwQXBNdGc0VU5JK0k3cllPZDBO?=
 =?utf-8?B?VTYwN3BFK1U1YnN5cnlrczlHbWhBT011aHNFcXdqOUtXZ3FQSXg2SHZXUDhT?=
 =?utf-8?B?VDk4U0VxNXdNbmMyUFdXcTd3dFkvL1FNYThZN1p2ZjFtOGVReC90YnlVY0x5?=
 =?utf-8?B?bWZXS0ZqQjRYYTNISXNmemlnaVRWZERPRHh6K0VmaFp2ZHRHSHAxSGlNRG42?=
 =?utf-8?B?RVVGdmdvckxZWTREQkc4S3loWU4zTnpmUVVmZFFaaElob0FtUkJWaFZ2WXIy?=
 =?utf-8?B?amxCaHF4MDR0V0lHZ005R0EyblF5d3h4SVZadUp0MHo3NldYRk5SVUxLM25Z?=
 =?utf-8?B?Vks5Yng4NFlxR05FYXV4UkVKMExPRWgzSFFtU0d1c2N6RWM2b1dUeXNDWFl5?=
 =?utf-8?B?NkV4ME5xQnp1VHJsSHF5SDRsV3NmUEx6WE83UjhUUmwrdFZBRGRVbDBTa0VW?=
 =?utf-8?B?L0hQZnhxVWJYdjNveTlWWm1ISk5ZZ1MwZWt1MTV0Qm80UzNuYzZsZnBrOE1P?=
 =?utf-8?B?WmpvWGRGVm1oWmtIKzVTM1V5WE5HTVBDVy9BSWFaejhVYVJIUXlGNEFqQWk4?=
 =?utf-8?B?V0l6cGUvdzY3enNoUjZKcnNnK2ZtZkJZR1BiRExudDJkczlxVnI4NW5SVHdi?=
 =?utf-8?B?VkRNMEswYys2RXNFdUJMV2xEbXUrcGJxeFBOam9MMTM4SHdTazFxQlZWb2s0?=
 =?utf-8?B?bi93QTgxQ1FTeFFYMGJLNElrWEMva1o1bDlXaCtlU1dsaTZTVzZVd3pqdnRh?=
 =?utf-8?B?RUpGTUpvWUNud2MyaHl6TVQ2UkQySGlHNzFsR2tJbTRiTHpDZXMzWFFwQ1lX?=
 =?utf-8?Q?0SAaHHp+GLyJayjWXYjz0vgew?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e6f0f2fd-2aea-47a4-f1ba-08dc62db2c34
X-MS-Exchange-CrossTenant-AuthSource: PH7PR12MB5685.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Apr 2024 14:47:46.1634
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: cH9dNdeiMAq2Ndyg1Im77U1/ZEvkI4XeK8i1w7nlvw9DONrl2832QrrYe/npoMqf
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB7269

Am 22.04.24 um 16:40 schrieb Alex Deucher:
> On Mon, Apr 22, 2024 at 9:00 AM Christian König
> <christian.koenig@amd.com> wrote:
>> Am 22.04.24 um 14:33 schrieb Qiang Ma:
>>> On Mon, 22 Apr 2024 11:40:26 +0200
>>> Christian König <christian.koenig@amd.com> wrote:
>>>
>>>> Am 22.04.24 um 07:26 schrieb Qiang Ma:
>>>>> Some boards(like Oland PRO: 0x1002:0x6613) seem to have
>>>>> garbage in the upper 16 bits of the vram size register,
>>>>> kern log as follows:
>>>>>
>>>>> [    6.000000] [drm] Detected VRAM RAM=2256537600M, BAR=256M
>>>>> [    6.007812] [drm] RAM width 64bits GDDR5
>>>>> [    6.031250] [drm] amdgpu: 2256537600M of VRAM memory ready
>>>>>
>>>>> This is obviously not true, check for this and clamp the size
>>>>> properly. Fixes boards reporting bogus amounts of vram,
>>>>> kern log as follows:
>>>>>
>>>>> [    2.789062] [drm] Probable bad vram size: 0x86800800
>>>>> [    2.789062] [drm] Detected VRAM RAM=2048M, BAR=256M
>>>>> [    2.789062] [drm] RAM width 64bits GDDR5
>>>>> [    2.789062] [drm] amdgpu: 2048M of VRAM memory ready
>>>> Well we had patches like this one here before and so far we always
>>>> rejected them.
>>>>
>>>> When the mmCONFIG_MEMSIZE register isn't properly initialized then
>>>> there is something wrong with your hardware.
>>>>
>>>> Working around that in the software driver is not going to fly.
>>>>
>>>> Regards,
>>>> Christian.
>>>>
>>> Hi Christian:
>>> I see that two patches for this issue have been merged, and the
>>> patches are as follows:
>>>
>>> 11544d77e397 drm/amdgpu: fixup bad vram size on gmc v8
>>> 0ca223b029a2 drm/radeon: fixup bad vram size on SI
>> Mhm, I remember that we discussed reverting those but it looks like that
>> never happened. I need to ask around internally.
>>
>> Question is do you see any other problems with the board? E.g. incorrect
>> connector or harvesting configuration?
> I'll need to dig up the past discussion again, but IIRC, the issue was
> only seen on some non-x86 platforms.  Maybe something specific to MMIO
> on those?

I honestly doesn't remember it either, but in general it's the job of 
the VBIOS to init this register.

So if we see the upper bits mangled the VBIOS hasn't done that correctly 
and it's quite likely that this is only the tip of the iceberg of problems.

Christian.

>
> Alex
>
>
>> Regards,
>> Christian.
>>
>>> Qiang Ma
>>>
>>>>> Signed-off-by: Qiang Ma <maqianga@uniontech.com>
>>>>> ---
>>>>>     drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c | 11 +++++++++--
>>>>>     drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c | 13 ++++++++++---
>>>>>     2 files changed, 19 insertions(+), 5 deletions(-)
>>>>>
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c
>>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c index
>>>>> 23b478639921..3703695f7789 100644 ---
>>>>> a/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c +++
>>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v6_0.c @@ -309,8 +309,15 @@ static
>>>>> int gmc_v6_0_mc_init(struct amdgpu_device *adev) }
>>>>>      adev->gmc.vram_width = numchan * chansize;
>>>>>      /* size in MB on si */
>>>>> -   adev->gmc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>>>>> 1024ULL * 1024ULL;
>>>>> -   adev->gmc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>>>>> 1024ULL * 1024ULL;
>>>>> +   tmp = RREG32(mmCONFIG_MEMSIZE);
>>>>> +   /* some boards may have garbage in the upper 16 bits */
>>>>> +   if (tmp & 0xffff0000) {
>>>>> +           DRM_INFO("Probable bad vram size: 0x%08x\n", tmp);
>>>>> +           if (tmp & 0xffff)
>>>>> +                   tmp &= 0xffff;
>>>>> +   }
>>>>> +   adev->gmc.mc_vram_size = tmp * 1024ULL * 1024ULL;
>>>>> +   adev->gmc.real_vram_size = adev->gmc.mc_vram_size;
>>>>>
>>>>>      if (!(adev->flags & AMD_IS_APU)) {
>>>>>              r = amdgpu_device_resize_fb_bar(adev);
>>>>> diff --git a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c
>>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c index
>>>>> 3da7b6a2b00d..1df1fc578ff6 100644 ---
>>>>> a/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c +++
>>>>> b/drivers/gpu/drm/amd/amdgpu/gmc_v7_0.c @@ -316,10 +316,10 @@
>>>>> static void gmc_v7_0_mc_program(struct amdgpu_device *adev) static
>>>>> int gmc_v7_0_mc_init(struct amdgpu_device *adev) {
>>>>>      int r;
>>>>> +   u32 tmp;
>>>>>
>>>>>      adev->gmc.vram_width =
>>>>> amdgpu_atombios_get_vram_width(adev); if (!adev->gmc.vram_width) {
>>>>> -           u32 tmp;
>>>>>              int chansize, numchan;
>>>>>
>>>>>              /* Get VRAM informations */
>>>>> @@ -363,8 +363,15 @@ static int gmc_v7_0_mc_init(struct
>>>>> amdgpu_device *adev) adev->gmc.vram_width = numchan * chansize;
>>>>>      }
>>>>>      /* size in MB on si */
>>>>> -   adev->gmc.mc_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>>>>> 1024ULL * 1024ULL;
>>>>> -   adev->gmc.real_vram_size = RREG32(mmCONFIG_MEMSIZE) *
>>>>> 1024ULL * 1024ULL;
>>>>> +   tmp = RREG32(mmCONFIG_MEMSIZE);
>>>>> +   /* some boards may have garbage in the upper 16 bits */
>>>>> +   if (tmp & 0xffff0000) {
>>>>> +           DRM_INFO("Probable bad vram size: 0x%08x\n", tmp);
>>>>> +           if (tmp & 0xffff)
>>>>> +                   tmp &= 0xffff;
>>>>> +   }
>>>>> +   adev->gmc.mc_vram_size = tmp * 1024ULL * 1024ULL;
>>>>> +   adev->gmc.real_vram_size = adev->gmc.mc_vram_size;
>>>>>
>>>>>      if (!(adev->flags & AMD_IS_APU)) {
>>>>>              r = amdgpu_device_resize_fb_bar(adev);