Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <de34e018-679d-893e-683e-5c10cbc9c5b2@amd.com>
Date:   Tue, 3 Jan 2023 15:40:26 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.13.1
Subject: Re: [PATCH v2 00/11] Recover from failure to probe GPU
Content-Language: en-US
To:     Mario Limonciello <mario.limonciello@amd.com>,
        Javier Martinez Canillas <javierm@redhat.com>,
        Alex Deucher <alexander.deucher@amd.com>
Cc:     linux-kernel@vger.kernel.org, dri-devel@lists.freedesktop.org,
        amd-gfx@lists.freedesktop.org, Daniel Vetter <daniel@ffwll.ch>,
        Carlos Soriano Sanchez <csoriano@redhat.com>,
        David Airlie <airlied@gmail.com>, christian.koenig@amd.com
References: <20221228163102.468-1-mario.limonciello@amd.com>
From:   "Lazar, Lijo" <lijo.lazar@amd.com>
In-Reply-To: <20221228163102.468-1-mario.limonciello@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?ckpyejFaZm9CTlZHZkgvMEZzVHoyOGpmbzRXNHpIcS9UZUwrZkRac2VIejVY?=
 =?utf-8?B?QzRmaFNJNUo4UWxTVWN2NFBzbURwaXJWUmV5bHFQYWJ0MEgrTU1xVDVNYit6?=
 =?utf-8?B?SUhYdjMzUG4xeUlXTW93WUt3TU9LSjJiMWc4Qk9NampSQU5OL0ZuMU9JRUVS?=
 =?utf-8?B?d0NZbkNTQnovY3plSnp2aENYTXM4WTFiOFc4dlViTHkrZ1hDRGw0V2tpd2p2?=
 =?utf-8?B?TXVTQVdsL2VPdlFPaGNnMU92c1h6NGpZaDVlOGo2OXkrZTdFRlUwck95QU5H?=
 =?utf-8?B?VW5kTWxWOHFNSEk0a1FiV2pKZnd1WjE3MmMrVGRFc0Exc3g2Y09hWm9lb2xC?=
 =?utf-8?B?eVFZcHJONnVGQW5sVXJ0RThQOVE0VG9UcHVzMHE4SmcwYzhBWFduTFZSb2gz?=
 =?utf-8?B?UDliQkhnVzJ1eWhnU295V1VTTGFtM3Q1MTA2a1ZxZG5EVUx6ZHNPNFpqQ1p0?=
 =?utf-8?B?YVkrRXYySW9LUU45eU5NbWlKRTNGTG16YjUwYWZFMmVqUkhoVXd0SnRFR2pX?=
 =?utf-8?B?UHdCN3YwZm1Pemd1N1lCRTFNcG1McDYyNVJvRGNjZU03cExHQlR2cXpNc0cv?=
 =?utf-8?B?UndSRE1qeEpIVUJFNStJcDc3UWF0ZXF4bVFBVWVsWE82SkZ4TWlWZFlTTkVV?=
 =?utf-8?B?T0UzUWFYK3AwcHUwU2NLbmFvNWw3VzhnQ0xXYWFaN0xUaHRXNVZyeHMzRk1I?=
 =?utf-8?B?NWVDcHFzaG9EZWJ6VHpjcWhGaE9iczNSOXVrSDV6b1JWU3VIWnpYbW9BNFN1?=
 =?utf-8?B?M2dSZ0REODFESnI4T2xpTXNEa0NRdEtGSDRZY1B5aEg3ajlsZVBYNVpSUTBt?=
 =?utf-8?B?UTV1UGg3ck13cE01VGQvZGFTZE9CUHVWbmhKYnFxOXIzeVBOLytNeDB0Tk1u?=
 =?utf-8?B?WklDbWdPRjM4d3VaYTJ6N1E3OU1IMkM4WkVJU0lUM2t2T2ZYN29OL01wS1pN?=
 =?utf-8?B?QmpDbGszb0dhdXBKOFlLV1dUSTZYSlBBSFVNajlMQU1naFM2dmFjejBVZVBR?=
 =?utf-8?B?Q0FVeUhpZHFyUE1DLytjelFOOUlZNEpkT0ZQTzFaOVdueGNmK0hVM1BGK1lN?=
 =?utf-8?B?UmhHbVVFWjVIY0hWcjkwWXZnQVFuNGVKTkF4alBlNVJIbG44TzREZTFLSkUv?=
 =?utf-8?B?MjB4QXVZdUFycjJYdlp3M1FGaEIvUUx6NkJmeEJFd05DcW45THpWWmxDampm?=
 =?utf-8?B?cm1WZHFCZ0tuUE9wQUk0RzIwSmV1Sm5FMm82ZlROOUJKMTVtWkVJb29Nb2Jn?=
 =?utf-8?B?Wks1QzlRaExVdlE1ditmUVp5c25IUzBMOG8rM0JnR2tnd2JJTG4wRU91QTZw?=
 =?utf-8?B?enpkamUrY2trVVhKWFQ5QndvZDZJc1dWV25WWGUwMUpXZ3c4KzRpZDh1MFFD?=
 =?utf-8?B?UE45Qllac05RakNEa1FPR2ljU2VQUHJVRnZmQUNKNHMyWS9EUTdmYWF4Z1h6?=
 =?utf-8?B?cTVVRm5ndmhpUnd6SUFPaDJtZVJiTEFRbE42ZEVPeWk4bm92YS9xQUFIYW5I?=
 =?utf-8?B?U1o5SE41VmRjcjVYN0hQanNIYnQ4RVVXYU8wQkZDeGQxY0gyZVlReEs1bE1X?=
 =?utf-8?B?bTVvK0RzVnQyaUxNTjJxNHpTbWpmdllxY3R0QlZhK2ltTXp5L2RuN2xrS0w2?=
 =?utf-8?B?ZVJ0LytobGpINWQzeEtmTHJkZFVhazlQVFFoUzlKTlU4UmtRWmZUb1JZcjln?=
 =?utf-8?B?YnVwREhIQzRrL3cyZCttNy9zM1d3bC9HSUU2OGRMaUJSUFNoZ0VITm1LYllt?=
 =?utf-8?B?VktHMGxIcFpEd0xDcW9IVWpETjZFWXhkRGIrY2ZlOUR2SGZtbVFUZEFFUGZ6?=
 =?utf-8?B?SkcxcXA5Z2V4VVc5QlIvOGFhNVFzdmRoTXJaK2VpdFVDanEvZWRSeTdDRzZt?=
 =?utf-8?B?TzUvRXp5QklNK3p2SXIwczhNOWlBRmc0YTV6WlN2RXFoZzJvWVRESWxscUpD?=
 =?utf-8?B?L0FkcStiQ1U5cTNwR0RkeFM5c01lWmhNTnNOaTRDa2s3V2hNd0FMMmJJTnBG?=
 =?utf-8?B?ajZRbVVZcno0Z2VyM0QvekdyMG9udUFGU1NZYUg3dzJMQWJMV3N5Sy9PUGsz?=
 =?utf-8?B?UXJoajI0bVF1dUZWSE9LajNxYlhXRUphcWRzKzE3U0pHdU1JV3BaNTJSNlQ4?=
 =?utf-8?Q?hrtp/EBDSxnjJqBrYkk9d+tHM?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fe3742e8-f67b-4f32-ae65-08daed72c49a
X-MS-Exchange-CrossTenant-AuthSource: BYAPR12MB4614.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jan 2023 10:10:41.0934
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: czT3iqq+7/MDy5WMWK2Uk+AM8ThUOaIUDPeiShJkHZuOMU3jn9dlaumshT5d3fSs
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR12MB6214
X-Spam-Status: No, score=-5.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,TRACKER_ID autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 12/28/2022 10:00 PM, Mario Limonciello wrote:
> One of the first thing that KMS drivers do during initialization is
> destroy the system firmware framebuffer by means of
> `drm_aperture_remove_conflicting_pci_framebuffers`
> 
> This means that if for any reason the GPU failed to probe the user
> will be stuck with at best a screen frozen at the last thing that
> was shown before the KMS driver continued it's probe.
> 
> The problem is most pronounced when new GPU support is introduced
> because users will need to have a recent linux-firmware snapshot
> on their system when they boot a kernel with matching support.
> 
> However the problem is further exaggerated in the case of amdgpu because
> it has migrated to "IP discovery" where amdgpu will attempt to load
> on "ALL" AMD GPUs even if the driver is missing support for IP blocks
> contained in that GPU.
> 
> IP discovery requires some probing and isn't run until after the
> framebuffer has been destroyed.
> 
> This means a situation can occur where a user purchases a new GPU not
> yet supported by a distribution and when booting the installer it will
> "freeze" even if the distribution doesn't have the matching kernel support
> for those IP blocks.
> 
> The perfect example of this is Ubuntu 22.10 and the new dGPUs just
> launched by AMD.  The installation media ships with kernel 5.19 (which
> has IP discovery) but the amdgpu support for those IP blocks landed in
> kernel 6.0. The matching linux-firmware was released after 22.10's launch.
> The screen will freeze without nomodeset. Even if a user manages to install
> and then upgrades to kernel 6.0 after install they'll still have the
> problem of missing firmware, and the same experience.
> 
> This is quite jarring for users, particularly if they don't know
> that they have to use "nomodeset" to install.
> 
> To help the situation make changes to GPU discovery:
> 1) Delay releasing the firmware framebuffer until after IP discovery has
> completed.  This will help the situation of an older kernel that doesn't
> yet support the IP blocks probing a new GPU.
> 2) Request loading all PSP, VCN, SDMA, MES and GC microcode into memory
> during IP discovery. This will help the situation of new enough kernel for
> the IP discovery phase to otherwise pass but missing microcode from
> linux-firmware.git.
> 
> Not all requested firmware will be loaded during IP discovery as some of it
> will require larger driver architecture changes. For example SMU firmware
> isn't loaded on certain products, but that's not known until later on when
> the early_init phase of the SMU load occurs.
> 
> v1->v2:
>   * Take the suggestion from v1 thread to delay the framebuffer release until
>     ip discovery is done. This patch is CC to stable to that older stable
>     kernels with IP discovery won't try to probe unknown IP.
>   * Drop changes to drm aperature.
>   * Fetch SDMA, VCN, MES, GC and PSP microcode during IP discovery.
> 

What is the gain here in just checking if firmware files are available? 
It can fail anywhere during sw_init and it's the same situation.

Restricting IP FWs to IP specific files looks better to me than 
centralizing and creating interdependencies.

Thanks,
Lijo

> Mario Limonciello (11):
>    drm/amd: Delay removal of the firmware framebuffer
>    drm/amd: Add a legacy mapping to "amdgpu_ucode_ip_version_decode"
>    drm/amd: Convert SMUv11 microcode init to use
>      `amdgpu_ucode_ip_version_decode`
>    drm/amd: Convert SMU v13 to use `amdgpu_ucode_ip_version_decode`
>    drm/amd: Request SDMA microcode during IP discovery
>    drm/amd: Request VCN microcode during IP discovery
>    drm/amd: Request MES microcode during IP discovery
>    drm/amd: Request GFX9 microcode during IP discovery
>    drm/amd: Request GFX10 microcode during IP discovery
>    drm/amd: Request GFX11 microcode during IP discovery
>    drm/amd: Request PSP microcode during IP discovery
> 
>   drivers/gpu/drm/amd/amdgpu/amdgpu_device.c    |   8 +
>   drivers/gpu/drm/amd/amdgpu/amdgpu_discovery.c | 590 +++++++++++++++++-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c       |   6 -
>   drivers/gpu/drm/amd/amdgpu/amdgpu_psp.c       |   2 -
>   drivers/gpu/drm/amd/amdgpu/amdgpu_sdma.c      |   9 +-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_sdma.h      |   2 +-
>   drivers/gpu/drm/amd/amdgpu/amdgpu_ucode.c     | 208 ++++++
>   drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c       |  85 +--
>   drivers/gpu/drm/amd/amdgpu/gfx_v10_0.c        | 180 +-----
>   drivers/gpu/drm/amd/amdgpu/gfx_v11_0.c        |  64 +-
>   drivers/gpu/drm/amd/amdgpu/gfx_v9_0.c         | 143 +----
>   drivers/gpu/drm/amd/amdgpu/mes_v10_1.c        |  28 -
>   drivers/gpu/drm/amd/amdgpu/mes_v11_0.c        |  25 +-
>   drivers/gpu/drm/amd/amdgpu/psp_v10_0.c        | 106 +---
>   drivers/gpu/drm/amd/amdgpu/psp_v11_0.c        | 165 +----
>   drivers/gpu/drm/amd/amdgpu/psp_v12_0.c        | 102 +--
>   drivers/gpu/drm/amd/amdgpu/psp_v13_0.c        |  82 ---
>   drivers/gpu/drm/amd/amdgpu/psp_v13_0_4.c      |  36 --
>   drivers/gpu/drm/amd/amdgpu/psp_v3_1.c         |  36 --
>   drivers/gpu/drm/amd/amdgpu/sdma_v4_0.c        |  61 +-
>   drivers/gpu/drm/amd/amdgpu/sdma_v5_0.c        |  42 +-
>   drivers/gpu/drm/amd/amdgpu/sdma_v5_2.c        |  65 +-
>   drivers/gpu/drm/amd/amdgpu/sdma_v6_0.c        |  30 +-
>   .../gpu/drm/amd/pm/swsmu/smu11/smu_v11_0.c    |  35 +-
>   .../gpu/drm/amd/pm/swsmu/smu13/smu_v13_0.c    |  12 +-
>   25 files changed, 919 insertions(+), 1203 deletions(-)
> 
> 
> base-commit: de9a71e391a92841582ca3008e7b127a0b8ccf41