Message-ID: <e17d1ee9-7383-6717-6f10-9465b273b1ed@amd.com>
Date:   Fri, 17 Feb 2023 10:53:11 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.8.1
Subject: =?UTF-8?Q?Re=3a_=5bregression=2c_bisected=2c_pci/iommu=5d_Bug=c2=a0?=
 =?UTF-8?Q?216865_-_Black_screen_when_amdgpu_started_during_6=2e2-rc1_boot_w?=
 =?UTF-8?Q?ith_AMD_IOMMU_enabled?=
Content-Language: en-US
To:     Matt Fagnani <matt.fagnani@bell.net>,
        Felix Kuehling <felix.kuehling@amd.com>,
        Bjorn Helgaas <helgaas@kernel.org>,
        Baolu Lu <baolu.lu@linux.intel.com>,
        "Huang, Shimmer" <Shimmer.Huang@amd.com>,
        "Liu, Aaron" <Aaron.Liu@amd.com>, Jason Gunthorpe <jgg@nvidia.com>
Cc:     Joerg Roedel <jroedel@suse.de>,
        "regressions@lists.linux.dev" <regressions@lists.linux.dev>,
        Thorsten Leemhuis <regressions@leemhuis.info>,
        Linux PCI <linux-pci@vger.kernel.org>,
        "Pan, Xinhui" <Xinhui.Pan@amd.com>, amd-gfx@lists.freedesktop.org,
        LKML <linux-kernel@vger.kernel.org>,
        Bjorn Helgaas <bhelgaas@google.com>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "Deucher, Alexander" <Alexander.Deucher@amd.com>,
        =?UTF-8?Q?Christian_K=c3=b6nig?= <christian.koenig@amd.com>
References: <20230215153913.GA3189407@bhelgaas>
 <e3b866eb-830c-9037-39c7-978714aaf4d2@amd.com>
 <51861ff2-380f-640d-d5e8-7efe979a97ea@amd.com>
 <40b2da4a-a205-3cf2-0c78-c94c28b2d3f4@bell.net>
From:   Vasant Hegde <vasant.hegde@amd.com>
In-Reply-To: <40b2da4a-a205-3cf2-0c78-c94c28b2d3f4@bell.net>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?eTBpUnk5bXU1RjZCWUhuM0N5S0lCR0hDc1dXb3M4WjQ1QVZtRm1jTzMrZzVk?=
 =?utf-8?B?eEg3OEFSdUhIaGh3T0sxYXZjalpqNmJYN1lpR2NPWURoNEZ0YnN3U0xHSWNj?=
 =?utf-8?B?TkNiSzFHVnAzaElXWHR2aFU3cUtkN2U2c2tkVVpwZm1zWWxXQUgzRy9vY3Bw?=
 =?utf-8?B?bkF5UVZzS1VrS29EcHk5UWxOMDJXYnBhWkdzWEp4VmFCSi9TODVpMjgrVlo5?=
 =?utf-8?B?bnpGY3NlTERSMHZ3T1E0WnBYWW9xY2MvZ1VrVFJnY3QwVVpKV2YzTzQ2VHAz?=
 =?utf-8?B?SkZJeFBSSDJJMURVemNlU0toemo4Sk15cjllaVdON0NUcll2MzQyLzFQbGt2?=
 =?utf-8?B?aE04SWNKSTBEU21ZS0VRQ2s2bnBSbGhJQVRTbDVKb1ZzU0labWZNNWNveHM3?=
 =?utf-8?B?QXp6bXVMVEcrT3JlRkJoeC9CcndsQjlkc3Q0ZnI5UnU4cE9ubllWU3Jqdm8x?=
 =?utf-8?B?UkJHdE0zY3JXdk5jZEk4N2huNDdVZnBsYUVJVzZBWUxXcEc0N0V2azdjcW52?=
 =?utf-8?B?N0MwZ29NZXFURWFHanNWM0IwMHRFYzJvTGZZMTZXcHdGcVljYmNjNVpFOEZS?=
 =?utf-8?B?eXp6MFBFK0FEbjVNeFRmSzJsWkRiNHdYN1lNRHdVMlZMVERNcCtRMndoMC90?=
 =?utf-8?B?NWlsUWpIK0ZsUDM2UmpzVjVvbkdyb05pTHh4Y0lNWGJyWTh2UGttenlPUVY0?=
 =?utf-8?B?VXcrVDFaSWEvNlFvY1ZIKzRqSnFiek9pQUR5SkFkNEpqbWEwdzdiTkk3alZx?=
 =?utf-8?B?Q2ZSMWU2OVBaU2N3UkVBM1lrYzFFajVVTCsxZHZPdkJHbEE2VzRqRkFZcS9T?=
 =?utf-8?B?UnBGeWRlVnEzVzFKR1UwazcvazVQSlJmNnl6OWR4VFErSklqR2N0RDhoYk9J?=
 =?utf-8?B?M2k0QTNDZGx4Nlo3MzBIWVVNekE0QlV4cy9vMk1CemQ3WFBKb2tXSU9PZHM2?=
 =?utf-8?B?aHR0ekl5aTBQTFljb1RYY2hKTFNJc0NRRWN1b0VnVlFlNTExVDlWUWRzQ2t1?=
 =?utf-8?B?QmhrdS80N2ZHMEhxUnhzWkFsb3dIbXZMYXhZVjEwbVlpYmRBQi9JLy94Y0hE?=
 =?utf-8?B?NXgrKzNFbEt6VzlkeHg2Q3k4cDB5cVNSckhWa1RIajlUK0hjWHNnYXVNTFla?=
 =?utf-8?B?VUNYNWhyQ3MrQkxCd1lRWUhSY0xKRnhtSzRIUjlqZytTbU1ldktZSmFlU2xK?=
 =?utf-8?B?cjZQRnVOWkQwMmZNYWoxbHQxQm9zcnl4TFpUcGFuR1htTEw1SEZsMWJVajVn?=
 =?utf-8?B?WUQvS2lXaG9iczZZNFRaMTgrYnNQK0dEOEZIMXZTd0htUjJJNXRpMnBwRFJ4?=
 =?utf-8?B?ZVg5QWJqK2k2SDJDSUFLQkJlN2pFRTlIRHU0aFV0UGlvSFBSVXZ0a3djeDEv?=
 =?utf-8?B?UEEzYjdCTmo2S1N6bkxlV1M3U3QvalVGTmF5ZW1PZ2VHRmVMSnVOb1E4bTM5?=
 =?utf-8?B?eGdSYUZ4eEFlREplNG0rc25JWHlTcldLUFZUNHlXcCtlZCtQa0YrcHpkMHRj?=
 =?utf-8?B?ZlJZeFpjOThibG1uSlE2cVZBSytjZDBHdEZzNWdueklobmc1L1A2SnNlMExD?=
 =?utf-8?B?eFhtdGY0V0VzNVdLUVZ2d29nUzZYNlIwMy94cXR1L2ZhTElJUmQzK2puaE5l?=
 =?utf-8?B?TzZPSE4raEgvcjNzSUwySGl3d2xtSnlBRFBMbDJWVG1IcUZRVTNlTGlJVkZQ?=
 =?utf-8?B?Qng0V1M5ZUxMbHorQjAzRXpscGMvdmRRQ1RNb3d5cnFWRmplZ1VjWEFOakJ5?=
 =?utf-8?B?bWthWFZHcWlCVzlkeFcwbGhZS2h1VGFsZExhYzFmZmNjK0RndDg1OVBKKzhM?=
 =?utf-8?B?K1B1cWs0UU9LSE44dWRPZUxuNzFPY3dia2YwVDlFTEI0b1MwMDJvTHlSR2Fa?=
 =?utf-8?B?OG96ZEhtRVZ3VHJQM1E2aXRtVnd2UFJaaGNDOFY0UXBwUklCb3h2MGtGcyty?=
 =?utf-8?B?MHdET1pJZkMxditjckt0Nk1yRWdZMExsMXNrNzRxU3hobjlsUkVuajlOU3dD?=
 =?utf-8?B?UWg0YUp5aVlOa3hwRm5yZ0hzN2VudlVtV29Dc2VOcFA0bGhMa2JMSk9RRVZt?=
 =?utf-8?B?YWJERVRFajFFbnBmQTRGK3MyVTBvS28rSXNma29CK0NUUEpUQ09Za2RLUkRq?=
 =?utf-8?Q?F0chvBLuO0UjtYCUpVkTJKJPl?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 0fd483f2-d449-40cc-2e75-08db10a718af
X-MS-Exchange-CrossTenant-AuthSource: DS7PR12MB6048.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Feb 2023 05:23:26.4180
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: noH4ElEMWkScXYnM9mce1qqX4dcZ68YM41N2GHHXaD4VQAazcpWlY6UccLEaEdj+Jea+UsKYjnsMwYAr7OGf+A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB6269
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Matt,

Thanks a lot for testing and the dmesg log.

On 2/17/2023 12:29 AM, Matt Fagnani wrote:
> Vasant,
> 
> I applied your four patches to 6.2-rc8 and built that. The black screen, null
> pointer dereference, and warnings didn't happen when booting 6.2-rc8 with your
> patches. There were errors that the IOMMU wasn't restarted when amdgpu and
> amdkfd was starting though at kernel: kfd kfd: amdgpu: Failed to resume IOMMU
> for device 1002:9874. I don't know if those IOMMU errors were expected or not,

This patch is not for fixing PASID enablement issue. Its more of gracefully
handling the error path.

This means patch worked in expected way. i. e. It failed to enable PASID because
of original patch (commit 201007ef70), it didn't attach devices to new domain
and attach devices back to default domain.
It returned error to GPU saying we couldn't enable PASID/PRI. Hence we saw above
error message.

-Vasant