Received-SPF: pass (google.com: domain of linux-kernel+bounces-199468-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Message-ID: <141a9666-f3cf-4323-9536-4367f489be43@amd.com>
Date: Mon, 3 Jun 2024 11:56:01 -0500
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v7 1/3] efi/x86: Fix EFI memory map corruption with kexec
Content-Language: en-US
To: Mike Rapoport <rppt@kernel.org>
Cc: Borislav Petkov <bp@alien8.de>, tglx@linutronix.de, mingo@redhat.com,
 dave.hansen@linux.intel.com, x86@kernel.org, rafael@kernel.org,
 hpa@zytor.com, peterz@infradead.org, adrian.hunter@intel.com,
 sathyanarayanan.kuppuswamy@linux.intel.com, jun.nakajima@intel.com,
 rick.p.edgecombe@intel.com, thomas.lendacky@amd.com, michael.roth@amd.com,
 seanjc@google.com, kai.huang@intel.com, bhe@redhat.com,
 kirill.shutemov@linux.intel.com, bdas@redhat.com, vkuznets@redhat.com,
 dionnaglaze@google.com, anisinha@redhat.com, jroedel@suse.de,
 ardb@kernel.org, kexec@lists.infradead.org, linux-coco@lists.linux.dev,
 linux-kernel@vger.kernel.org
References: <20240528095522.509667-1-kirill.shutemov@linux.intel.com>
 <cover.1717111180.git.ashish.kalra@amd.com>
 <f4be03b8488665f56a1e5c6e6459f447352dfcf5.1717111180.git.ashish.kalra@amd.com>
 <20240603085654.GBZl2FVjPd-gagt-UA@fat_crate.local>
 <8e3dfc15-f609-4839-85c7-1cc8cefd7acc@amd.com> <Zl3HfiQ6oHdTdOdA@kernel.org>
 <1ef36309-8d7f-447b-a54a-3cdafeccca64@amd.com> <Zl3hPefvOwPv0Mjn@kernel.org>
From: "Kalra, Ashish" <ashish.kalra@amd.com>
In-Reply-To: <Zl3hPefvOwPv0Mjn@kernel.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?Vk0vRnk4elBhV1BnblIyT0tZRnNycllxWGo0dkJMRnpSNnp3YjNJaFd4R2kx?=
 =?utf-8?B?QUg4TkRScXVBSjgvNjNLZ1FjbXBMc0tvU3Q4a3RCUWs3YTRkZlB6SUJVanh6?=
 =?utf-8?B?R0k3bzEyQzRwSWxkOVh1azVLVWhpeXUrV0VJMGRaTUJBMnJsYm1TK1RMbVVx?=
 =?utf-8?B?aVlWZ2Y5VnFycm9GSTRCY3dFYXZHZG5LdlM0NmFta283dTBpR2JRT0RmT21i?=
 =?utf-8?B?TDIzcVlxTVY1SG84MGNzSWkrMUpkOURJMXQyVnd4R3ZkMlFMdTZlanpicTJv?=
 =?utf-8?B?cmROYlZtTG9vKzZ4VVhkdjM3WHE0VTdLZ1M0NXhhV01zZy9IeS8wMVM3NkM0?=
 =?utf-8?B?dUVRRU9NSFc1Ykt1Snl0OTlkaytlYWdmZjkwbDU3TC9rcjVPTkY5bGJCOVZS?=
 =?utf-8?B?bnZ2SGUzN0ZTa2QzYTgzQTFRZlRYLzgrS1dicGloam9mQTdKaVRuUE50SHV6?=
 =?utf-8?B?T2EzWUdPQVJsNHBsZ3hYZjRDZFlvTGIwUExrKzNRUUxOUTRpUzJ4bllCZkt3?=
 =?utf-8?B?S2QwNnp2Z0NVQVVtaFY4VkMwdWgxd2FIRGc0eVdWNUxmOUJFYU11a1BDN3Fy?=
 =?utf-8?B?M0NzSWU2T0hLQi9tSC95TGpHTXlDLzJmRGl6L0dRK0JEWWlhMHVuQUx6RUN0?=
 =?utf-8?B?M1V2UXRDUXdkZXR2ekx1LzRmaVFYeWVTT2ZLeHFsRWF1STBHaCt4S0pTbG5N?=
 =?utf-8?B?TjBXR3JaMlhPaWxwT1RSRFlQZHlZZUhId08zaCtTcnhxamgzY0tvTE5xdjJr?=
 =?utf-8?B?S0grY21SaUs0Tjd5V2cvNTNHU0ludWZWa3dsREhjUC9sZ2U0eGFjb2UvRVQ1?=
 =?utf-8?B?RFUvTnlONkMwU2tlSEJJTDFtVGtpMDk3QjJuc3JoVWdzYWFnL3ZBZnFHQnBS?=
 =?utf-8?B?d0ZkWVJvNlE4a0t2SXg5WWo2V2hOblBwVmJzaDFkeFpjTUlKYzZGRko5R0hR?=
 =?utf-8?B?NlNqL0xuYmJCVzZZMHJicHNPSGh4Wmo2MmZlWjF3TnJnVVdRbzVtWjlsYXcy?=
 =?utf-8?B?T0ZZUnVvVHA4c0JqZkRoUU5VTm9INFJKY1ZXbEIrZmluampJSmJMczBwNGhI?=
 =?utf-8?B?SDBHdmpEN3l2TWRvMjR4NWpYZWN2SEh1NmRHemJ1Z3ZVdUgrQ3BvVE5sTmNE?=
 =?utf-8?B?SEEvUnd4YXZDZFV1NlJLQXBHVzlNcEpPSFdGRjlrQ2JOcWlkYVpWS2tkazNW?=
 =?utf-8?B?M2dpY1NKZlFudUxxTytjNWtIZG42TjN3NWFHcUJBUDhyVlEwaEhuK1RRNmE3?=
 =?utf-8?B?TUlzU2I2TXdQRG1yNjh1WDQvbDQ4QXFxSkZXcUxhczgwaFNtYUkzQ3diQ24x?=
 =?utf-8?B?QnRrWWNZUWVINzJSSzlwRjlBbElNRU92eXpHYlo1QmNKbDBIYVcrTVNiMVVY?=
 =?utf-8?B?aUtVMzhobm1vaVBkWEJtOURhSWcwaDJvbU9HRW9NVCtNUkZkNGZRZFFXWGxq?=
 =?utf-8?B?c25tVWZoR3d0bWM1V2ttMXpQWmw2VkVRci9CbWh2aDZaZ0dTZEpSTEhTNXNr?=
 =?utf-8?B?TnJRaW1OTGgwTGVvRW5tMGZSNWs0WXI1ZWlpYVlKdlh6V09Hc0t4YXl4WjRR?=
 =?utf-8?B?eml5K2dOTm55elZaTEdHQjRDR3ZQNnBaMlhpcDM1ajJ4NkM5aG5IYWozUmFP?=
 =?utf-8?B?eTZxNFdqN1AxL2oxeHlLVFJ2UkFUdDk5S1BTUEhwelRxUGxpR3lTY24rVDFi?=
 =?utf-8?B?UXN0dEgxbStrdkkrU0JnNDdVMU43WUt2TFB3N0Z5TXNHYlBEbmQ1eWFQSEJP?=
 =?utf-8?B?ak1PTldndFhFSmN3RnhxN0RYbzhGNEZ4UUMyenVQV09ubnM2YzNPZUxqbmtt?=
 =?utf-8?B?aHV4bGkyMDBpb0c2c2cwcUsvVnI3L2xxNElGZmFRaGpOaGR4elpBYm9HaThr?=
 =?utf-8?B?dHNJbzBSQjFKUUp4Snp2RUVyZlhmcWtnZWY0emhQTUhUcGdGUEZaYk0ybG5S?=
 =?utf-8?B?QnpyTFAzZk9odzJ0RnltUnNWcmticHRQVTBQNnJmcXRkcG80TTd3bG1qOERn?=
 =?utf-8?B?UHNHNHRQMkhxT0RjWWJ0eDlONVMza1g1NUpEK2RZdzZ0T3JZNEFPY1M2dUR5?=
 =?utf-8?B?QlhzVllRVEtFZ09hUVlsSWI2ei8zcDQyU2JOZnBVT0hTV2N6V3R4S2c5ZGEw?=
 =?utf-8?Q?cLpo6fvVVDbwcu7V4XPlaLPQn?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: f5bcafc9-22da-4df1-c54b-08dc83ee0e59
X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB2767.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2024 16:56:04.7561
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: FRZXhty8sQ+YgIvxaydqkeFRyeau/RodS2LbsnGusC1zCvrP7j6cyyRPufKOXqu7qnqhP+Nhco/9EvKuh7jo4w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA1PR12MB6947

On 6/3/2024 10:29 AM, Mike Rapoport wrote:

> On Mon, Jun 03, 2024 at 09:01:49AM -0500, Kalra, Ashish wrote:
>> On 6/3/2024 8:39 AM, Mike Rapoport wrote:
>>
>>> On Mon, Jun 03, 2024 at 08:06:56AM -0500, Kalra, Ashish wrote:
>>>> On 6/3/2024 3:56 AM, Borislav Petkov wrote
>>>>
>>>>>> EFI memory map and due to early allocation it uses memblock allocation.
>>>>>>
>>>>>> Later during boot, efi_enter_virtual_mode() calls kexec_enter_virtual_mode()
>>>>>> in case of a kexec-ed kernel boot.
>>>>>>
>>>>>> This function kexec_enter_virtual_mode() installs the new EFI memory map by
>>>>>> calling efi_memmap_init_late() which remaps the efi_memmap physically allocated
>>>>>> in efi_arch_mem_reserve(), but this remapping is still using memblock allocation.
>>>>>>
>>>>>> Subsequently, when memblock is freed later in boot flow, this remapped
>>>>>> efi_memmap will have random corruption (similar to a use-after-free scenario).
>>>>>>
>>>>>> The corrupted EFI memory map is then passed to the next kexec-ed kernel
>>>>>> which causes a panic when trying to use the corrupted EFI memory map.
>>>>> This sounds fishy: memblock allocated memory is not freed later in the
>>>>> boot - it remains reserved. Only free memory is freed from memblock to
>>>>> the buddy allocator.
>>>>>
>>>>> Or is the problem that memblock-allocated memory cannot be memremapped
>>>>> because *raisins*?
>>>> This is what seems to be happening:
>>>>
>>>> efi_arch_mem_reserve() calls efi_memmap_alloc() to allocate memory for
>>>> EFI memory map and due to early allocation it uses memblock allocation.
>>>>
>>>> And later efi_enter_virtual_mode() calls kexec_enter_virtual_mode()
>>>> in case of a kexec-ed kernel boot.
>>>>
>>>> This function kexec_enter_virtual_mode() installs the new EFI memory map by
>>>> calling efi_memmap_init_late() which does memremap() on memblock-allocated memory.
>>> Does the issue happen only with SNP?
>> This is observed under SNP as efi_arch_mem_reserve() is only being called
>> with SNP enabled and then efi_arch_mem_reserve() allocates EFI memory map
>> using memblock.
> I don't see how efi_arch_mem_reserve() is only called with SNP. What did I
> miss?
>   

This is the call stack for efi_arch_mem_reserve():

[    0.310010]  efi_arch_mem_reserve+0xb1/0x220
[    0.311382]  efi_mem_reserve+0x36/0x60
[    0.311973]  efi_bgrt_init+0x17d/0x1a0
[    0.313265]  acpi_parse_bgrt+0x12/0x20
[    0.313858]  acpi_table_parse+0x77/0xd0
[    0.314463]  acpi_boot_init+0x362/0x630
[    0.315069]  setup_arch+0xa88/0xf80
[    0.315629]  start_kernel+0x68/0xa90
[    0.316194]  x86_64_start_reservations+0x1c/0x30
[    0.316921]  x86_64_start_kernel+0xbf/0x110
[    0.317582]  common_startup_64+0x13e/0x141

So, probably it is being invoked specifically for AMD platform ?

>> If we skip efi_arch_mem_reserve() (which should probably be anyway skipped
>> for kexec case), then for kexec boot, EFI memmap is memremapped in the same
>> virtual address as the first kernel and not the allocated memblock address.
> Maybe we should skip efi_arch_mem_reserve() for kexec case, but I think we
> still need to understand what's causing memory corruption.

When, efi_arch_mem_reserve() allocates memory for EFI memory map using 
memblock and then later in boot, kexec_enter_virtual_mode() does 
memremap on this memblock allocated memory, subsequently after this i 
see EFI memory map corruption, so are there are any issues doing 
memremap on memblock-allocated memory ?

Thanks, Ashish

>>> I didn't really dig, but my theory would be that it has something to do
>>> with arch_memremap_can_ram_remap() in arch/x86/mm/ioremap.c
>>>> Thanks, Ashish