Received-SPF: pass (google.com: domain of linux-kernel+bounces-130606-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Message-ID: <ff810b13-b534-4e4f-a2e2-3cdf2b563494@amd.com>
Date: Wed, 3 Apr 2024 16:08:35 -0500
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] x86/sev: Apply RMP table fixups for kexec.
Content-Language: en-US
To: Michael Roth <michael.roth@amd.com>
Cc: Tom Lendacky <thomas.lendacky@amd.com>,
 linux-tip-commits@vger.kernel.org, bp@alien8.de, x86@kernel.org,
 linux-kernel@vger.kernel.org
References: <20240312184757.52699-1-Ashish.Kalra@amd.com>
 <b1822884-cba3-4009-d685-81a36dbb4353@amd.com>
 <b2da0a15-c0f5-49a7-b34c-eb408e9aa293@amd.com>
 <20240402224222.vxaqenqvyi32zleb@amd.com>
From: "Kalra, Ashish" <ashish.kalra@amd.com>
In-Reply-To: <20240402224222.vxaqenqvyi32zleb@amd.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OWFBY2dPdUxaL0ZGZ0xFcEg3WWFlMENjdGFPZ05aWnQ3MTBXVWNyMFd3RC8r?=
 =?utf-8?B?a09vZFFLWmc1eGUyS2U0ZnRlcW5TbUtpdUxpaVhrNlk2MGQzSGx6N0M0eHZP?=
 =?utf-8?B?UloxdWd1SFI2NEFrRnFFL0hnZXBqT2pHdms3R2hrRGVDRnZDQ2NjM3VYRkVw?=
 =?utf-8?B?dXFWTzhPZTNZN2l0STNnNVpUUWk2QmxoQi9SY2VoT1c3VjFBWVBwK0h6Y0lw?=
 =?utf-8?B?R2QvNXVtVHpDRVhZTmMvY3piejhiU3FoSEU3MHBXRFRyRG9uYmZEZ21RUFoy?=
 =?utf-8?B?MW9IU2RhbjFScktvMml6M0c2czNjQjBwdmVhKzhEdkpSSDJPK1dtSFZNMmpS?=
 =?utf-8?B?b2FtYXB2M0RHTWwwdVlXZVZQeWQwem1sclA0WG44dEIyOUtycHFacHRyRk8r?=
 =?utf-8?B?OS9KNUV0dy9qclliOThZNFNENm5QbTNUVnhwTzFvakFuckxRRFlQcGlIWGFw?=
 =?utf-8?B?UXU2N2FHZTN2WmVYMEI5K2F0aDM3RUd6OG5iOFY1NjZwS3BLNzNRbEMxQVBW?=
 =?utf-8?B?b1M5R0NnZGh4dGNWL0lJcnpZclFoZ1UxWnZqRXZqTHg1c0t2b0dtQmsvaEFl?=
 =?utf-8?B?blNHRW5VL1RSejNiQzdNbVdpSlpVaW9ZakJYWEg0TGI2Q1ExUnNYUG1UZ0kz?=
 =?utf-8?B?MmI0eFN0UEEwYWxCNzRMbVA0ZmViWlU4MGZnYUZRcmpmcVJ6bFVvNnIxUlo5?=
 =?utf-8?B?REZseVAvODE4YzROazlhQTYrS3oxcWhhS1Q0RUxXZFZiT0QwR25rd0xob2px?=
 =?utf-8?B?cjhibkxWUVI5UHdqTTc0a3R1OUMwQWF1SjFtMS9YemZFTVZZamxhTmlzOUd5?=
 =?utf-8?B?c24xVG9ZdmlGQXZLTHp6SjBiZmFUZlpCRU9uZjdEWjNzc0gvMDV6SVhaa2d0?=
 =?utf-8?B?NEZGVmd6T2V5RDBwR2x1ZTk0QlFJQnF2bTVuSUNpUWUzSWxQU2VxOHdmQ2p1?=
 =?utf-8?B?SFFkaEtwWjgxc0ZTcy9BWVZzWlNWSTI2NHBKemp2c3NMcU5wQzhzYnlJdCtS?=
 =?utf-8?B?NzE4NkF4QXB1QnZDQWJIN0VXQ3Rqc1E3cS83UGhQNkxtSzU1VjdqenZ3NkFL?=
 =?utf-8?B?THhUU25kMnl0bXlWWlRUc1B3Z0ZyWnZYVTRBTWpxenFUUFBrU2pkcVNSZktZ?=
 =?utf-8?B?STdOTUJGaW1JbkdGdk9wbWVnbXJFYlV2aHlMZjk0bm96Sk5yTXJQQm9CQVRT?=
 =?utf-8?B?N1ZDRHlVWVNMNWdYR3FzR2JDRmZldEFYd0JZcWdWVGFsbW5SdjdyTzk4QWxD?=
 =?utf-8?B?Y2hvZ3FocnVjYTdCVklvczIraXBuZVVjSDRNWUx4UjdURG1WL0tmaERLNWpi?=
 =?utf-8?B?STBhVGxUalRXeDc0NzA2U2l4WVVCb0tpZXNqcjZwQXFRUVpDVHR6bHlISlVQ?=
 =?utf-8?B?UDVGYVI5cGxkNmszRjdvNU1wd1FuZWN4N2p4ajBwUmdMSEdlaGFsNjkvcGlx?=
 =?utf-8?B?ZjB6K21rOTZ5VUhiN3BMRXNHNE41ejlPNzRMNnlxay9heS8wYkNicFlsMUZz?=
 =?utf-8?B?VlYraTBha3FqMXVWL0V0ZnZaSHROL2llMmpwNCtoaGhBYjcybWZBSERnMU9w?=
 =?utf-8?B?bVk1MVdyV3dqbFRpL0kvRVhnTm1BU0VmdkNkMU5FaW15MUdpdXZ6SGFpbHFL?=
 =?utf-8?B?UVk2YVMxN2VLNUd1dmFBU2RWMElBSjg5QnU1WlV2ZGl4V1hTckVNNDl2ODMw?=
 =?utf-8?B?MzdmS1ZUNTlLZjd2bHo5YndVWUswU3FGQjBoZENLQUZlaTI5elFyMFRvbnRG?=
 =?utf-8?B?bkZrQUxzSkFuR256K1JyclFOOUE3NHpjbFZ4YmxYSzRueC9mZTBMNXpRRFl2?=
 =?utf-8?B?Y3p6M2RFN0duRkNwd25HaytqaUw1M1kranVYcHJSTERpYTUrejlXZHJxRnJG?=
 =?utf-8?B?T1c3MlEwUDEyTFNsUWFoTlRaTDlzbnhGcmxVUWV0cDlQb1Fqa2g2UXdmeXJ3?=
 =?utf-8?B?Q09TK3Fsbnl0amxwOHU3b3Jqc3ZDMkk2dGk2YmZ2SVJHTUlhTHkvZDVBSzJD?=
 =?utf-8?B?Wll0L1VsWE9UZWlGc2NsZWZ4dWR1SzJrd3BwM3lOMGc1VmI3dU00TjdyZTZq?=
 =?utf-8?B?SFYxT1Z4NGNFMzQxLzV2dmhYQXBFd2hGaDNvUy9POENqV0Uwa2hiS3RWNXhr?=
 =?utf-8?Q?SYdW4bkoDfryX4F/cjT364ATl?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: a92ca992-dae9-4c83-265e-08dc54223a9b
X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB2767.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Apr 2024 21:08:37.0599
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: qoHdY3q5cPoHZck7KwvAqpwqJTbjcQ2EnkOsDGwqyZXg3Jctp5xK/wiSaBfnFbgDgyU2jwaWtY22Q/K9yofc+w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR12MB5653


On 4/2/2024 5:42 PM, Michael Roth wrote:
> On Tue, Apr 02, 2024 at 05:31:09PM -0500, Kalra, Ashish wrote:
>> On 4/2/2024 5:09 PM, Tom Lendacky wrote:
>>> On 3/12/24 13:47, Ashish Kalra wrote:
>>>> From: Ashish Kalra <ashish.kalra@amd.com>
>>>>
>>>> RMP table start and end physical range may not be aligned to 2MB in
>>>> the e820 tables causing fatal RMP page faults during kexec boot when
>>>> new page allocations are done in the same 2MB page as the RMP table.
>>>> Check if RMP table start and end physical range in e820_table is not
>>>> aligned to 2MB and in that case use e820__range_update() to map this
>>>> range to reserved.
>>>>
>>>> Override e820__memory_setup_default() to check and apply these RMP table
>>>> fixups in e820_table before e820_table is used to setup
>>>> e280_table_firmware and e820_table_kexec.
>>>>
>>>> Fixes: c3b86e61b756 ("x86/cpufeatures: Enable/unmask SEV-SNP CPU
>>>> feature")
>>>> Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
>>>> ---
>>>>    arch/x86/virt/svm/sev.c | 52 +++++++++++++++++++++++++++++++++++++++++
>>>>    1 file changed, 52 insertions(+)
>>>>
>>>> diff --git a/arch/x86/virt/svm/sev.c b/arch/x86/virt/svm/sev.c
>>>> index cffe1157a90a..e0d7584df28f 100644
>>>> --- a/arch/x86/virt/svm/sev.c
>>>> +++ b/arch/x86/virt/svm/sev.c
>>>> @@ -65,6 +65,8 @@ static u64 probed_rmp_base, probed_rmp_size;
>>>>    static struct rmpentry *rmptable __ro_after_init;
>>>>    static u64 rmptable_max_pfn __ro_after_init;
>>>>    +static char *__init snp_rmptable_e820_fixup(void);
>>>> +
>>>>    static LIST_HEAD(snp_leaked_pages_list);
>>>>    static DEFINE_SPINLOCK(snp_leaked_pages_list_lock);
>>>>    @@ -160,9 +162,59 @@ bool snp_probe_rmptable_info(void)
>>>>        pr_info("RMP table physical range [0x%016llx - 0x%016llx]\n",
>>>>            probed_rmp_base, probed_rmp_base + probed_rmp_size - 1);
>>>>    +    /*
>>>> +     * Override e820__memory_setup_default() to do any RMP table fixups
>>>> +     * for kexec if required.
>>>> +     */
>>>> +    x86_init.resources.memory_setup = snp_rmptable_e820_fixup;
>>> This produces a build warning:
>>>
>>> WARNING: modpost: vmlinux: section mismatch in reference:
>>> snp_probe_rmptable_info+0x95 (section: .text) -> x86_init (section:
>>> .init.data)
>>> WARNING: modpost: vmlinux: section mismatch in reference:
>>> snp_probe_rmptable_info+0x99 (section: .text) -> snp_rmptable_e820_fixup
>>> (section: .init.text)
>>>
>> Oh, so this requires snp_probe_rmptable_info() to be fixed to use the __init
>> macro.
>>
>> I believe that snp_probe_rmptable_info() should be anyway using the __init
>> macro and this fix for snp_probe_rmptable_info() needs to be sent as a
>> separate patch and regardless of this patch getting merged or not.
> I think you'll hit issues with:
>
>    bsp_determine_snp() ->           //non-__init
>      snp_probe_rmptable_info()      //__init
>
> and bsp_determine_snp() sticks around as a function pointer assigned to
> cpuinfo_x86 so I don't think you can use __init there.
>
> So might need to just drop __init from snp_rmptable_e820_fixup().

Actually, that will not help as snp_probe_rmptable_info() is *also* 
accessing x86_init.resources.memory_setup

Thanks, Ashish

>
>> Thanks, Ashish
>>
>>> Thanks,
>>> Tom
>>>
>>>> +
>>>>        return true;
>>>>    }