Received-SPF: pass (google.com: domain of linux-kernel+bounces-131090-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Date: Thu, 4 Apr 2024 01:17:45 -0700
From: Jeremi Piotrowski <jpiotrowski@linux.microsoft.com>
To: "Kalra, Ashish" <ashish.kalra@amd.com>
Cc: Michael Roth <michael.roth@amd.com>,
	Tom Lendacky <thomas.lendacky@amd.com>,
	linux-tip-commits@vger.kernel.org, bp@alien8.de, x86@kernel.org,
	linux-kernel@vger.kernel.org
Subject: Re: [PATCH] x86/sev: Apply RMP table fixups for kexec.
Message-ID: <20240404081745.GA15636@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net>
References: <20240312184757.52699-1-Ashish.Kalra@amd.com>
 <b1822884-cba3-4009-d685-81a36dbb4353@amd.com>
 <b2da0a15-c0f5-49a7-b34c-eb408e9aa293@amd.com>
 <20240402224222.vxaqenqvyi32zleb@amd.com>
 <ff810b13-b534-4e4f-a2e2-3cdf2b563494@amd.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <ff810b13-b534-4e4f-a2e2-3cdf2b563494@amd.com>
User-Agent: Mutt/1.5.21 (2010-09-15)

On Wed, Apr 03, 2024 at 04:08:35PM -0500, Kalra, Ashish wrote:
> 
> On 4/2/2024 5:42 PM, Michael Roth wrote:
> >On Tue, Apr 02, 2024 at 05:31:09PM -0500, Kalra, Ashish wrote:
> >>On 4/2/2024 5:09 PM, Tom Lendacky wrote:
> >>>On 3/12/24 13:47, Ashish Kalra wrote:
> >>>>From: Ashish Kalra <ashish.kalra@amd.com>
> >>>>
> >>>>RMP table start and end physical range may not be aligned to 2MB in
> >>>>the e820 tables causing fatal RMP page faults during kexec boot when
> >>>>new page allocations are done in the same 2MB page as the RMP table.
> >>>>Check if RMP table start and end physical range in e820_table is not
> >>>>aligned to 2MB and in that case use e820__range_update() to map this
> >>>>range to reserved.
> >>>>
> >>>>Override e820__memory_setup_default() to check and apply these RMP table
> >>>>fixups in e820_table before e820_table is used to setup
> >>>>e280_table_firmware and e820_table_kexec.
> >>>>
> >>>>Fixes: c3b86e61b756 ("x86/cpufeatures: Enable/unmask SEV-SNP CPU
> >>>>feature")
> >>>>Signed-off-by: Ashish Kalra <ashish.kalra@amd.com>
> >>>>---
> >>>>   arch/x86/virt/svm/sev.c | 52 +++++++++++++++++++++++++++++++++++++++++
> >>>>   1 file changed, 52 insertions(+)
> >>>>
> >>>>diff --git a/arch/x86/virt/svm/sev.c b/arch/x86/virt/svm/sev.c
> >>>>index cffe1157a90a..e0d7584df28f 100644
> >>>>--- a/arch/x86/virt/svm/sev.c
> >>>>+++ b/arch/x86/virt/svm/sev.c
> >>>>@@ -65,6 +65,8 @@ static u64 probed_rmp_base, probed_rmp_size;
> >>>>   static struct rmpentry *rmptable __ro_after_init;
> >>>>   static u64 rmptable_max_pfn __ro_after_init;
> >>>>   +static char *__init snp_rmptable_e820_fixup(void);
> >>>>+
> >>>>   static LIST_HEAD(snp_leaked_pages_list);
> >>>>   static DEFINE_SPINLOCK(snp_leaked_pages_list_lock);
> >>>>   @@ -160,9 +162,59 @@ bool snp_probe_rmptable_info(void)
> >>>>       pr_info("RMP table physical range [0x%016llx - 0x%016llx]\n",
> >>>>           probed_rmp_base, probed_rmp_base + probed_rmp_size - 1);
> >>>>   +    /*
> >>>>+     * Override e820__memory_setup_default() to do any RMP table fixups
> >>>>+     * for kexec if required.
> >>>>+     */
> >>>>+    x86_init.resources.memory_setup = snp_rmptable_e820_fixup;
> >>>This produces a build warning:
> >>>
> >>>WARNING: modpost: vmlinux: section mismatch in reference:
> >>>snp_probe_rmptable_info+0x95 (section: .text) -> x86_init (section:
> >>>.init.data)
> >>>WARNING: modpost: vmlinux: section mismatch in reference:
> >>>snp_probe_rmptable_info+0x99 (section: .text) -> snp_rmptable_e820_fixup
> >>>(section: .init.text)
> >>>
> >>Oh, so this requires snp_probe_rmptable_info() to be fixed to use the __init
> >>macro.
> >>
> >>I believe that snp_probe_rmptable_info() should be anyway using the __init
> >>macro and this fix for snp_probe_rmptable_info() needs to be sent as a
> >>separate patch and regardless of this patch getting merged or not.
> >I think you'll hit issues with:
> >
> >   bsp_determine_snp() ->           //non-__init
> >     snp_probe_rmptable_info()      //__init
> >
> >and bsp_determine_snp() sticks around as a function pointer assigned to
> >cpuinfo_x86 so I don't think you can use __init there.
> >
> >So might need to just drop __init from snp_rmptable_e820_fixup().
> 
> Actually, that will not help as snp_probe_rmptable_info() is *also*
> accessing x86_init.resources.memory_setup
> 

What if we flipped the whole flow? Borislav is adding a CC_ATTR to indicate
HOST_SEV_SNP support, we don't need to clear X86_FEATURE_SEV_SNP at this phase
(or at all for that matter). snp_probe_rmptable_info() can be done later
during kernel init, once e820 has been parsed.

One way of doing this would be to override x86_init.resources.memory_setup()
to do both snp_probe_rmptable_info() and snp_rmptable_e820_fixup().

What do you think?