Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id 7FA9BC636D6 for ; Thu, 16 Feb 2023 09:14:37 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S229541AbjBPJOg (ORCPT ); Thu, 16 Feb 2023 04:14:36 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57842 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S229815AbjBPJOe (ORCPT ); Thu, 16 Feb 2023 04:14:34 -0500 Received: from linux.microsoft.com (linux.microsoft.com [13.77.154.182]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 60F5513D61; Thu, 16 Feb 2023 01:14:32 -0800 (PST) Received: by linux.microsoft.com (Postfix, from userid 1112) id E19042010BA9; Thu, 16 Feb 2023 01:14:31 -0800 (PST) DKIM-Filter: OpenDKIM Filter v2.11.0 linux.microsoft.com E19042010BA9 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.microsoft.com; s=default; t=1676538871; bh=su4MyopmGks9+XinlO9URPSoTj3r37ukK0Z6205Xb1U=; h=Date:From:To:Cc:Subject:References:In-Reply-To:From; b=MOPEOVjogcXLnxi9sng4dFKrUjv46A0s7xGOc+WhSNTPdmmosBJiPV6WGUQF0Olk3 oDxHAgDyt4slQZvQDiNxdBDRG+Jnrc1TT6b35SFonXvu78z5kyXha6VCDxZIsznVBb 0NjNlp1WeNqZoUnA1d7rcpSLNqVhxwNHfoBEfGb8= Date: Thu, 16 Feb 2023 01:14:31 -0800 From: Jeremi Piotrowski To: Dexuan Cui Cc: Tom Lendacky , Borislav Petkov , "sandipan.das@amd.com" , "Gupta, Pankaj" , "ray.huang@amd.com" , "brijesh.singh@amd.com" , "michael.roth@amd.com" , "kvm@vger.kernel.org" , "x86@kernel.org" , Tianyu Lan , "linux-hyperv@vger.kernel.org" , "linux-kernel@vger.kernel.org" Subject: Re: Does earlyprintk=ttyS0 work for an AMD SNP guest on KVM? Message-ID: <20230216091431.GA10166@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net> References: MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: User-Agent: Mutt/1.5.21 (2010-09-15) Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Thu, Feb 16, 2023 at 04:40:14AM +0000, Dexuan Cui wrote: > Hi all, > With the earlyprintk=ttyS0 kernel parameter, a C-bit mode Linux SNP guest > on Hyper-V always decides to crash via sev_es_terminate() in > do_boot_stage2_vc(), because early_setup_ghcb() fails: > > early_setup_ghcb() -> > set_page_decrypted() -> > set_clr_page_flags() -> > split_large_pmd() -> > alloc_pgt_page() fails to allocate memory. > > static void *alloc_pgt_page(void *context) > { > ... > /* Validate there is space available for a new page. */ > if (pages->pgt_buf_offset >= pages->pgt_buf_size) { > ... > return NULL; > } > ... > } > > alloc_pgt_page() fails to allocate memory because both > pages->pgt_buf_offset and pages->pgt_buf_size are zero. > > > pgt_data.pgt_buf_size is zero because of this line in > initialize_identity_maps() > pgt_data.pgt_buf_size = BOOT_PGT_SIZE - BOOT_INIT_PGT_SIZE; > > void initialize_identity_maps(void *rmode) > { > ... > top_level_pgt = read_cr3_pa(); > if (p4d_offset((pgd_t *)top_level_pgt, 0) == (p4d_t *)_pgtable) { > pgt_data.pgt_buf = _pgtable + BOOT_INIT_PGT_SIZE; > pgt_data.pgt_buf_size = BOOT_PGT_SIZE - BOOT_INIT_PGT_SIZE; > memset(pgt_data.pgt_buf, 0, pgt_data.pgt_buf_size); > } else { > pgt_data.pgt_buf = _pgtable; > pgt_data.pgt_buf_size = BOOT_PGT_SIZE; > memset(pgt_data.pgt_buf, 0, pgt_data.pgt_buf_size); > top_level_pgt = (unsigned long)alloc_pgt_page(&pgt_data); I just tested an SNP guest on KVM with and without CONFIG_RANDOMIZE_BASE. In both cases we end up in the else() branch. With CONFIG_RANDOMIZE_BASE BOOT_PGT_SIZE=0x13000 Without CONFIG_RANDOMMIZE_BASE BOOT_PGT_SIZE=0x6000. So in both cases pgt_data.pgt_buf_size != 0. Getting into that first branch would require having 5-level paging supported (CONFIG_X86_5LEVEL=y) and enabled inside the guest, I don't have that on any hardware I have access to. Jeremi > } > > In arch/x86/include/asm/boot.h, BOOT_PGT_SIZE equals > BOOT_INIT_PGT_SIZE if CONFIG_RANDOMIZE_BASE is not defined > (which is my case): > > # define BOOT_INIT_PGT_SIZE (6*4096) > > # ifdef CONFIG_RANDOMIZE_BASE > ... > # ifdef CONFIG_X86_VERBOSE_BOOTUP > # define BOOT_PGT_SIZE (19*4096) > # else /* !CONFIG_X86_VERBOSE_BOOTUP */ > # define BOOT_PGT_SIZE (17*4096) > # endif > # else /* !CONFIG_RANDOMIZE_BASE */ > # define BOOT_PGT_SIZE BOOT_INIT_PGT_SIZE > # endif > > I think this means: if CONFIG_RANDOMIZE_BASE is not defined, > earlyprintk=ttyS0 also doesn't work for an SNP guest on KVM? > Sorry I don't have a KVM environment at hand to test it. > > If I define CONFIG_RANDOMIZE_BASE, my C-bit mode SNP guest crashes > even ealier -- it looks like CONFIG_RANDOMIZE_BASE is incompatible > with my guest on Hyper-V due to some reason I don't know. > > Do you always use CONFIG_RANDOMIZE_BASE for a SNP guest on KVM > and does earlyprintk=ttyS0 work for you? > > Can you please share your thoughts? Thanks! > > Thanks, > -- Dexuan