Message-ID: <54a4ca2e-3389-dd48-1d1f-8b385fbfc0cf@amd.com>
Date:   Fri, 17 Mar 2023 13:07:01 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.8.0
Subject: Re: Does earlyprintk=ttyS0 work for an AMD SNP guest on KVM?
Content-Language: en-US
To:     Dexuan Cui <decui@microsoft.com>,
        Jeremi Piotrowski <jpiotrowski@linux.microsoft.com>
Cc:     Borislav Petkov <bp@alien8.de>,
        "sandipan.das@amd.com" <sandipan.das@amd.com>,
        "Gupta, Pankaj" <pankaj.gupta@amd.com>,
        "ray.huang@amd.com" <ray.huang@amd.com>,
        "brijesh.singh@amd.com" <brijesh.singh@amd.com>,
        "michael.roth@amd.com" <michael.roth@amd.com>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "x86@kernel.org" <x86@kernel.org>,
        Tianyu Lan <Tianyu.Lan@microsoft.com>,
        "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
References: <SA1PR21MB13359DBABC5625368E369A42BFA09@SA1PR21MB1335.namprd21.prod.outlook.com>
 <20230216091431.GA10166@linuxonhyperv3.guj3yctzbm1etfxqx2vob5hsef.xx.internal.cloudapp.net>
 <SA1PR21MB13358707043E2901958819AEBFA09@SA1PR21MB1335.namprd21.prod.outlook.com>
 <e06d90cc-544b-5280-f8cd-b25684214b4a@linux.microsoft.com>
 <SA1PR21MB13352DCAA7371B90CB8A637CBFA69@SA1PR21MB1335.namprd21.prod.outlook.com>
From:   Tom Lendacky <thomas.lendacky@amd.com>
In-Reply-To: <SA1PR21MB13352DCAA7371B90CB8A637CBFA69@SA1PR21MB1335.namprd21.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TzI5Yk4xa3NyTjFaKzQzZU5NNXJ2SUNPNm0wNXpyelZuZzlRRU0zUXBrZFg3?=
 =?utf-8?B?VzhFVE1UaFJjUjNaajg5Nzh5Nk1KZ3AwU3lCNVUzR0NKa0E5V09USnl6aDBG?=
 =?utf-8?B?dkRCMW9PMFhJQ3B2Vyt4ak54WFloOTJyQ0hhY25LbnhNNGhXK2RlK1I3aC9o?=
 =?utf-8?B?SW9iUjR5aDE3dmsvbW9EQnhxdmFhY3Z3eGtweUJFam5ZTllveFFUc1NVMTFV?=
 =?utf-8?B?d1FLa2lOOU1yRjRLNEo4ZnBBN2hLTno1M2xwbDBjWUMrcitwN29ocVcvcEhy?=
 =?utf-8?B?SkNFb2JLczNRUnVjamc3UFY2SjZWaHJtdzd0aWhWV2lCOCt2VzdXZGxlR0wr?=
 =?utf-8?B?S2tkRlBENXFWM0pXdm95RjRVZENKOEJVbi80WEtPTVhZaWJrS0tWcGpzbUpO?=
 =?utf-8?B?ckk5b0h4NXNqdUZtRG5tUDU2alZvMzZGTjVieERlNTBjdlh6Z015RDVpbGYr?=
 =?utf-8?B?b01Hb3AzTVF6Z0VsangzSDlSMXBCcGorT3Y2YjJxMGRWVVpkMXpTdStCeFJM?=
 =?utf-8?B?bitvaFFmZkw2ejRlMGNsRVNiZkpvMjMzNTZkcitxWUp2OHh0S2dnR1AxT2Nq?=
 =?utf-8?B?OG5yTGxLU2NkV2k3WnJsdDRJVU9pcm91Y2hCMS9xQ2dlVmxMTGloejdQR0tr?=
 =?utf-8?B?anlGZlFyekh0NHpjU1BjQzluTlhWZmduSGtEZDRPaGFBUnNYT3lOeHI4OU9Y?=
 =?utf-8?B?MndCL0kwZSs3SmhKZGQ3NWs3azJsT2lHNHNNZWN0OStGZEZ0dDdmMmx6VTlW?=
 =?utf-8?B?UXpDZGZFNUhQVTBmVkFhTDBuVUpWMlU0MDZ4dmlNM2RhYzhoZG1rUnN4K3ht?=
 =?utf-8?B?OXZSSUtCMmN3bXZ0LzJxT3RwNExkMHBOWjRicHl3OUwxSnRCWnN5aUFYVFUw?=
 =?utf-8?B?QmYxTzRaeUlzUXdRYjVucUNCNCs5ZVdScmxPUzVmS2lNVFJ0djhJc0UraVhC?=
 =?utf-8?B?bUplQ1dKWHdkSFZNdmU1QWZDTE5rOE1ldFAxNzRTMnRMMGluR0paQUtCM28x?=
 =?utf-8?B?dW1WMm1WZlZPWm1JbW9SU1RHeWFJZUQ2WDVIczNuSzFzWWVQNWtObGpPQUR2?=
 =?utf-8?B?bW9pRGNFVVEybC9UaGd5bTFUQ3R2U0tFQlQ3TGd2ZzJWZ1IzK3BxNHl6R1ZV?=
 =?utf-8?B?eTR5STY2amh0V1lZOTZveUQ5WVZSM0U0Y2ZPZUwyOXZFMWR5RmN1MVF1NlpI?=
 =?utf-8?B?NGJBQXYraUhqY1Y5OXRkSkNBQWM0MEkrdmFKM0k2Yitid0tiRXlpeGthOCtV?=
 =?utf-8?B?UEJxZ1JSV1BmUUNEUjlyaHdTNXV0QVprZXdMbmVON1lFTHNVN3hOUWRNRFFT?=
 =?utf-8?B?bTJnYWprdXYwaEJ3VWtTcC9tSUdNc0JjNnU1dTFrcUlBMzIya2F0dFNZSVdD?=
 =?utf-8?B?VXdJSWFxTU8xSzQ0NlIvUkExdk5tckFPeGNiU2gxWnBRN3AvOTBPeVFnRFgr?=
 =?utf-8?B?ZXpjM1o0K1RoWlVLejQ4U0w1UGpwY25qN0t0cDUzT3QvaGlEZzRNaEtkamts?=
 =?utf-8?B?M28yc0pBT3NwNURnSkdhOWFpOGFxT0RXVThlSnJML2srZWx6OFZxNEtxUERH?=
 =?utf-8?B?Q21MZHlOb0l2K1dTTlNWcWdxbGFtNDBCM1NuZFhMT1VDbTNCK2lzNkdCNkts?=
 =?utf-8?B?LzQ3YW5qajJXQUl6bWRhdTZDb2lLd1EycG04L2psRWM3RFVqUWlOSW1SbkNF?=
 =?utf-8?B?dEVPRHlkaXo0bjJTczg4WmNmOVI1Y1lhY0RCbFA5R1IvTCtZMzg2bnBpUTdn?=
 =?utf-8?B?L3YvSGN6WE96TDBlVTlCaEk5VzhIMWJxWjgxK0kva1RkTGdqVklMa3hzazVR?=
 =?utf-8?B?SVNPOUF0VVR2M044UDdGU3FzME9iU0xxcHlOTVFqdVcyUm00RFN0aHJibDJX?=
 =?utf-8?B?UWtaUWpYODhmdHlROUR6UmR3YkgrU25FWE1tM0FaNXBEU3RQaC9IVVhZdVdn?=
 =?utf-8?B?NlBGeFRnOU13eGRSM29YTGNBc0R3TXc1QkJEMzBTYXI1WmxkQTJMTHhqazdZ?=
 =?utf-8?B?Z0l0QmRlQ0NJYUFZamJUakhwZnhHNkdCbVcweXUxV2NpckllZzczZnlyZlp2?=
 =?utf-8?B?d0xMeHc5ZVppQncxZkJqeWJtUHZQNXVJMWlsWVIzVkE5K3JoOS9KR2J6YWxC?=
 =?utf-8?Q?VGmAx1X8joemJd3JVT9EdqdVH?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c160acbc-64bc-48e6-48b6-08db271269dd
X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB5229.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Mar 2023 18:07:04.3779
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 6cx9CT3ZlU5kyQI2DXNfdBLEifiI+8dvcg9rTXZ5IH8MYZLyI2d8D3ITwBMf6A9eM0b6W3gK54eu+U1ifUN8OA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH3PR12MB8583
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 2/17/23 20:54, Dexuan Cui wrote:
>> From: Jeremi Piotrowski <jpiotrowski@linux.microsoft.com>
>> Sent: Friday, February 17, 2023 4:51 AM
>> To: Dexuan Cui <decui@microsoft.com>
>>> [...]
>>> The comment before the first branch says:
>>>    On 4-level paging, p4d_offset(top_level_pgt, 0) is equal to 'top_level_pgt'.
>>>
>>> IIUC this means 'top_level_pgt' is equal to '_pgtable'? i.e. without
>>> CONFIG_RANDOMIZE_BASE, pgt_data.pgt_buf_size should be 0.
>>>
>>> Not sure why it's not getting into the first branch for you.
>>
>> Sorry, I got two things confused here. The relevant part of the comment is this:
>> "If we came here via startup_32(), cr3 will be _pgtable already".
>>
>> Booting a (non-SNP) guest via BIOS I end up in the first branch. Upstream SNP
>> support requires OVMF (UEFI) so we'll always reach the kernel in 64-bit mode
>> (startup_64?), and end up in the second branch.
>>
>> Jeremi
> 
> Here I'm running a C-bit mode SNP guest on Hyper-V via "direct-boot" (i.e.
> I run Set-VMFirmware to tell Hyper-V to boot the kernel directly without
> UEFI). Looks like arch/x86/boot/compressed/head_64.S: startup_32 runs
> first and calls startup_64 later (?) This might explain why I'm getting into
> the first branch, which I hope could be fixed by someone...

It sounds like there aren't enough pages available to satisfy the page 
split in order to make the GHCB shared. Have you tried changing 
BOOT_INIT_PGT_SIZE to increase it by 1 page. Splitting the page will 
require an additional page table, I think that is all that would be needed.

Thanks,
Tom