Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <2c3dd773-31ca-d4b1-78a2-b39f3785c7c5@amd.com>
Date:   Thu, 23 Mar 2023 08:16:49 -0500
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.8.0
Subject: Re: [PATCH v16 8/8] x86/smpboot: Allow parallel bringup for SEV-ES
Content-Language: en-US
To:     Borislav Petkov <bp@alien8.de>,
        Usama Arif <usama.arif@bytedance.com>
Cc:     dwmw2@infradead.org, tglx@linutronix.de, kim.phillips@amd.com,
        brgerst@gmail.com, piotrgorski@cachyos.org,
        oleksandr@natalenko.name, arjan@linux.intel.com, mingo@redhat.com,
        dave.hansen@linux.intel.com, hpa@zytor.com, x86@kernel.org,
        pbonzini@redhat.com, paulmck@kernel.org,
        linux-kernel@vger.kernel.org, kvm@vger.kernel.org,
        rcu@vger.kernel.org, mimoja@mimoja.de, hewenliang4@huawei.com,
        seanjc@google.com, pmenzel@molgen.mpg.de, fam.zheng@bytedance.com,
        punit.agrawal@bytedance.com, simon.evans@bytedance.com,
        liangma@liangbit.com, gpiccoli@igalia.com,
        David Woodhouse <dwmw@amazon.co.uk>,
        Sabin Rapan <sabrapan@amazon.com>
References: <20230321194008.785922-1-usama.arif@bytedance.com>
 <20230321194008.785922-9-usama.arif@bytedance.com>
 <20230322224735.GAZBuFh9ld6FuYEyoH@fat_crate.local>
From:   Tom Lendacky <thomas.lendacky@amd.com>
In-Reply-To: <20230322224735.GAZBuFh9ld6FuYEyoH@fat_crate.local>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RzM2K0lUWUswZjRVY2wyS3N2amNtTkQ5UkY3cW1pSmhQYytXTG9JQmZwanFC?=
 =?utf-8?B?QVRzVkNpaDBTZ2RCaW9mQnlyV0FtUm5rSUIvUnVVZGE2YkJaMGpibFZaeUdx?=
 =?utf-8?B?N0hxKzdlclVqT1FaeTBRbXRpcWJlenhYaTNsU0VhY2ErOVpNMk0ySFhicGpF?=
 =?utf-8?B?ZlgxRW8vNlQxakVpdnZFakswejgwRHByZTdLTVhlVTNEZ2FRRzBYbEZ2M1Zr?=
 =?utf-8?B?N2tzQjVSblhyWWh3akFrZ0U4dlB2YkF0eWlvSVdmUVdRVjVUb1N5dXlUVEhr?=
 =?utf-8?B?WmxJMVNJUldMd3ZCZWRmUHFKQnpWMnQ3MHhsQmtLOFVVUDFzWkh6NjZWVkU4?=
 =?utf-8?B?ZEF4dmRRYWxQY0JzNFBEWXZoSzdjQ1hkMWVlZHBpalMxbmZZNXZtWlRKd1Fx?=
 =?utf-8?B?QytqU0FnTXQyUW5IbTR3YWl2L1JzOTNyVXU0ektNTWZVNDNsbWZqNkJWNStl?=
 =?utf-8?B?TzlrdGNFTHk1c1RaQ3BJaXpMWU1YOHFlSzY1NDBZazBsNkFCV1R5MS9MSnVo?=
 =?utf-8?B?N3dXUWc4WTY2Yzc5N0RQTHpJZTRPbm51Zk1qemRPL0RTZ05EYmJyVG9xUEo0?=
 =?utf-8?B?aE1ZSmpVRUV0a1o2SU9yZEtWZkh2bXk1T2ZOdkt2SXJxbm8xdURuZW1mOGlL?=
 =?utf-8?B?WWl2ay9Ld1F6b3VFK2MxaDY4ejc2Q0hUYisyMGFjWHlHZnFIL2V1MG8xVEhp?=
 =?utf-8?B?Q25pbHl1U0s1L0RRQVVDMU1HUmpTV0xSdlFLQzZvdkF2Z2M4UzhMbEFkeVY1?=
 =?utf-8?B?MXJZK0JmVzZvWDR0cUxuN3JXQUZGVnFXSEdTdHhZM0VEa3RxSVcxYnJhbDJi?=
 =?utf-8?B?TGorWk1qdVN0c1NoQU90NktzamRXeldyd0ZQT20reEpnTUoxRFdGWmhpOW9O?=
 =?utf-8?B?eUhTTnZtWlROQU5VOFAvcStkWkRTc1o5SXNGZ1BGWFcxMGRJL0hNNXVOb1hr?=
 =?utf-8?B?MkFwd2c3OStCb3pjNEFFQXBCQTl1bVBSTjZCZFVaQnZWdjJjQ2hnL1FreXpt?=
 =?utf-8?B?K244d09jRGZlR1RGRlVwaE5maUNGZXNDQ3g3dGJLZCtiQnBFYTVxbHNsYitL?=
 =?utf-8?B?WkdCOXpFSWNKNDJlN2QzQzJndUF5VTNOTE5mQytHbjVPQWpLUXoxa3ltemxE?=
 =?utf-8?B?aTlEZkNuTUJBTXRjMTdaQW92aUZ4Zks5eEcrZTJPejRxR3hSbEozVUt1NkhH?=
 =?utf-8?B?U2tMeDEwNWZlblhIb1EzaFo4d0pjVnUyT2x2Z0JUZzFTVHUvWURmSXdxT1lD?=
 =?utf-8?B?NVYwK1RZRW5HYVFDZWNadG43WVFYQ2EwMEFCYnk2WEtQb0oxYk1aVTJNcW5w?=
 =?utf-8?B?VzVDeU1kaWRzendiYjZZMHh4clJOZjhXdkMvRHl3Q085Nkd3Z0JRWmNGL2xD?=
 =?utf-8?B?WEIwRDc5L2t4dnJ2WFRFeVhnTXphNmg0NlB6TDRVeUswTXh6TkkxYWZ2WEFv?=
 =?utf-8?B?ZSt4VUVteldMa2hzVXlFejRBUFhCTjBDTis5WFRjN0ZyNm0wRklveWpQaE9X?=
 =?utf-8?B?bXRIcTErby80cXNMQlZjRm9PdmtIbDc4OGZPbGpoLy9CZEJpeHRyT2wwNVor?=
 =?utf-8?B?NG0wM1g0dTFZUmorZkxDTXQxYSt4OCtabnhTSEtYQmhnbnJzZVhSdU9KTzhN?=
 =?utf-8?B?REFVV1JJTlBKR1hiZ2tSc211ZnZwdGhzWVUybExaNnNmSFUvOU15L0tDbUJi?=
 =?utf-8?B?a1U4bWNwTDFKK0hkKzFJclBRTUZ4VmtraVpWVCtyVDd4NWtvVUdoSHJJc3NK?=
 =?utf-8?B?Z2NQV0U3RU1meXZWd2VWVU5OZm52UzFIQ0p4L1pkVGI1SUN0TmNoM0YvUlVE?=
 =?utf-8?B?NVB3QTNUWFpZQTdYZHJ4UHFMM3M0dWdrR292T0lIQWxDUUdHTTNQcGZ2NUs2?=
 =?utf-8?B?c25iZ3FIemZmUHVtdmpxaEh2YnRlM1dOVUd6WXYzOFFKM0paVTNkUmhBUzJz?=
 =?utf-8?B?TW5rcWRrZTJGcUlXWndQQlFzN3ZwaUFDNUxBcjRIZm1TNkZNQWY5M3BBWVpj?=
 =?utf-8?B?VTJUZzk3a1hIL0FZTDhLY3UwbThQWkhKczdoS2p1Vnh1ZWFRcXZsVXdZYWo4?=
 =?utf-8?B?cVFzckxWSVVwMWdVb090a3JHd3RXYU5zVG1VZmJlZkZqcWNSN1d1Rmo0YnhU?=
 =?utf-8?Q?B5oLP4yHz3RFOKig4fEglHDTR?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: fea24352-2b6f-42ce-92ac-08db2ba0dde8
X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB5229.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2023 13:16:52.6761
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: M2UDgv6leL8Q6F5BjAWXD7F9zrD8PMw31rvdOUjT03/Km054ml9PqtFeejAa5LymfYo47rQFDLsxnuwOhwWwtQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB4232
X-Spam-Status: No, score=-0.2 required=5.0 tests=DKIM_SIGNED,DKIM_VALID,
        DKIM_VALID_AU,DKIM_VALID_EF,FORGED_SPF_HELO,NICE_REPLY_A,
        RCVD_IN_DNSWL_NONE,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_NONE
        autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 3/22/23 17:47, Borislav Petkov wrote:
> On Tue, Mar 21, 2023 at 07:40:08PM +0000, Usama Arif wrote:
>> From: David Woodhouse <dwmw@amazon.co.uk>
>>
>> Enable parallel bringup for SEV-ES guests. The APs can't actually
>> execute the CPUID instruction directly during early startup, but they
>> can make the GHCB call directly instead, just as the VC trap handler
>> would do.
>>
>> Thanks to Sabin for talking me through the way this works.
>>
>> Suggested-by: Sabin Rapan <sabrapan@amazon.com>
>> Signed-off-by: David Woodhouse <dwmw@amazon.co.uk>
>> Signed-off-by: Usama Arif <usama.arif@bytedance.com>
>> Reviewed-by: Tom Lendacky <thomas.lendacky@amd.com>
>> ---
>>   arch/x86/coco/core.c              |  5 ++++
>>   arch/x86/include/asm/coco.h       |  1 +
>>   arch/x86/include/asm/sev-common.h |  3 +++
>>   arch/x86/include/asm/smp.h        |  5 +++-
>>   arch/x86/kernel/head_64.S         | 30 ++++++++++++++++++++++++
>>   arch/x86/kernel/smpboot.c         | 39 ++++++++++++++++++++++++++-----
>>   6 files changed, 76 insertions(+), 7 deletions(-)
> 
> How's the below (totally untested yet but it builds) instead after
> applying those two prerequisites:
> 
> https://lore.kernel.org/r/20230318115634.9392-1-bp@alien8.de
> 
> ---

...

> diff --git a/arch/x86/kernel/sev.c b/arch/x86/kernel/sev.c
> index ce371f62167b..96ff63cb5622 100644
> --- a/arch/x86/kernel/sev.c
> +++ b/arch/x86/kernel/sev.c
> @@ -1142,6 +1142,24 @@ void snp_set_wakeup_secondary_cpu(void)
>   	apic->wakeup_secondary_cpu = wakeup_cpu_via_vmgexit;
>   }
>   
> +u32 sev_get_cpuid_0b(void)

Maybe call this sev_early_get_cpuid_0b() or such to imply that the MSR 
protocol is being used and only retrieving / returning edx.

Also, since it is a function now and can be used at any point, the current 
GHCB MSR should be saved on entry and restored on exit.

Thanks,
Tom

> +{
> +	u32 eax, edx;
> +
> +	/* Request CPUID 0xB_EDX through GHCB protocol */
> +	native_wrmsr(MSR_AMD64_SEV_ES_GHCB,
> +		     (GHCB_CPUID_REQ_EDX << 30) | GHCB_MSR_CPUID_REQ,
> +		     0xb);
> +	VMGEXIT();
> +
> +	native_rdmsr(MSR_AMD64_SEV_ES_GHCB, eax, edx);
> +
> +	if ((eax & GHCB_MSR_INFO_MASK) == GHCB_MSR_CPUID_RESP)
> +		return edx;
> +
> +	return 0;
> +}
> +
>   int __init sev_es_setup_ap_jump_table(struct real_mode_header *rmh)
>   {
>   	u16 startup_cs, startup_ip;