Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <d06bb33e-047f-c849-de6a-246bc361c7af@gmail.com>
Date:   Tue, 27 Jun 2023 18:57:28 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.12.0
Subject: Re: [EXTERNAL] Re: [PATCH 5/9] x86/hyperv: Use vmmcall to implement
 Hyper-V hypercall in sev-snp enlightened guest
From:   Tianyu Lan <ltykernel@gmail.com>
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     kys@microsoft.com, haiyangz@microsoft.com, wei.liu@kernel.org,
        decui@microsoft.com, tglx@linutronix.de, mingo@redhat.com,
        bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org,
        hpa@zytor.com, daniel.lezcano@linaro.org, arnd@arndb.de,
        michael.h.kelley@microsoft.com, Tianyu Lan <tiala@microsoft.com>,
        linux-arch@vger.kernel.org, linux-hyperv@vger.kernel.org,
        linux-kernel@vger.kernel.org, vkuznets@redhat.com
References: <20230601151624.1757616-1-ltykernel@gmail.com>
 <20230601151624.1757616-6-ltykernel@gmail.com>
 <20230608132127.GK998233@hirez.programming.kicks-ass.net>
 <8b93aa93-903f-3a69-77f9-0c6b694d826b@gmail.com>
In-Reply-To: <8b93aa93-903f-3a69-77f9-0c6b694d826b@gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk


On 6/8/2023 11:15 PM, Tianyu Lan wrote:
> On 6/8/2023 9:21 PM, Peter Zijlstra wrote:
>> On Thu, Jun 01, 2023 at 11:16:18AM -0400, Tianyu Lan wrote:
>>> From: Tianyu Lan <tiala@microsoft.com>
>>>
>>> In sev-snp enlightened guest, Hyper-V hypercall needs
>>> to use vmmcall to trigger vmexit and notify hypervisor
>>> to handle hypercall request.
>>>
>>> There is no x86 SEV SNP feature flag support so far and
>>> hardware provides MSR_AMD64_SEV register to check SEV-SNP
>>> capability with MSR_AMD64_SEV_ENABLED bit. ALTERNATIVE can't
>>> work without SEV-SNP x86 feature flag. May add later when
>>> the associated flag is introduced.
>>>
>>> Signed-off-by: Tianyu Lan <tiala@microsoft.com>
>>> ---
>>>   arch/x86/include/asm/mshyperv.h | 44 ++++++++++++++++++++++++---------
>>>   1 file changed, 33 insertions(+), 11 deletions(-)
>>>
>>> diff --git a/arch/x86/include/asm/mshyperv.h 
>>> b/arch/x86/include/asm/mshyperv.h
>>> index 31c476f4e656..d859d7c5f5e8 100644
>>> --- a/arch/x86/include/asm/mshyperv.h
>>> +++ b/arch/x86/include/asm/mshyperv.h
>>> @@ -61,16 +61,25 @@ static inline u64 hv_do_hypercall(u64 control, 
>>> void *input, void *output)
>>>       u64 hv_status;
>>>   #ifdef CONFIG_X86_64
>>> -    if (!hv_hypercall_pg)
>>> -        return U64_MAX;
>>> +    if (hv_isolation_type_en_snp()) {
>>> +        __asm__ __volatile__("mov %4, %%r8\n"
>>> +                     "vmmcall"
>>> +                     : "=a" (hv_status), ASM_CALL_CONSTRAINT,
>>> +                       "+c" (control), "+d" (input_address)
>>> +                     :  "r" (output_address)
>>> +                     : "cc", "memory", "r8", "r9", "r10", "r11");
>>> +    } else {
>>> +        if (!hv_hypercall_pg)
>>> +            return U64_MAX;
>>> -    __asm__ __volatile__("mov %4, %%r8\n"
>>> -                 CALL_NOSPEC
>>> -                 : "=a" (hv_status), ASM_CALL_CONSTRAINT,
>>> -                   "+c" (control), "+d" (input_address)
>>> -                 :  "r" (output_address),
>>> -                THUNK_TARGET(hv_hypercall_pg)
>>> -                 : "cc", "memory", "r8", "r9", "r10", "r11");
>>> +        __asm__ __volatile__("mov %4, %%r8\n"
>>> +                     CALL_NOSPEC
>>> +                     : "=a" (hv_status), ASM_CALL_CONSTRAINT,
>>> +                       "+c" (control), "+d" (input_address)
>>> +                     :  "r" (output_address),
>>> +                    THUNK_TARGET(hv_hypercall_pg)
>>> +                     : "cc", "memory", "r8", "r9", "r10", "r11");
>>> +    }
>>>   #else
>>
>> Remains unanswered:
>>
>> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Flkml.kernel.org%2Fr%2F20230516102912.GG2587705%2540hirez.programming.kicks-ass.net&data=05%7C01%7CTianyu.Lan%40microsoft.com%7C60a576eb67634ffa27b108db68234d5a%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C638218273105649705%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000%7C%7C%7C&sdata=MFj67DON0K%2BUoUJbeaIA5oVTxyrzO3fb5DbxYgDWwX0%3D&reserved=0
>>
>> Would this not generate better code with an alternative?
> 
> 
> Hi Peter:
>      Thanks to review. I put the explaination in the change log.
> 
> "There is no x86 SEV SNP feature(X86_FEATURE_SEV_SNP) flag
> support so far and hardware provides MSR_AMD64_SEV register
> to check SEV-SNP capability with MSR_AMD64_SEV_ENABLED bit
> ALTERNATIVE can't work without SEV-SNP x86 feature flag."
> There is no cpuid leaf bit to check AMD SEV-SNP feature.
> 
> After some Hyper-V doesn't provides SEV and SEV-ES guest before and so
> may reuse X86_FEATURE_SEV and X86_FEATURE_SEV_ES flag as alternative
> feature check for Hyper-V SEV-SNP guest. Will refresh patch.
> 

Hi Peter:
      I tried using alternative for "vmmcall" and CALL_NOSPEC in a single
Inline assembly. The output is different in the SEV-SNP mode. When SEV-
SNP is enabled, thunk_target is not required. While it's necessary in
the non SEV-SNP mode. Do you have any idea how to differentiate outputs 
in the single Inline assembly which just like alternative works for
assembler template.