Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <a1412227-151c-dc2a-1592-3eb71eb9f24f@amd.com>
Date:   Fri, 26 Aug 2022 21:56:55 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.11.0
Subject: Re: [PATCHv3 5/8] KVM: SVM: Add VNMI support in inject_nmi
Content-Language: en-US
To:     "Maciej S. Szmigiero" <mail@maciej.szmigiero.name>,
        "Shukla, Santosh" <santosh.shukla@amd.com>
Cc:     Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <seanjc@google.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Jim Mattson <jmattson@google.com>,
        Joerg Roedel <joro@8bytes.org>,
        Tom Lendacky <thomas.lendacky@amd.com>, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, mlevitsk@redhat.com
References: <20220810061226.1286-1-santosh.shukla@amd.com>
 <20220810061226.1286-6-santosh.shukla@amd.com>
 <bf9e8a9c-5172-b61a-be6e-87a919442fbd@maciej.szmigiero.name>
 <e10b3de6-2df0-1339-4574-8477a924b78e@amd.com>
 <f96b867f-4c32-4950-8508-234fe2cda7b9@maciej.szmigiero.name>
 <1062bf85-0d44-011b-2377-d6be1485ce65@amd.com>
 <3752b74b-74e1-00fd-d80d-41104e07fe95@maciej.szmigiero.name>
 <ce3c852e-4ee0-36df-383c-0957a6e02a6d@amd.com>
 <ddbb19b8-417f-f839-591d-a0610ea9629b@maciej.szmigiero.name>
 <ca464c8c-872f-f86b-6fca-7ef7b374a304@amd.com>
 <d30192e4-9e12-f770-e944-e3c38b9514b8@maciej.szmigiero.name>
From:   "Shukla, Santosh" <santosh.shukla@amd.com>
In-Reply-To: <d30192e4-9e12-f770-e944-e3c38b9514b8@maciej.szmigiero.name>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NUpkb3dFVVRFbW1rQnBrdW44VFRxRWdBRndxQm9IUTdzYW80ZGlhMC9pUlFR?=
 =?utf-8?B?ZWh2NFc1TDB5S3hCWnBwNmRoUGxVbzdyc2VHd0J6cGdjbTRiODhYRXZVWWJq?=
 =?utf-8?B?STRpRmJ0dUJoSFBWajRJd3hNc1kwdDZyZlU4SjFqMmFvcXEvVGUzVGlXNmd2?=
 =?utf-8?B?dGRIM2JCQ21ZZkJicU5hakJUKzh0MjZwWW9NZGNiRlQ2Zzg0TC9vM1ZySFgr?=
 =?utf-8?B?bWVsUXArUnVldkFuR3hWY0JoZUpFdUVBK2NxVkZEaFRpR0lpVTBCazJhMkFt?=
 =?utf-8?B?elNCWm5GRXhGV0FwK3B1TmVhUGlpS01jQkI5R3daYlVLVUdMZkxxb0g0M2R3?=
 =?utf-8?B?ZHdxODNyVjVEUDhwd2pMdXprSkZ0K2RORUJuMDgvM0FBcG9HcEx3dWx1Tksz?=
 =?utf-8?B?VWRCb1k0dXVPeXQ3cFNVUmxtT0RmSGVZK29RTGVZTG5nWFAzZUFIajFKNno4?=
 =?utf-8?B?d1RGeElvRjF6SFNMUDRZUWFHTDNYNlppTTFjbUp4TDRDOTVHbHBoQ0hCZ1Yw?=
 =?utf-8?B?a2p0MHNqcTNpZnpxUHhyTUtlMWgyL0lPcUZmd0hneWdtNGFGTWJ1ZzFMWWd1?=
 =?utf-8?B?QlJaUWpDNmFxVTNZTUo5U08yUG50cTliWENuRFVzM1VhQThreFpWanRuT28x?=
 =?utf-8?B?OExlMlNoUFlLeld6elU2N2JXZmVMajArZzQ2OHhGaWtNWUNTWWhzZ0J0UVZx?=
 =?utf-8?B?dktlWEltVlQ5MGNxcW5nTmd1Q2ZSaytRSEtsUW02TXVsRW0wQW9mdTVTcHJC?=
 =?utf-8?B?YmVhaFZmOVc1b3R6ZkoyMkhQb0pvYXMxYVYvM1RvUVVVRVd0UDd3d1lsRTho?=
 =?utf-8?B?NjhORS9kdlExbzhrZHIrd0lvRFhMbVNaQXdrNklzK2VKTkVvdnBUUU1KSGNM?=
 =?utf-8?B?RTZvMktjYkMxUFplZnQ1Q1Fsbjd0Uk1lM1ZUOU1OaVNMRkdiNTBkK2ZhTDd1?=
 =?utf-8?B?K3VBUGpjUGNpMnU4YkpSVFlXZEticEZQdkZ1UEZUdEV0UUovS3BseldwQkk4?=
 =?utf-8?B?YVRYL3RaV0lmeXIyaDQwYzF1K2cvRkNuVkRGSW1VSTdwSldRaWlYSC9IMEJL?=
 =?utf-8?B?Tkt5NFlhditJMllCV0M1Vk01eDJ3bXhHbzB4aG5wT3liT01iVUNrVDVUc3g3?=
 =?utf-8?B?QmFlU3lUQVM1Zms3SDFDR0FHVGxwSEhtQStOODVCVUwyVFp3dTNibWZUOElx?=
 =?utf-8?B?eTdLL2F2Tm9iTGNsMnNKc3QwOU9BMk5nWVdFSU9nUmkremhwbHl0endieXBD?=
 =?utf-8?B?UUM4N2EwbWRzb1VlbEsreEZhMUhrcktudzRIQmc5VklwaFVBQ21wV2RYYkwv?=
 =?utf-8?B?TDV5emVPcm9yeDRjWjBpUk5tTEFFOW83T2p3VERJRzZSTXBXTXd1cGZmb21x?=
 =?utf-8?B?WUhkVFhNcU1GWllLREFxaTNGS2djNkFiUlZ4RWZBcnlvVzdYU1R1U1d3V1Bh?=
 =?utf-8?B?RmFmaThzTHhRNDQxb0JrS0NMd3JJVis0d1Y3ZW1qMFpPaDBWTzNTMVRQM3FO?=
 =?utf-8?B?dHlPazd2d0dIcTNmOHlRWlpSYkh3QVlCMUt2MzZDNVptVGVRc3pwRXVxL0hP?=
 =?utf-8?B?eGNqc2UyM0NNK2NFTEVtcmU3MkRRcDNpbjZIeUgvMWY4SzRLbXZmejJEMUxH?=
 =?utf-8?B?VjVvOU5OSGt1VjQyZ09uWjNTa2kzdlVLRlMxMlY5U2tIK3VUYlVibVl4ZWJ2?=
 =?utf-8?B?U0pGRDNiR2pNT2FHdHZVV1BnUlZXaGswSmFUVTFXR0hEZjV6K0tJbHFrcVov?=
 =?utf-8?B?K2FsQ1Q5L05yZGZkRnJVMFRTYzEzMk9lbExxWkM4Sm1maW9NOGhzRlVoRHZY?=
 =?utf-8?B?NjR4RmRheWJlYXBlVjV2SFJjejFJK0xKejB0eURnMjkySklwdFFvTWltSmx5?=
 =?utf-8?B?Z1h2R3lmbHpLSUtqdVJLaVpId2dBMkF6T3hVVkpNdkxLVmN5ZlZWZlVGSFoz?=
 =?utf-8?B?ZEVMc3hwbURFczBTZG13b2IrWm50d3RjK21rQktZMWJ4ajgrV2Z1WGhWblhC?=
 =?utf-8?B?MEtFSzFRMERHeFNDVmdESENuVzUrYzJpWmhGS2VxRWdsR25sa1FkK2g1UW9r?=
 =?utf-8?B?WXhIelJiUUtoR1E3WFJkSDhVTUFFbWh0VVJoVlBNUWIyUVo4N1VLb00wN0kx?=
 =?utf-8?Q?u261kbovXRkO2bqAb4qj67ROu?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9ec63181-e7fa-4212-eb2d-08da877fd26f
X-MS-Exchange-CrossTenant-AuthSource: CY5PR12MB6323.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Aug 2022 16:27:09.4852
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: VlO5L1V/dB//0SJNHFxPofq+le4Natc9ZjZ21xQixwjYTYU6aXIXJ21MNTJwpLP5NT0rSwp/FS9wF+4nJsPBsw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4732
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 8/26/2022 5:50 PM, Maciej S. Szmigiero wrote:
> On 26.08.2022 11:35, Shukla, Santosh wrote:
>> On 8/25/2022 7:46 PM, Maciej S. Szmigiero wrote:
>>> On 25.08.2022 16:05, Shukla, Santosh wrote:
>>>> On 8/25/2022 6:15 PM, Maciej S. Szmigiero wrote:
>>>>> On 25.08.2022 12:56, Shukla, Santosh wrote:
>>>>>> On 8/24/2022 6:26 PM, Maciej S. Szmigiero wrote:
>>>>>>> On 24.08.2022 14:13, Shukla, Santosh wrote:
>>>>>>>> Hi Maciej,
>>>>>>>>
>>>>>>>> On 8/11/2022 2:54 AM, Maciej S. Szmigiero wrote:
>>>>>>>>> On 10.08.2022 08:12, Santosh Shukla wrote:
>>>>>>>>>> Inject the NMI by setting V_NMI in the VMCB interrupt control. processor
>>>>>>>>>> will clear V_NMI to acknowledge processing has started and will keep the
>>>>>>>>>> V_NMI_MASK set until the processor is done with processing the NMI event.
>>>>>>>>>>
>>>>>>>>>> Signed-off-by: Santosh Shukla <santosh.shukla@amd.com>
>>>>>>>>>> ---
>>>>>>>>>> v3:
>>>>>>>>>> - Removed WARN_ON check.
>>>>>>>>>>
>>>>>>>>>> v2:
>>>>>>>>>> - Added WARN_ON check for vnmi pending.
>>>>>>>>>> - use `get_vnmi_vmcb` to get correct vmcb so to inject vnmi.
>>>>>>>>>>
>>>>>>>>>>       arch/x86/kvm/svm/svm.c | 7 +++++++
>>>>>>>>>>       1 file changed, 7 insertions(+)
>>>>>>>>>>
>>>>>>>>>> diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
>>>>>>>>>> index e260e8cb0c81..8c4098b8a63e 100644
>>>>>>>>>> --- a/arch/x86/kvm/svm/svm.c
>>>>>>>>>> +++ b/arch/x86/kvm/svm/svm.c
>>>>>>>>>> @@ -3479,7 +3479,14 @@ static void pre_svm_run(struct kvm_vcpu *vcpu)
>>>>>>>>>>       static void svm_inject_nmi(struct kvm_vcpu *vcpu)
>>>>>>>>>>       {
>>>>>>>>>>           struct vcpu_svm *svm = to_svm(vcpu);
>>>>>>>>>> +    struct vmcb *vmcb = NULL;
>>>>>>>>>>       +    if (is_vnmi_enabled(svm)) {
>>>>>>>>>
>>>>>>>>> I guess this should be "is_vnmi_enabled(svm) && !svm->nmi_l1_to_l2"
>>>>>>>>> since if nmi_l1_to_l2 is true then the NMI to be injected originally
>>>>>>>>> comes from L1's VMCB12 EVENTINJ field.
>>>>>>>>>
>>>>>>>>
>>>>>>>> Not sure if I understood the case fully.. so trying to sketch scenario here -
>>>>>>>> if nmi_l1_to_l2 is true then event is coming from EVTINJ. .which could
>>>>>>>> be one of following case -
>>>>>>>> 1) L0 (vnmi enabled) and L1 (vnmi disabled)
>>>>>>>
>>>>>>> As far as I can see in this case:
>>>>>>> is_vnmi_enabled() returns whether VMCB02's int_ctl has V_NMI_ENABLE bit set.
>>>>>>>
>>>>>>
>>>>>> For L1 with vnmi disabled case - is_vnmi_enabled()->get_vnmi_vmcb() will return false so the
>>>>>> execution path will opt EVTINJ model for re-injection.
>>>>>
>>>>> I guess by "get_vnmi_vmcb() will return false" you mean it will return NULL,
>>>>> since this function returns a pointer, not a bool.
>>>>>
>>>>
>>>> Yes, I meant is_vnmi_enabled() will return false if vnmi param is unset.
>>>>
>>>>> I can't see however, how this will happen:
>>>>>> static inline struct vmcb *get_vnmi_vmcb(struct vcpu_svm *svm)
>>>>>> {
>>>>>>       if (!vnmi)
>>>>>>           return NULL;
>>>>>           ^ "vnmi" variable controls whether L0 uses vNMI,
>>>>>          so this variable is true in our case
>>>>>
>>>>
>>>> No.
>>>>
>>>> In L1 case (vnmi disabled) - vnmi param will be false.
>>>
>>> Perhaps there was a misunderstanding here - the case here
>>> isn't the code under discussion running as L1, but as L0
>>> where L1 not using vNMI - L1 here can be an old version of KVM,
>>> or Hyper-V, or any other hypervisor.
>>>
>>
>> Ok.
>>
>>> In this case L0 is re-injecting an EVENTINJ NMI into L2 on
>>> the behalf of L1.
>>> That's when "nmi_l1_to_l2" is true.
>>   hmm,. trying to understand the event re-injection flow -
>> First L1 (non-vnmi) injecting event to L2 guest, in-turn
>> intercepted by L0, 
> 
> That's right, the L1's VMRUN of L2 gets intercepted by L0.
> 
>> L0 sees event injection through EVTINJ
> 
> It sees that L1 wants to inject an NMI into L2 via VMCB12 EVTINJ.
> 
>> so sets the 'nmi_l1_to_l2' var 
> 
> That's right, L0 needs to keep track of this fact.
> 
>> and then L0 calls svm_inject_nmi()
> 
> Not yet - at this point svm_inject_nmi() is NOT called
> (rather than, VMCB12 EVTINJ is directly copied into VMCB02 EVTINJ).
> 
> Now L0 does the actual VMRUN of L2.
> 
> Let's say that there is an intervening VMExit during delivery of
> that NMI to L2, of type which is handled by L0 (perhaps a NPF on
> L2 IDT or so).
> 
> In this case the NMI will be returned in VMCB02 EXITINTINFO and
> needs to be re-injected into L2 on the next VMRUN,
> again via EVTINJ.
> 
> That's when svm_inject_nmi() will get called to re-inject
> that NMI.
> 
>> to re-inject event in L2 - is that correct (nmi_l1_to_l2) flow?
> Hope the flow is clear now.
> 

Yes, Thank-you for the clarification :).

Santosh.

>>
>> Thanks,.
>> Santosh
> 
> Thanks,
> Maciej