Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <ca464c8c-872f-f86b-6fca-7ef7b374a304@amd.com>
Date:   Fri, 26 Aug 2022 15:05:46 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.11.0
Subject: Re: [PATCHv3 5/8] KVM: SVM: Add VNMI support in inject_nmi
Content-Language: en-US
To:     "Maciej S. Szmigiero" <mail@maciej.szmigiero.name>,
        "Shukla, Santosh" <santosh.shukla@amd.com>
Cc:     Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <seanjc@google.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Jim Mattson <jmattson@google.com>,
        Joerg Roedel <joro@8bytes.org>,
        Tom Lendacky <thomas.lendacky@amd.com>, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org, mlevitsk@redhat.com
References: <20220810061226.1286-1-santosh.shukla@amd.com>
 <20220810061226.1286-6-santosh.shukla@amd.com>
 <bf9e8a9c-5172-b61a-be6e-87a919442fbd@maciej.szmigiero.name>
 <e10b3de6-2df0-1339-4574-8477a924b78e@amd.com>
 <f96b867f-4c32-4950-8508-234fe2cda7b9@maciej.szmigiero.name>
 <1062bf85-0d44-011b-2377-d6be1485ce65@amd.com>
 <3752b74b-74e1-00fd-d80d-41104e07fe95@maciej.szmigiero.name>
 <ce3c852e-4ee0-36df-383c-0957a6e02a6d@amd.com>
 <ddbb19b8-417f-f839-591d-a0610ea9629b@maciej.szmigiero.name>
From:   "Shukla, Santosh" <santosh.shukla@amd.com>
In-Reply-To: <ddbb19b8-417f-f839-591d-a0610ea9629b@maciej.szmigiero.name>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VU5NdElGeE1wRGNRVEQ2Q3d2c1JGSWRqcHJWY015cTJYRXFlU09CK3lnNStM?=
 =?utf-8?B?RGUxd0g2L1pDUkNsWXp4UVY5NHNuUDhtaEw4cjFURFJDUUQwVVdnYVR3cGJt?=
 =?utf-8?B?R1VCSzdTWXpsTHpiM2FickJQZHE2RjJsT0lUdFVZbElmVnJuOXBCb0Fkemp5?=
 =?utf-8?B?aDVKMVdVQ2VlTnd2RVRLeVowZFVSaVplaGVXS1ZNYVpUOWdhbDIvb0Vrc2Uw?=
 =?utf-8?B?Qmd1NnFMMUMwVGtXODMwWmFoT3pmUzVvVzVROXU4alRVbzJqWkI5bVdnQ0N3?=
 =?utf-8?B?ZnFSV1JwemJmdXNtM0ZLdXU0ekNhbStIbCtUNHBpelNOY3FpMGNJdmVuMDZk?=
 =?utf-8?B?SENkKzZkL1hEbmZ6Skl6QmJSUUhrUUYva0cvWEZGVzZUb3ZBYUxiL1NscmJO?=
 =?utf-8?B?SFJvNllTdnJtZ3JEVVdOSnJpYWxRNEYwcU9LRWhncXJPU0prWFVlbVE4dWF4?=
 =?utf-8?B?VnQ3WmM0ZHl6ZHRXZ0x5S1VxZlZHQ3NGWXJaRUxoYkVNZkpJVExUbnEzOFUw?=
 =?utf-8?B?S09zOFdFQjdLeG9MSzFYUW9hTHVSWWNGTkh5enNkS01GZWFiY1RaWW9DQ2dQ?=
 =?utf-8?B?aitTc2FpQVIybUVxSlVsbCtFRXZzdU8zZDN2ekZXekc4VVZSVjY0STB1V1Rr?=
 =?utf-8?B?QU83VjVyemlGd2FqbFEvNFh0d0JmOWlvMnEwemdCR0k5cDIzVUcvbk96NGlU?=
 =?utf-8?B?WmJXQjRqSm9Kb1BhSzRKSU1CME95bEhTSXBKbjlkR1RWRmRsZjM0eUxtZk1P?=
 =?utf-8?B?NElQd2daS2RySS9QeGlab0MwTUU3WG5DeG43UEkwcmFmUlJHV0MvcndBNWtu?=
 =?utf-8?B?dzV3VmhzQXh4NUZia0tsQUhzWnh2a25lZFlyU3dWc2wvUGhzeGwyay9xL3Ez?=
 =?utf-8?B?Y2MxRkM2MzkrZUZlTzdKZzZTeUVTemorRWpTc2lFdXdQeW1lTzFINWRiang0?=
 =?utf-8?B?NlBGek5IUnJvTUZIb0ZTZm1CT0paQXNBRnk5bjhRcjNyVUZNNEtUQUx5eU52?=
 =?utf-8?B?dDZUNjVjcWNkMTlDZENXemNqeGU3NFhhdXUrL2gvcW9WbjhQUkhwSVJQUUNn?=
 =?utf-8?B?WmhrWmZlVm1tUWFEMkk1MStocWkwZU9zU3NPU09BbDh0MWcyQnhDR2lldm9m?=
 =?utf-8?B?cFZRUWpLSW4zaGxkUVlvTmtxKzY4UVZpcWQydCtmdUJwTURHRHJGeHpzcXFs?=
 =?utf-8?B?Y3drSmx1MW1VRFZkT2Q3STlnNUVXakM0OGQwenNsdDQ3c0xIMExGTEs1SjlO?=
 =?utf-8?B?RHhSeno0ZGhVSnE0TVExeDVldW93VDBMV0Rvb0xmcldUL0h3WW9GU0tmejZy?=
 =?utf-8?B?V1NlNXBoT1VaWGJKcG9TT29rK1c0b2pNd1N6aWVNVGg4QlJZR3pJN1lNVzIv?=
 =?utf-8?B?a1I3KzRpbzBoUG5GMVBHN2FxbWVza0RwdndRbnZsT2plTjg4aStGUUF3NGJt?=
 =?utf-8?B?OTkwQnl0QnRHbmtBKzJXbklaR0J6MVFPWHdZZU1oUDNFbDNTRWNYOHBhVlNh?=
 =?utf-8?B?VnlKanpDM21GaXJ0TGVzOHRvblNUNmx2M1ZWSHROQ0ZER05SUU5EcmhaWTg2?=
 =?utf-8?B?RmdRR3FRTzJCNVZxSm9KQTZOZVRkZ3ZZOElpMkhiNTBTZ1Z5dDZKdTdYUVRI?=
 =?utf-8?B?S3oreXFIRXgzdkhxbzJ3VGloTTA1bUhtdWVLWnNPN042YVozWjhvTmdQMVcz?=
 =?utf-8?B?cStYWDJkMmZZMEZTeHNpWHQvWE9WOUltdFV3NHJ0QnV0K2RBTEtSTzdHeDk2?=
 =?utf-8?B?RUZmeXR1dWxXSktaOE9NbXVhNWFhYVMzM0o4R1NtcjZRK0VsUkZ6c0wyNk55?=
 =?utf-8?B?YmdnOEdZV3VPUDVaWlhrK3k5Ny9tdXZuWmlleVd5Y3RwU0x6OGtXU3hrNnUv?=
 =?utf-8?B?RDUxSzJnSTlCQksxVnR5Y1NhMVpLY2pWTmZLV2s2OTJYK0M1b1pNQmJxRjd4?=
 =?utf-8?B?LzBBRUdHYXRKTTRUUWcySG5sQTZaeXFkeENFaStaTnJkL3orSWF1NDllQkFo?=
 =?utf-8?B?WWwxTE5LVk1pVnR5aGF4K1BMcFlpOGdCNVFCQkRyMjRhNWQyTEJGaTRaTzA0?=
 =?utf-8?B?Vm0rRUMvUURyR0R3N1BSZFZyeHFBWFQ5emw1VVRyRzlKYjRDNzl4dDNja3lD?=
 =?utf-8?Q?aiUDkwCmCdrADUcDB5KiXi9ib?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6072a13b-e455-4984-d22b-08da874660e9
X-MS-Exchange-CrossTenant-AuthSource: IA1PR12MB6305.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Aug 2022 09:35:57.4129
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: uig2xgi3Nyuk6GyamHxJwu7TRaX6kX/96VUXkMPmL3OACPilYMR6F7xLu7/1apZM69Z3wBC83WhvCr5s/iQGGg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR12MB2340
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 8/25/2022 7:46 PM, Maciej S. Szmigiero wrote:
> On 25.08.2022 16:05, Shukla, Santosh wrote:
>> On 8/25/2022 6:15 PM, Maciej S. Szmigiero wrote:
>>> On 25.08.2022 12:56, Shukla, Santosh wrote:
>>>> On 8/24/2022 6:26 PM, Maciej S. Szmigiero wrote:
>>>>> On 24.08.2022 14:13, Shukla, Santosh wrote:
>>>>>> Hi Maciej,
>>>>>>
>>>>>> On 8/11/2022 2:54 AM, Maciej S. Szmigiero wrote:
>>>>>>> On 10.08.2022 08:12, Santosh Shukla wrote:
>>>>>>>> Inject the NMI by setting V_NMI in the VMCB interrupt control. processor
>>>>>>>> will clear V_NMI to acknowledge processing has started and will keep the
>>>>>>>> V_NMI_MASK set until the processor is done with processing the NMI event.
>>>>>>>>
>>>>>>>> Signed-off-by: Santosh Shukla <santosh.shukla@amd.com>
>>>>>>>> ---
>>>>>>>> v3:
>>>>>>>> - Removed WARN_ON check.
>>>>>>>>
>>>>>>>> v2:
>>>>>>>> - Added WARN_ON check for vnmi pending.
>>>>>>>> - use `get_vnmi_vmcb` to get correct vmcb so to inject vnmi.
>>>>>>>>
>>>>>>>>      arch/x86/kvm/svm/svm.c | 7 +++++++
>>>>>>>>      1 file changed, 7 insertions(+)
>>>>>>>>
>>>>>>>> diff --git a/arch/x86/kvm/svm/svm.c b/arch/x86/kvm/svm/svm.c
>>>>>>>> index e260e8cb0c81..8c4098b8a63e 100644
>>>>>>>> --- a/arch/x86/kvm/svm/svm.c
>>>>>>>> +++ b/arch/x86/kvm/svm/svm.c
>>>>>>>> @@ -3479,7 +3479,14 @@ static void pre_svm_run(struct kvm_vcpu *vcpu)
>>>>>>>>      static void svm_inject_nmi(struct kvm_vcpu *vcpu)
>>>>>>>>      {
>>>>>>>>          struct vcpu_svm *svm = to_svm(vcpu);
>>>>>>>> +    struct vmcb *vmcb = NULL;
>>>>>>>>      +    if (is_vnmi_enabled(svm)) {
>>>>>>>
>>>>>>> I guess this should be "is_vnmi_enabled(svm) && !svm->nmi_l1_to_l2"
>>>>>>> since if nmi_l1_to_l2 is true then the NMI to be injected originally
>>>>>>> comes from L1's VMCB12 EVENTINJ field.
>>>>>>>
>>>>>>
>>>>>> Not sure if I understood the case fully.. so trying to sketch scenario here -
>>>>>> if nmi_l1_to_l2 is true then event is coming from EVTINJ. .which could
>>>>>> be one of following case -
>>>>>> 1) L0 (vnmi enabled) and L1 (vnmi disabled)
>>>>>
>>>>> As far as I can see in this case:
>>>>> is_vnmi_enabled() returns whether VMCB02's int_ctl has V_NMI_ENABLE bit set.
>>>>>
>>>>
>>>> For L1 with vnmi disabled case - is_vnmi_enabled()->get_vnmi_vmcb() will return false so the
>>>> execution path will opt EVTINJ model for re-injection.
>>>
>>> I guess by "get_vnmi_vmcb() will return false" you mean it will return NULL,
>>> since this function returns a pointer, not a bool.
>>>
>>
>> Yes, I meant is_vnmi_enabled() will return false if vnmi param is unset.
>>
>>> I can't see however, how this will happen:
>>>> static inline struct vmcb *get_vnmi_vmcb(struct vcpu_svm *svm)
>>>> {
>>>>      if (!vnmi)
>>>>          return NULL;
>>>          ^ "vnmi" variable controls whether L0 uses vNMI,
>>>         so this variable is true in our case
>>>
>>
>> No.
>>
>> In L1 case (vnmi disabled) - vnmi param will be false.
> 
> Perhaps there was a misunderstanding here - the case here
> isn't the code under discussion running as L1, but as L0
> where L1 not using vNMI - L1 here can be an old version of KVM,
> or Hyper-V, or any other hypervisor.
> 

Ok.

> In this case L0 is re-injecting an EVENTINJ NMI into L2 on
> the behalf of L1.
> That's when "nmi_l1_to_l2" is true.
 
hmm,. trying to understand the event re-injection flow -
First L1 (non-vnmi) injecting event to L2 guest, in-turn
intercepted by L0, L0 sees event injection through EVTINJ
so sets the 'nmi_l1_to_l2' var and then L0 calls svm_inject_nmi()
to re-inject event in L2 - is that correct (nmi_l1_to_l2) flow?

Thanks,.
Santosh
> Since the code is physically running on L0 (which makes use of vNMI)
> it has the "vnmi" param set.
> 
> So is_vnmi_enabled() will return true and vNMI will be used
> for the re-injection instead of the required EVENTINJ.
> 
>> In L0 case (vnmi enabled) - vnmi param will be true.
>>
>> So in L1 case, is_vnmi_enabled() will return false and
>> in L0 case will return true.
>>
>> Thanks,
>> Santosh
> 
> Thanks,
> Maciej