Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.35 as permitted sender) client-ip=23.128.96.35;
Date:   Fri, 8 Dec 2023 09:56:31 +0800
From:   Chao Gao <chao.gao@intel.com>
To:     "Li, Xin3" <xin3.li@intel.com>
CC:     "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "linux-doc@vger.kernel.org" <linux-doc@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
        "linux-kselftest@vger.kernel.org" <linux-kselftest@vger.kernel.org>,
        "seanjc@google.com" <seanjc@google.com>,
        "pbonzini@redhat.com" <pbonzini@redhat.com>,
        "corbet@lwn.net" <corbet@lwn.net>,
        "kys@microsoft.com" <kys@microsoft.com>,
        "haiyangz@microsoft.com" <haiyangz@microsoft.com>,
        "wei.liu@kernel.org" <wei.liu@kernel.org>,
        "Cui, Dexuan" <decui@microsoft.com>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "mingo@redhat.com" <mingo@redhat.com>,
        "bp@alien8.de" <bp@alien8.de>,
        "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
        "x86@kernel.org" <x86@kernel.org>, "hpa@zytor.com" <hpa@zytor.com>,
        "vkuznets@redhat.com" <vkuznets@redhat.com>,
        "peterz@infradead.org" <peterz@infradead.org>,
        "Shankar, Ravi V" <ravi.v.shankar@intel.com>
Subject: Re: [PATCH v1 13/23] KVM: VMX: Handle VMX nested exception for FRED
Message-ID: <ZXJ3z0yCqnHm52sC@chao-email>
References: <20231108183003.5981-1-xin3.li@intel.com>
 <20231108183003.5981-14-xin3.li@intel.com>
 <ZVMkVmBPVfaMjDTL@chao-email>
 <SA1PR11MB67348D3637C2BC6B107C5CCAA8B1A@SA1PR11MB6734.namprd11.prod.outlook.com>
 <SA1PR11MB6734EFF17E15C68AAD12A227A884A@SA1PR11MB6734.namprd11.prod.outlook.com>
 <ZXGFcuwwmkBlqq5s@chao-email>
 <SA1PR11MB67342A44E1C2E17E127AA44AA88BA@SA1PR11MB6734.namprd11.prod.outlook.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <SA1PR11MB67342A44E1C2E17E127AA44AA88BA@SA1PR11MB6734.namprd11.prod.outlook.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?2rcLwWwekIRLlYct7i02xhVE+Mt3TyRQd6AfrvIiv4IvPew/fqKm57MCnwk2?=
 =?us-ascii?Q?SxZ94dMx7sScZ4CHJtPlzifWrIYkUV6W4RcBKifpSYIZLfkDosjsDwDaQj6H?=
 =?us-ascii?Q?DwGe/wpoxwdqeY2Yq7UC9HMkQ47+QaB88kjWCc8oXai9kOuYtkBSAXQyq07F?=
 =?us-ascii?Q?zkIyxTG3U/BTWV24K6oyWNf+zSpfVPGMR1IxfEL5ETtN96p6y0bGFmoWwqwV?=
 =?us-ascii?Q?XLHaZNsdpWwpXO17ppUEpMb2ytQJRqMx4cVl+kIZo1V6GGfZ3cPsuu+Una24?=
 =?us-ascii?Q?pzHegLP43jNSJh+vw6gs7Cj/qpKIXkxrwY2W+U8gAkY6b6/mfEJ/aFVbD1gH?=
 =?us-ascii?Q?B3Xnf0p9+Q0na6gzCLfjHBLbxBHhLzDDjCchHFZbTWzPhzH6hJNmEzlfLWeX?=
 =?us-ascii?Q?AEd+Es2kt24oeroI8Lu5rgE+fv3zCvuUaYpTCbIfka78z/WFfCGhd0pZ+Qb+?=
 =?us-ascii?Q?+rLsHezBaqJpeF4Gczrwg7d3EKmoYd3s9vnsXswmxXxTpxsLZ3pWeQqL0IG4?=
 =?us-ascii?Q?9FhyqrGLAzNAk0nZZz//UcAUqhUx/dfMDCKGKQ6/jn6K2ZU3dS/Fac+Ljlvn?=
 =?us-ascii?Q?l1GDo4wR8qJSz04fsUvLtLm/XcG5h8NEmTq8tZVDdcjZo9821X529HQObetW?=
 =?us-ascii?Q?Gy5FN6UZrGraDrtevUmQ4BeQI0RuJlmEs7qullpeEC/R00Gd3imjDenmBMmo?=
 =?us-ascii?Q?ZEY2rVoKHwATWryNqJoOjwmyCglluKVbp4ODe3Nx0qVhbBSaVAu42xOjSFGv?=
 =?us-ascii?Q?bBXwp6tB5FDUbyAPflGNb5pGGzmsSByqMQigoJtOltoYv2BFhzl9JbrI4iFm?=
 =?us-ascii?Q?jGkrTnsIIkSLrYzLUPornE94q+xEyen2jJ8IeO+t0R48YPEBssdq/aaXuhZ3?=
 =?us-ascii?Q?oLNRouvy87g8j84wQNYcoqZvJ7DsOCNG/zmyVZk3K0w5ioSln7NUBM3zCxAI?=
 =?us-ascii?Q?30MxF47ObgXASCWSbsEn8Ll9IeF5mLRLnm5n9dMnT7bMTiEMRrJe6fkeK1LD?=
 =?us-ascii?Q?hlqlpiQbpz+Q/8i+/b8tLFFBMB4Js3xYzPBm/osEWh97Gsp506WnvEICjcU8?=
 =?us-ascii?Q?yBxMLQMNBay1chlMVtGHBzV4kOnWs8fHHjtyIZvQ5MTcfy2FdZEAnjwAtUeL?=
 =?us-ascii?Q?HQElpPZeTdumktJ4TKP/air1fsmqBoNJU3B/obMQWb+8AjkqedrPk3MslWl5?=
 =?us-ascii?Q?BXlsUU+iNpvA6mnt2l0TzGwhx2gxukgZcIIQsPP8P2h/7j4im5onLPiH+Qdf?=
 =?us-ascii?Q?F2n9rv6SovROeLPwf/pDHA8iOviUnGe5HCj6Bms7hHKkpF9aVF99MvTH9D3m?=
 =?us-ascii?Q?/O/TkOTwM339qNR8/VCV+nYM5J2CJ2R+zIqK6g91An6O2m9NHyMk/P54uY5f?=
 =?us-ascii?Q?Ppuwg8FC0cr4mF32BCxrlB9n945LRExW2QTRhMz8b2Rg/uMHzUJXKZZh7+Hg?=
 =?us-ascii?Q?w6xSiv215kYity9A74G3XimOTEoFEso6XoZikEqf3BxMt/Ju82PN9R6WKWaH?=
 =?us-ascii?Q?ZkqVzyPfO2Bfkrfv+ttJdUtgmyY/tNikYQWL/spb9Hsz65u/HM/ramutP6cY?=
 =?us-ascii?Q?2ntanI4nl/NKXDv4H1foqunFwjTwBZIlFUZAz2wm?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 5468d439-62ce-4ad8-5b0c-08dbf790edcf
X-MS-Exchange-CrossTenant-AuthSource: CH3PR11MB8660.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 Dec 2023 01:56:44.7457
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: uehR7lT0zt1K2k9DRILTSM/GVZChnPAlFMxa3qh2otcMynn/kQjMH4CiYry6z2jzKrgRJcNI7cfeWwVtAbqCAQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR11MB4574
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on groat.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (groat.vger.email [0.0.0.0]); Thu, 07 Dec 2023 17:57:01 -0800 (PST)

On Thu, Dec 07, 2023 at 06:09:46PM +0800, Li, Xin3 wrote:
>> >> > Exiting-event identification can also have bit 13 set, indicating a
>> >> > nested exception encountered and caused VM-exit. when reinjecting the
>> >> > exception to guests, kvm needs to set the "nested" bit, right? I
>> >> > suspect some changes to e.g., handle_exception_nmi() are needed.
>> >>
>> >> The current patch relies on kvm_multiple_exception() to do that.  But TBH, I'm
>> >> not sure it can recognize all nested cases.  I probably should revisit it.
>> >
>> >So the conclusion is that kvm_multiple_exception() is smart enough, and
>> >a VMM doesn't have to check bit 13 of the Exiting-event identification.
>> >
>> >In FRED spec 5.0, section 9.2 - New VMX Feature: VMX Nested-Exception
>> >Support, there is a statement at the end of Exiting-event identification:
>> >
>> >(The value of this bit is always identical to that of the valid bit of
>> >the original-event identification field.)
>> >
>> >It means that even w/o VMX Nested-Exception support, a VMM already knows
>> >if an exception is a nested exception encountered during delivery of
>> >another event in an exception caused VM exit (exit reason 0).  This is
>> >done in KVM through reading IDT_VECTORING_INFO_FIELD and calling
>> >vmx_complete_interrupts() immediately after VM exits.
>> >
>> >vmx_complete_interrupts() simply queues the original exception if there is
>> >one, and later the nested exception causing the VM exit could be cancelled
>> >if it is a shadow page fault.  However if the shadow page fault is caused
>> >by a guest page fault, KVM injects it as a nested exception to have guest
>> >fix its page table.
>> >
>> >I will add comments about this background in the next iteration.
>> 
>> is it possible that the CPU encounters an exception and causes VM-exit during
>> injecting an __interrupt__? in this case, no __exception__ will be (re-)queued
>> by vmx_complete_interrupts().
>
>I guess the following case is what you're suggesting:
>KVM injects an external interrupt after shadow page tables are nuked.
>
>vmx_complete_interrupts() are called after each VM exit to clear both
>interrupt and exception queues, which means it always pushes the
>deepest event if there is an original event.  In the above case, the
>original event is the external interrupt KVM just tried to inject.

in my understanding, your point is:
1. if bit 13 of the Exiting-event identification is set. the original-event
identification field should be valid.
2. vmx_complete_interrupts() is done immediately after VM exits and reads
original-event identification and reinjects the event there.
3. if KVM injects the exception in exiting-event identification
to guest, KVM doesn't need to read the bit 13 because kvm_multiple_exception()
is "smart enough" and recognize the exception as nested-exception because if
bit 13 is 1, one exception must has been queued in #2.

my question is:
what if the event in original-event identification is an interrupt e.g.,
external interrupt or NMI, rather than exception. vmx_complete_interrupts()
won't queue an exception, then how can KVM or kvm_multiple_exception() know the
exception that caused VM-exit is an nested exception w/o reading bit 13 of the
Exiting-event identification?