Received-SPF: pass (google.com: domain of linux-kernel+bounces-17646-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45d1:ec00::1 as permitted sender) client-ip=2604:1380:45d1:ec00::1;
Message-ID: <35ab017e-bf9e-479e-8f06-06f12ba85ec8@intel.com>
Date: Fri, 5 Jan 2024 17:04:57 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v8 00/26] Enable CET Virtualization
Content-Language: en-US
To: "Edgecombe, Rick P" <rick.p.edgecombe@intel.com>
CC: "Gao, Chao" <chao.gao@intel.com>, "Hansen, Dave" <dave.hansen@intel.com>,
	"seanjc@google.com" <seanjc@google.com>, "peterz@infradead.org"
	<peterz@infradead.org>, "john.allen@amd.com" <john.allen@amd.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>, "pbonzini@redhat.com"
	<pbonzini@redhat.com>, "mlevitsk@redhat.com" <mlevitsk@redhat.com>
References: <20231221140239.4349-1-weijiang.yang@intel.com>
 <93f118670137933980e9ed263d01afdb532010ed.camel@intel.com>
 <5f57ce03-9568-4739-b02d-e9fac6ed381a@intel.com>
 <6179ddcb25c683bd178e74e7e2455cee63ba74de.camel@intel.com>
From: "Yang, Weijiang" <weijiang.yang@intel.com>
In-Reply-To: <6179ddcb25c683bd178e74e7e2455cee63ba74de.camel@intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?MzJzcTBacEF3QkpCVU9rVVZrZXN1NlExeFYzMHpjL2NUa3AzcTZUWjF6TFhr?=
 =?utf-8?B?akRjZFhDZHI3OHRQTHdoRVBnYjRJaFEyZWxCWjJzdkpQcGptK1k2L0FLeFJH?=
 =?utf-8?B?RFdsdU9RRmFtc2kvZ3VlODBDejF2MjFoMmdQS0VaZ1YzMDUzZTBiQitpNnBX?=
 =?utf-8?B?VTRrSVVwd0RrdkdOUlU2VjhMVDVuY2o2azUzbkJ1eS9YMlNIckorWGRiOHdv?=
 =?utf-8?B?cjkvajB5SVBMQk1uOVdETVNRSzFwdDQ2YU5vNzVHbjI0MW1jc1JsTGRTOWx0?=
 =?utf-8?B?Q0dIRkViQlJJVXNwNmN2RWszUjVWU3l2TFJiaUp0c0RhVFlEQ0MzK08xWUtM?=
 =?utf-8?B?eHdYaVNiK1Vxajd4MkE0TElCMXNEMEM0bGVnYjd3aGhpU0VsOEtLVUpqQmZO?=
 =?utf-8?B?ZXdhZjlaREtLMTNqY2dCd0RicGxkWnlpZWhMNUFuazdidUVrd3NBNytSUkhx?=
 =?utf-8?B?d1cyWVpLdG9oMTZpZzJ1NUtzakJUUks4ZWtJM1pwamFqTmlpY1dIMXd5bkZx?=
 =?utf-8?B?c0tUWmM4OEsyRTd3U1RvTnZVQ2ZMemNMZVF0QjBFeEFxVG5hWkhhQVdmMS9l?=
 =?utf-8?B?MjFuaEdxWjI4Vmt2dll6Z1ZOcklaN0RKYS84ZW1aWElxUWtJVUhwWnhoNjNV?=
 =?utf-8?B?MVBQVGxURXVlVGF1bjhZY1hjODM3dWRkYTd6NUF4MlE1NGY0MjJBY1J0ZWhs?=
 =?utf-8?B?MDJzWkZCOERSTVh6ZjVSenZSMkRKMUFJcEJOSlpJdHdzMkE1cXExdHdqMnJK?=
 =?utf-8?B?U1VlOFZ2WDNiRWtmQ21WWnN0cVRBdytaRkpxR2FWMExpUEdaeStuUzgyS2pW?=
 =?utf-8?B?ZGJlVWlNTGJ5SGtGSklORUc0L2tvQkNPbThBRndJS0lyRVlUZ2tGVTlleFQ3?=
 =?utf-8?B?K25ocmY0ZTVHZG02YVlxVDRNeXRtM1pmM1NCWUJHZERmYjhQWEZuTkMvOHFH?=
 =?utf-8?B?VzFUZzNSblVmZWlFMDErb3V2eWVGVlVvcWNIRnhEUmMxdlp0bUlESkZVckFP?=
 =?utf-8?B?ZXMwamI3MStTb0VveWdVNGVyMmtxU0JhOGhNZXkvdXZnQnlNMmpib0xmN1h5?=
 =?utf-8?B?ZzZwOHpla2NnbS93SlFWaXExNW9oanBpaE9GdE1xS1c2MzkyTEdtODBnaHpS?=
 =?utf-8?B?SUJWcnBTakpVUG9hRHBCT3VudzVGdW82cUVNcDNiRjN4dDV1dGVDWlNaZElW?=
 =?utf-8?B?cFJvL3h2TWtqSFFMS0VrclpmYTlMa3d6WVhtVndtVURoTlpyMFRvUTJNcTdx?=
 =?utf-8?B?Sk5DMk5uWThkN1c4SGY5V0tEcDhOSWo4QnI5V25jc2hjbG9SaFR0SEZEcVp1?=
 =?utf-8?B?RWRvWmNuT3Yva3VDQWM3WnE3V2NvNHlub2dHUmpMaytxWGpFak5Yb296SDNF?=
 =?utf-8?B?RlpRd0d6WG83eHRhRVliZkJRVFlwaVNOTHF3aHJQZU9uVDA0dzl1a01GaHEx?=
 =?utf-8?B?bFZ0SDh4TG9UQURUclpkYVhqazZIc0Z0NmkveENDby95OGdFUmZHTVVXenZn?=
 =?utf-8?B?VVVSZUJYc1ZuOXYxOFRZdmVWQWJKS1RxOG1tZzR1N0VHV0trbGVQR29uSkl5?=
 =?utf-8?B?MzluU2hyL1liald2citWTUZjcXc4T1Y5TkdUdkY1b3hzRllHZWh6QXdqcnhJ?=
 =?utf-8?B?QWRlcTMrZmVaM01xL0pLcU4xWTgxdVl5Rm1ESzFTcmg3RVk0Q3dKNllINHlE?=
 =?utf-8?B?K0U3ODg5dUxFWDdCRmNFTjEvQS9JS3FNa2JrWmJxOFIvb0hPbjdwWCttd0Ur?=
 =?utf-8?B?bHdGSTlNZXg3K1NnNVVnZnMvVXR4M3lrYXFhVzhNRlhEbDI4TWFYWVdLR0h4?=
 =?utf-8?B?QnRRWDZFOVlOZjV0aDVQU3B3TjV4MGJJdEduNzZyczFkd1FSZXFNNFU5OWpx?=
 =?utf-8?B?ekhHMTJwRzREZjVUZkVXdU05OVdaS0tkTk0wdGdhZUx6bXJLczFod1JpekVH?=
 =?utf-8?B?RkVLWFA1TjU1ZFFYWlM2U2tZK2ZoK1hLbldDb1pmY0xteUdzb1ZIdjNVSHFE?=
 =?utf-8?B?dmJWanA4OXppQXBIaDdRb3VLK0FoeWF1dHd2NTZ1OFY1a1dDV2hIR0xGbHdG?=
 =?utf-8?B?eHBaZGpmM3lnQVBMMTl5TC9MTER0UGZDZzM3WTZVK08vcFVUNkZwVkdWR3BL?=
 =?utf-8?B?N1ZEeFhRa0R5M2MyeHU0T2J3aisyOTlpYzIvdmV1S1JKZGo3WE5wZUVodkpB?=
 =?utf-8?B?Umc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 4c0a4328-c78f-4352-0997-08dc0dcd6a9c
X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB4965.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jan 2024 09:05:09.1577
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: kO4Axn8cXGhRPDPzeI2amxCnPjQoGIiYSLiLosWxzpIwsyECVcTKQ5keBFwS1lOdCC2duIv89PMn5XSEB8YIlA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB6742
X-OriginatorOrg: intel.com

On 1/5/2024 5:10 AM, Edgecombe, Rick P wrote:
> On Thu, 2024-01-04 at 15:11 +0800, Yang, Weijiang wrote:
[...]
>>> My understanding is that the KVM emulator kind of does what it has
>>> to
>>> keep things running, and isn't expected to emulate every possible
>>> instruction. With CET though, it is changing the behavior of
>>> existing
>>> supported instructions. I could imagine a guest could skip over CET
>>> enforcement by causing an MMIO exit and racing to overwrite the
>>> exit-
>>> causing instruction from a different vcpu to be an indirect
>>> CALL/RET,
>>> etc.
>> Can you elaborate the case? I cannot figure out how it works.
> The point that it should be possible for KVM to emulate call/ret with
> CET enabled. Not saying the specific case is critical, but the one I
> used as an example was that the KVM emulator can (or at least in the
> not too distant past) be forced to emulate arbitrary instructions if
> the guest overwrites the instruction between the exit and the SW fetch
> from the host.
>
> The steps are:
> vcpu 1                         vcpu 2
> -------------------------------------
> mov to mmio addr
> vm exit ept_misconfig
>                                 overwrite mov instruction to call %rax
> host emulator fetches
> host emulates call instruction
>
> So then the guest call operation will skip the endbranch check. But I'm
> not sure that there are not less exotic cases that would run across it.
> I see a bunch of cases where write protected memory kicks to the
> emulator as well. Not sure the exact scenarios and whether this could
> happen naturally in races during live migration, dirty tracking, etc.
> Again, I'm more just asking the exposure and thinking on it.

Now I get your points, I didn't think of exposure from guest and just thought of the
normal execution flow in guest, so I said let guest handle #CP directly.

Yes, I think we need to take these cases into account, as Sean suggested in following
replies, stopping emulation JMP/CALL/RET etc. instructions when guest CET is enabled
is effective and simple, I'll investigate the emulator code.

Thanks for raising the concerns!