Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <05938690-53f4-0173-9f3b-5acc729982a4@intel.com>
Date:   Mon, 26 Jun 2023 14:46:20 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Thunderbird/102.12.0
Subject: Re: [PATCH v3 00/21] Enable CET Virtualization
To:     Sean Christopherson <seanjc@google.com>
CC:     <pbonzini@redhat.com>, <kvm@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>, <peterz@infradead.org>,
        <rppt@kernel.org>, <binbin.wu@linux.intel.com>,
        <rick.p.edgecombe@intel.com>, <john.allen@amd.com>
References: <20230511040857.6094-1-weijiang.yang@intel.com>
 <ZIufL7p/ZvxjXwK5@google.com>
 <147246fc-79a2-3bb5-f51f-93dfc1cffcc0@intel.com>
 <ZIyiWr4sR+MqwmAo@google.com>
 <c438b5b1-b34d-3e77-d374-37053f4c14fa@intel.com>
 <ZJYF7haMNRCbtLIh@google.com>
Content-Language: en-US
From:   "Yang, Weijiang" <weijiang.yang@intel.com>
In-Reply-To: <ZJYF7haMNRCbtLIh@google.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NDZzV2JDNG9Dcm5sTmxrMlJJQUpaSlVxZXRLUEdCRkRsSTNodFBJeGsxRzlV?=
 =?utf-8?B?dVJhWXpOU1p0QmRHTC8yUE5DdUMxV0huSFFDSUdSRUlLdWRpbGx0UEVxVldO?=
 =?utf-8?B?MU9WWjYwMWJKOWhzYllrRWJqcytWTXlobGlJRzhYWEltYmhVcUJMQnBKVkVX?=
 =?utf-8?B?TFp6SlIvSy9PMHpYN1h0dVVzZjVRUXVkQmdMczNzUWNDOGw2UXFQKzRxM2dS?=
 =?utf-8?B?Q05adzl2bUdnMmFuemxJbUpzUDNQL2F5QTh1TWZsL3BLNXVFa29WTXFIWjcw?=
 =?utf-8?B?eWU1RGJKcExlL1Z3K25VY3BWV1VVTnVkamVNdlhrZWtrVSs0akZQSDU5VTZ3?=
 =?utf-8?B?SEdUQndxdGRIYU9KejRUTFUrUzZBdUJTQ2p6RVZHM0NTUVcrRmNBdDAzbmtF?=
 =?utf-8?B?RFVGUFdOVUZiWDJ3UWFjTXRUei9aaldhcGZzZDVxdnR5UXRTQkNpVFA3UUlk?=
 =?utf-8?B?MXRSVUdsQkRkeXF0djBZazNRZThySU9jc1NyR0V5c1dRZVVRNEk1azVTK2xh?=
 =?utf-8?B?Zkt5M1o2NWVJdUhLdkZGMk53b3l3UXp0Rjl5NzZOcklaeW1NTnk1UFlwZ2dt?=
 =?utf-8?B?dmFLYm1wanM0M1Qrb1NrYlFPYU44V3Y4S1piSWx5Zzl1eXdOc0ZFeTVzVlZO?=
 =?utf-8?B?NFBBTjlib2ZJSHNSSzJNZms5SERjQzVLQzlxU05mbTc5elBIbjJwQnd5VGYv?=
 =?utf-8?B?WDd1VEZoOFNXSytJN3lkRGxDU2dzbU91c1FhK2JRdW1jL3F6bzE5WGJTb0hK?=
 =?utf-8?B?M3JJZFJCTEFCNG1RTjZXT0RtME5ETElLa3RTQU5FS1RoTnhUTUQrY1BEVVA4?=
 =?utf-8?B?czdBZVJvY0lmOHlNblJ1ZGdMYTBtMjIyNFNhWmVGeGcvSldka0JaNnFibjZU?=
 =?utf-8?B?NjhuWWhWazFPOCszWUdXV3BPU0UwU3BHQW9DN2huOUFKK2ViWC9vU01lZFhT?=
 =?utf-8?B?Q2ZPUEl2WlArUmFpcXNYZW5FZmljLzN2NSt0Y1RWcGdubDVvTnZsclpMUXZB?=
 =?utf-8?B?Ujk2aHM3VXlVMU9MTGNQQ0loSlU2Wm81UlE4UWpZQ09GRXE0YUdkRHZMUHJL?=
 =?utf-8?B?UTZJMlc1bWZiVVlDQW4rYVJ6em5zT0dpUFJVZXQ5dmVWMkJSTXNLa2VmWE1J?=
 =?utf-8?B?WTRKNlExeVp3eWJxQ0M0RFpTbEVqY25LZWhSUXZoQlZnKzV4WWRaV0hRZG5a?=
 =?utf-8?B?NHVlRFlKUGFsdzFNbmUyZ0Qzb0pmdXRHUjhkVXVYVXAyUjVjd3hiZlprUXhI?=
 =?utf-8?B?d3pFMjk4NWM0VEJrUVFyV1ZVYWNZaG9Xa3dvRkpWTW1lZWlWdGxGMWdKUzFF?=
 =?utf-8?B?ODA3UWp6L1RNYVVjVXRMQnlPdkpoekNyaGhSQk8zWVJjRjBGOGhTZVgzSzhz?=
 =?utf-8?B?VmE3elc2V0lGT1dFWnhQVkFuYU1Wb21uVlBtQjBBems5ZDl2aERTUDRFMkpu?=
 =?utf-8?B?bDBxbjBySGd1VXV0Rk8yRkhUN0FDaFM5eFBWeUF6eWVFYXU1V0dlekk1U3Fj?=
 =?utf-8?B?YVFnOGg3YStmSlNWbFpsaU9JdTB1Q0hmTDA3cDhuMnJRWndmZVBXdWlMZ0JT?=
 =?utf-8?B?R0g0OWlaRTZoOUpwNVZsQnJqNFNKU3l6WEVnQWhFV3JmcFRRSEk2eUtPN2J6?=
 =?utf-8?B?WHQ0ZnJmN2dQNlBVeHg4V0FmT2Naa0FhSkcvVDQxODlzVk1PK0haUTIvSVF4?=
 =?utf-8?B?VElPbGYvMEM0MERCMVVOaEY0RFp6bitGVVFNK1BHQS8vWXZ4QjR6TlF2cUp3?=
 =?utf-8?B?UHVXQnp3SEE5ZThUL3dVV2tCVFJabzNqYmJ6ZzB6SFZ1RjluaTZ2QlZoR08z?=
 =?utf-8?B?UllJTHh5dE9DZmZVNU00SW5zZGFPT2RkaWt6UUU0cnNBVWk0SFI0eTg3NUFK?=
 =?utf-8?B?YTNXTUhXcW5Xd1NodDU4bjllb3h1TlFqMHlPL1kwNnJjTXZCRTJNLzN2bFBK?=
 =?utf-8?B?cHhGVm1MQ1h4ZEp6d1VRTWh1QmRRak16VG5VaWFHL2RNMmdIbEU5Y0VJNVB0?=
 =?utf-8?B?MFBIZVlUYXlIKzJnUVdLU2FHbkZZazdhVU5nSDhNMWxWWnE4TVpaVlNtcE9J?=
 =?utf-8?B?cnFSWWZ3ajhOb2NDNW83Y3ZuVkRxdUVhV0Q0bW9yRXdHQi83cDd2UjBKZzFS?=
 =?utf-8?B?M1lsVVZQbW8xR2MwdXhucG8zeDhLeU54OWtCVnp4cDZIZkEyQnpQQ1ltSU44?=
 =?utf-8?B?N0E9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 315702c7-188f-43f9-f1f7-08db761112b4
X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB4965.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 26 Jun 2023 06:46:30.7244
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9QEodK30OxER0kh9MAqGRsyKjWAtfzIqRPdtaluMXksEq/atGn7rdFCPicd7eS1beaBCYPVZsR/tRgFusmmcdA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA2PR11MB5019
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        RCVD_IN_DNSWL_MED,SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 6/24/2023 4:51 AM, Sean Christopherson wrote:
> On Mon, Jun 19, 2023, Weijiang Yang wrote:
>> On 6/17/2023 1:56 AM, Sean Christopherson wrote:
>>> On Fri, Jun 16, 2023, Weijiang Yang wrote:
>>>> On 6/16/2023 7:30 AM, Sean Christopherson wrote:
>>>>> On Thu, May 11, 2023, Yang Weijiang wrote:
>>>>>> The last patch is introduced to support supervisor SHSTK but the feature is
>>>>>> not enabled on Intel platform for now, the main purpose of this patch is to
>>>>>> facilitate AMD folks to enable the feature.
>>>>> I am beyond confused by the SDM's wording of CET_SSS.
>>>>>
>>>>> First, it says that CET_SSS says the CPU isn't buggy (or maybe "less buggy" is
>>>>> more appropriate phrasing).
>>>>>
>>>>>      Bit 18: CET_SSS. If 1, indicates that an operating system can enable supervisor
>>>>>      shadow stacks as long as it ensures that certain supervisor shadow-stack pushes
>>>>>      will not cause page faults (see Section 17.2.3 of the Intel® 64 and IA-32
>>>>>      Architectures Software Developer’s Manual, Volume 1).
>>>>>
>>>>> But then it says says VMMs shouldn't set the bit.
>>>>>
>>>>>      When emulating the CPUID instruction, a virtual-machine monitor should return
>>>>>      this bit as 0 if those pushes can cause VM exits.
>>>>>
>>>>> Based on the Xen code (which is sadly a far better source of information than the
>>>>> SDM), I *think* that what the SDM is trying to say is that VMMs should not set
>>>>> CET_SS if VM-Exits can occur ***and*** the bit is not set in the host CPU.  Because
>>>>> if the SDM really means "VMMs should never set the bit", then what on earth is the
>>>>> point of the bit.
>>>> I need to double check for the vague description.
>>>>
>>>>   From my understanding, on bare metal side, if the bit is 1, OS can enable
>>>> SSS if pushes won't cause page fault. But for VM case, it's not recommended
>>>> (regardless of the bit state) to set the bit as vm-exits caused by guest SSS
>>>> pushes cannot be fully excluded.
>>>>
>>>> In other word, the bit is mainly for bare metal guidance now.
>>>>
>>>>>> In summary, this new series enables CET user SHSTK/IBT and kernel IBT, but
>>>>>> doesn't fully support CET supervisor SHSTK, the enabling work is left for
>>>>>> the future.
>>>>> Why?  If my interpretation of the SDM is correct, then all the pieces are there.
>>> ...
>>>
>>>> And also based on above SDM description, I don't want to add the support
>>>> blindly now.
>>> *sigh*
>>>
>>> I got filled in on the details offlist.
>>>
>>> 1) In the next version of this series, please rework it to reincorporate Supervisor
>>>      Shadow Stack support into the main series, i.e. pretend Intel's implemenation
>>>      isn't horribly flawed.
>> Let me make it clear, you want me to do two things:
>>
>> 1)Add Supervisor Shadow Stack  state support(i.e., XSS.bit12(CET_S)) into
>> kernel so that host can support guest Supervisor Shadow Stack MSRs in g/h FPU
>> context switch.
> If that's necessary for correct functionality, yes.
>
>> 2) Add Supervisor Shadow stack support into KVM part so that guest OS is
>> able to use SSS with risk.
> Yes.  Architecturally, if KVM advertises X86_FEATURE_SHSTK, then KVM needs to
> provide both User and Supervisor support.  CET_SSS doesn't change the architecture,
> it's little more than a hint.  And even if the guest follows SDM's recommendation
> to not enable shadow stacks, a clever kernel can still utilize SSS assets, e.g. use
> the MSRs as scratch registers.

Understood, thanks!