Received-SPF: pass (google.com: domain of linux-kernel+bounces-185794-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Message-ID: <9316ce80-b9c5-44ce-ab66-e7cdb6893c1f@amd.com>
Date: Wed, 22 May 2024 11:41:10 +0530
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] KVM: SEV-ES: Don't intercept MSR_IA32_DEBUGCTLMSR for
 SEV-ES guests
To: Sean Christopherson <seanjc@google.com>
Cc: pbonzini@redhat.com, thomas.lendacky@amd.com, tglx@linutronix.de,
 mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org,
 hpa@zytor.com, michael.roth@amd.com, nikunj.dadhania@amd.com,
 kvm@vger.kernel.org, linux-kernel@vger.kernel.org, santosh.shukla@amd.com,
 ravi.bangoria@amd.com
References: <20240416050338.517-1-ravi.bangoria@amd.com>
 <ZjQnFO9Pf4OLZdLU@google.com> <9252b68e-2b6a-6173-2e13-20154903097d@amd.com>
 <Zjp8AIorXJ-TEZP0@google.com> <305b84aa-3897-40f4-873b-dc512a2da61f@amd.com>
 <ZkdqW8JGCrUUO3RA@google.com> <b66ea07a-f57e-014c-68b4-729f893c2fbd@amd.com>
 <Zk0ErRQt3XH7xK6O@google.com>
Content-Language: en-US
From: Ravi Bangoria <ravi.bangoria@amd.com>
In-Reply-To: <Zk0ErRQt3XH7xK6O@google.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?OTliR3dpQno0bVhlMzlnT25GU24yTUI1TjJTUTF3ankwUEZ5WiszcnBSb2Ja?=
 =?utf-8?B?UU1Uc29HMUZBKzhUMzRwbnY5QmtKcE1Qa2d4b2Y1WW9KTm1jNENFcWlOVjk3?=
 =?utf-8?B?NmdlZXltRWtMREJhUkVEcnpRVEZzSmFLM0MwMU50bHhMbGR5SVdyVk4yWEEr?=
 =?utf-8?B?ZHlETmlXbVZIMExndWRydkNWeUc4ZXVaVWFDaFdMT09GR1FLTXc4UzFvQllU?=
 =?utf-8?B?U3ZycER4UDdYMVgwem9id0xNalpSTmZXaGtXS2dSS0I1UVdyQnVUems2RGVM?=
 =?utf-8?B?KzJuVmZYd3FSYXNUdU1CK0ZwaDE3cnhNV09lR2gyWGxhSVp5RE0vWllSbHEz?=
 =?utf-8?B?dWNXWmNSOW5sZCtvQzNMZlpnVi9WRWg0NzNVd1hBTWFqdkhvbjhhcHV2T0lX?=
 =?utf-8?B?RzRYa0VHdDNaUEZKQWxjcE9zOEs3dFlTdG04ZWJ4aXRGU05XTjJ0bkN2blV4?=
 =?utf-8?B?dFg1NkRta1ZEK2o1N0J4SFd3cjFBOTV4S2cyNFhtYnA2a2dUa2hFRmxjY2hD?=
 =?utf-8?B?TlhYR1VMMUZsQXpiMDZyMlQ3OGxYNzJQUXloT2Ftd0l2eXJ5R0F5Y1NlSzRB?=
 =?utf-8?B?a1JrVmxpbmNEcHNpQkk2K1h6Qk9SbE4yNzhkS29iNW9OV1N4ek85M1Z4Qk05?=
 =?utf-8?B?bi8zUlp4QXUvTVZWcjg2MUN0a1NQMWg4eTY0V0xxb0l1b251TkFSTkFhS2VJ?=
 =?utf-8?B?bDBHcy9JekVid25zUk5WNUY3M1NkZE53NlZUeG5UU3ZoQzk1RHU4SlFMWU1y?=
 =?utf-8?B?L2g3V0hNZUhJL3VuU3ZxM2ZmU1F4SkhSKzlyMmZCUnRpS09JZlJvcVE0QVBO?=
 =?utf-8?B?VmdDNHUxMGNtRzN5bXRDcDNzeDIyaEcwdExwM3BUTks5MzJjdlN5T2NDZkxt?=
 =?utf-8?B?OTgzU0lQcEtXcmdOT1J6RGV4ZlRFaTNuL25yRlNLWnVFRjdaamlsQlhPNVpm?=
 =?utf-8?B?WjhZMXQzRlQzZEp1Y1I1YzBzTWd6dDQ0MEIzUE5Wckk5VEZDeDdvTXMwWDdj?=
 =?utf-8?B?aU9Qa3QzM0J2UFJJOEZpSlgyclA2VVdiZnlEdWZnWis5SlZkRms4UFdUczRW?=
 =?utf-8?B?S05TcDNSVm1oZitGOEo2UHRsZURVR2ROYjFIUjJudVkzVzB6bnpBT3dOclIw?=
 =?utf-8?B?MnA5Vk5zMC9ybm0yVC9pT2RvZjJBTklaZUJ1cVJxeWRVUEZ2cUZOd0VYVVMw?=
 =?utf-8?B?djA1OFMyb3RWZERSOHdjWnJJY0dEdzZrVVFuUEJ6dStzbmtlM0dPLy80b1JM?=
 =?utf-8?B?Um94RTB2QkFPazh0NDNZWlNHLytrM0ZWUGtEcFZYQlNibTEvejBzK1NlcGIx?=
 =?utf-8?B?M3hLRWF2TUZFVlkyVmNIcW0vMkEramszYWNrNnRFVkdoUmI1NUxBaGpMeHp0?=
 =?utf-8?B?SUNPb3d4N1VobXhUVnpHTUZ3ZjRldm1CcUZWUko1VXNjSEtJQ2VIcEJDQk5m?=
 =?utf-8?B?OHdKdG00TzFmbzRJUDRjamVrcGRjYzMvQ0pDRktLekFIczNWR2xURE1CM1E2?=
 =?utf-8?B?U09DaDhpb2owZHlTRy9aSThwRDJqdGc2dERLUWg4dzl1OEhvc1BGNkpDaUQy?=
 =?utf-8?B?anB5ZU1iN1ZuNTNaRk9RemcrWWUwZnZsNEFDc3BFNlBTTnFGTDV5b0tWVTlU?=
 =?utf-8?B?TEMxa1BuUStxZS9KbnIvUmxZY0xabzVrWG41TzRiYWhPYnZlVGJiRmVjM1FM?=
 =?utf-8?B?eEIxQXZsZklsMnZxUGc1bnR1QjNvZmduRmtiU1lraENHTzNJOGlxTjMrazEy?=
 =?utf-8?B?OTh2eTZ6anpEN1NxeXhDdmNnRm1HbTdoTUJyckw3djJJK3YwVEVoMStjd051?=
 =?utf-8?B?RVI0RjY1bWhCTDdIdERndEpLYTAyME1ZN1NMQjgzWkkvMEtISnRneFRrbitE?=
 =?utf-8?B?T0VEUU1nOUlSVURiWkJiWUVaZURhcTkvTEhXc2R4NnpnMUt4bTdKVHdtSmN2?=
 =?utf-8?B?cmJ0Nlp0eml0UUFMdzVyYkx1aUpxcVlOeHhsL3E4YUpaRG1MdE1LTWVqeWc3?=
 =?utf-8?B?N3hoS0V4bkFQcWsvbmh1dGRId0M3R0JuY1EvaTltSUVQby9XanlGNE9ycmxP?=
 =?utf-8?B?V3JJNHZiTHovREhzZnFIdm5KTmM3bEUzeHdGY0U5VmRYSmtlMEI5MDRqbzRI?=
 =?utf-8?Q?mwiGNd29Jpt98FHRvabdEppJx?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 929df9f2-9257-465b-ee01-08dc7a260030
X-MS-Exchange-CrossTenant-AuthSource: PH7PR12MB6588.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 06:11:21.3736
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: g56GXAOHD7FdKizcto2ongp9JN/3W5Rv0GuO6M1id3Lav2PUQTIX0F2B3ZdYeLPUyJUmyMzI/qQk8nBCm9HcSQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6177

On 5/22/2024 2:01 AM, Sean Christopherson wrote:
> On Mon, May 20, 2024, Ravi Bangoria wrote:
>> On 17-May-24 8:01 PM, Sean Christopherson wrote:
>>> On Fri, May 17, 2024, Ravi Bangoria wrote:
>>>> On 08-May-24 12:37 AM, Sean Christopherson wrote:
>>>>> So unless I'm missing something, the only reason to ever disable LBRV would be
>>>>> for performance reasons.  Indeed the original commits more or less says as much:
>>>>>
>>>>>   commit 24e09cbf480a72f9c952af4ca77b159503dca44b
>>>>>   Author:     Joerg Roedel <joerg.roedel@amd.com>
>>>>>   AuthorDate: Wed Feb 13 18:58:47 2008 +0100
>>>>>
>>>>>     KVM: SVM: enable LBR virtualization
>>>>>     
>>>>>     This patch implements the Last Branch Record Virtualization (LBRV) feature of
>>>>>     the AMD Barcelona and Phenom processors into the kvm-amd module. It will only
>>>>>     be enabled if the guest enables last branch recording in the DEBUG_CTL MSR. So
>>>>>     there is no increased world switch overhead when the guest doesn't use these
>>>>>     MSRs.
>>>>>
>>>>> but what it _doesn't_ say is what the world switch overhead is when LBRV is
>>>>> enabled.  If the overhead is small, e.g. 20 cycles?, then I see no reason to
>>>>> keep the dynamically toggling.
>>>>>
>>>>> And if we ditch the dynamic toggling, then this patch is unnecessary to fix the
>>>>> LBRV issue.  It _is_ necessary to actually let the guest use the LBRs, but that's
>>>>> a wildly different changelog and justification.
>>>>
>>>> The overhead might be less for legacy LBR. But upcoming hw also supports
>>>> LBR Stack Virtualization[1]. LBR Stack has total 34 MSRs (two control and
>>>> 16*2 stack). Also, Legacy and Stack LBR virtualization both are controlled
>>>> through the same VMCB bit. So I think I still need to keep the dynamic
>>>> toggling for LBR Stack virtualization.
>>>
>>> Please get performance number so that we can make an informed decision.  I don't
>>> want to carry complexity because we _think_ the overhead would be too high.
>>
>> LBR Virtualization overhead for guest entry + exit roundtrip is ~450 cycles* on
> 
> Ouch.  Just to clearify, that's for LBR Stack Virtualization, correct?

Includes both, since there is a single enable bit shared by them.

> Ugh, I was going to say that we could always enable "legacy" LBR virtualization,
> and do the dynamic toggling iff DebugExtnCtl.LBRS=1, but they share an enabling
> flag.  What a mess.

Agreed. can't help :(

>> a Genoa machine. Also, LBR MSRs (except MSR_AMD_DBG_EXTN_CFG) are of swap type
>> C so this overhead is only for guest MSR save/restore.
> 
> Lovely.
> 
> Have I mentioned that the SEV-ES behavior of force-enabling every feature under
> the sun is really, really annoying?
> 
> Anyways, I agree that we need to keep the dynamic toggling.

Sure. Will prepare v3 accordingly.

Thanks,
Ravi