Received-SPF: pass (google.com: domain of linux-kernel+bounces-185796-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Message-ID: <9dd51971-9df1-48ce-8e64-b2600a7b0467@amd.com>
Date: Wed, 22 May 2024 11:42:02 +0530
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2] KVM: SEV-ES: Don't intercept MSR_IA32_DEBUGCTLMSR for
 SEV-ES guests
To: Sean Christopherson <seanjc@google.com>,
 Paolo Bonzini <pbonzini@redhat.com>
Cc: thomas.lendacky@amd.com, tglx@linutronix.de, mingo@redhat.com,
 bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com,
 michael.roth@amd.com, nikunj.dadhania@amd.com, kvm@vger.kernel.org,
 linux-kernel@vger.kernel.org, santosh.shukla@amd.com, ravi.bangoria@amd.com
References: <20240416050338.517-1-ravi.bangoria@amd.com>
 <ZjQnFO9Pf4OLZdLU@google.com> <9252b68e-2b6a-6173-2e13-20154903097d@amd.com>
 <Zjp8AIorXJ-TEZP0@google.com> <305b84aa-3897-40f4-873b-dc512a2da61f@amd.com>
 <ZkdqW8JGCrUUO3RA@google.com> <b66ea07a-f57e-014c-68b4-729f893c2fbd@amd.com>
 <Zk0ErRQt3XH7xK6O@google.com>
 <CABgObfaXAERePMQrrpWg8PqM1TOq8TJT65i3WgU0n0-vePDGNg@mail.gmail.com>
 <Zk0elnvnF0n_exKt@google.com>
Content-Language: en-US
From: Ravi Bangoria <ravi.bangoria@amd.com>
In-Reply-To: <Zk0elnvnF0n_exKt@google.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?UHdTYTVvcUVSTDE4RUlidW0rZ2lWd0VrdXJmb1BDZkw3OHhkNGg3dmhIQzlk?=
 =?utf-8?B?cW1vazcybXdSS2FsWGZxZ25YZzZYcWVjSUxQWGd0L2NGMEs4blNKM0tWN28v?=
 =?utf-8?B?NzRYUnBkeVNwcTBvU21JVGxFd0o0NEE0SzJuWm1lMG0vOU1hSEtGOGs0cURY?=
 =?utf-8?B?VVNEMjQ3bHlYTFhVUS9PRmJGdjdsSm9JK0V5MkFXQmY3dVFvSzE3dUp3SS9K?=
 =?utf-8?B?ZVpKdVBtVHpwa3MxTTNaZTl2eHNuNWZuallQMm5DNjlmWCtiTHU4eXo4aVFk?=
 =?utf-8?B?Y0M5aVZVRm1ITDhvZmpYMGdnNTFGTEVZQXdhbTZPZXgvYk5QK0h6cWlTMlZK?=
 =?utf-8?B?MXVyRW1aNUZqNUU1Uk9Kd0tmaUtEV25DL3h6RmJsaXR3RVRRYlRxWERwLzhu?=
 =?utf-8?B?MU1mMEFaREx6SHBmQS9VcVB2dE1Xb3RkejNSeVpTL2c4TzBxYThxQjlOaXRK?=
 =?utf-8?B?MnNCZVRtVWo2b1dGQUZZYXd6ZUhwSjZBWjdmQkMxNCtYS2N1a2hvWmgrdkQ4?=
 =?utf-8?B?V3JzRnY5MUJrd0ZUNDRRRXdSS1NYRHpzUlNEZ0ZCWHFHSUVYN0dsVzhha0c5?=
 =?utf-8?B?UXA0K3dUQWVOTk1kWWx3YzR6VWQydTBEcTBSUkxJOGwrT2llQ0plclg5VUM4?=
 =?utf-8?B?NTlCZUxNR1lxZVNtTEszR3I4Qkt3UE1CUDc0SU5PdGhQTG9iMEE2S0piaU9B?=
 =?utf-8?B?ZUkwOFlaTDgzS0llZTZtcGRTdG12cEQyekZiSU9vZjZLeGNuNXR4bXZ4dnlh?=
 =?utf-8?B?dTFIUFZPU1YzK3I2WlVYekJ1L1ZCbnZTWmhVem1XalpXc2VqYzN5clZKcyt4?=
 =?utf-8?B?ditpQkNyNlMzQjR1V0pNTUljKzQyUWsyMllxOThqbTFpQjB0WkJ3SGgwQS9R?=
 =?utf-8?B?YUtXR1RBQjBqS0IyL2d2YWluK25aSDJuaDczMFNzTDc3M0JKdU5sUXBDUW9a?=
 =?utf-8?B?TEh4clpkSHNjaTlEOEFwc29wYTZHb1Buam9STEQramlQcGJ1VGx5Wm1MT2tK?=
 =?utf-8?B?MFB5YStydGZzYmZQZlZzeEd1VUROOHNBYzdreGI3WEppWGFmZ0tKdWRQdTF3?=
 =?utf-8?B?M3E3QzFRNkRKZ2V1YnZkaWhVQ2w4MlQ4V2Y4d1VCVUlkcGZXY1crekZYZ3Jh?=
 =?utf-8?B?L1VmWFFVVllFUHdRc0FyZkJQY0pZMGpiWURMWWZjN0E5Nlh4b0k5SHA2QUtw?=
 =?utf-8?B?TGhwT21sbWVzVmx4eFFpaUlyaUtMUWkwZXg4N3Rjcm8rdzlzdFEvMzdqd1B3?=
 =?utf-8?B?cDFvcHFzVXZUM3FOR2dDT2wvOWtKNUUvNm9TTVhMV3Bpc3U1MnlVNjBIMGlv?=
 =?utf-8?B?T0FMNDQ2ZlRSK0w1WWVRVnJSRkJ4Vm1NL3FYaFo4b1FuK0NBM2Vic3Z3aFdq?=
 =?utf-8?B?RlpMUmtIUnF6Qkh4Y0R0eTJLS0lZVFlBT2t3STlWR29YMENhOFZWenRBU28r?=
 =?utf-8?B?QVNTOWNPaDZtN000dndoekM1NzBTRmp4ZlNiZEp4VHhFZ2lveWdPZ1RGWFdL?=
 =?utf-8?B?VDF3Sk52QVBvYThham0xSmsyYWlvYlJ0WEYrbzI5WDlHblVoSDhoVVBRWjJS?=
 =?utf-8?B?cDFPdlBJZGtuMFc5TjJuUlVEMDlzbnl1bUJONms2WGViMWsvUDJtNmJWK1FM?=
 =?utf-8?B?b05wNFhESnF5UDk2c0xteTdlc3gydnFZb0JHY1pCdmR3eVNIZ2dCcVc0YlVv?=
 =?utf-8?B?OElsb1pXY3BpU1BZYUlDeWdLTVVDQU9IL0crYTcvUkhacjBPc3FiN0RVS1Q0?=
 =?utf-8?B?eXJSZjNNdGc1RmNSRkRDa0RYMFMzc1F2K0xJSTZwSkRTN3pGQ2NQeE9DMFlX?=
 =?utf-8?B?b1FVNUc5UXUrbEFBWlJCSTBZalgwNHgwYytKMVcwRE9XTEpFMGdxS0JYaC9S?=
 =?utf-8?B?QU4vRzM3USs3clhITytQRmx5N1Z0cEYyc1VhYUh2SmVRRWxndWdEalAxTTNB?=
 =?utf-8?B?TXNESUdyOFdZWU5ab1dGdU5iZDhPNkRmK21iNVFSZzVWcWNkQ1FVeVpJSjhN?=
 =?utf-8?B?YnNnMmZBcXdxdHl0aDVJbzJ5WmQxeUNZRG9PVGh5OFpvQVVsNWF4blU1dG8y?=
 =?utf-8?B?WDFUNVFrL1VIeXhGeFhVTFJQeVZZcXdPamJ2T1o3dk9CYTQ0V1U4aW5vR2lx?=
 =?utf-8?Q?EKvSqPetbAyqFGYE45YkUJQfN?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1847cdd7-b2de-4fee-ce37-08dc7a261eef
X-MS-Exchange-CrossTenant-AuthSource: PH7PR12MB6588.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 May 2024 06:12:12.9273
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: m7GkNLZZS1jkxk+5+s+pP0BczYIasTj3zqBGcwzXrN2sxEPDUrXJ1hhxBTEVKFGZo1H819t1c+zziS32Aw+gBA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR12MB6177

On 5/22/2024 3:52 AM, Sean Christopherson wrote:
> On Tue, May 21, 2024, Paolo Bonzini wrote:
>> On Tue, May 21, 2024 at 10:31 PM Sean Christopherson <seanjc@google.com> wrote:
>>>
>>> On Mon, May 20, 2024, Ravi Bangoria wrote:
>>>> On 17-May-24 8:01 PM, Sean Christopherson wrote:
>>>>> On Fri, May 17, 2024, Ravi Bangoria wrote:
>>>>>> On 08-May-24 12:37 AM, Sean Christopherson wrote:
>>>>>>> So unless I'm missing something, the only reason to ever disable LBRV would be
>>>>>>> for performance reasons.  Indeed the original commits more or less says as much:
>>>>>>>
>>>>>>>   commit 24e09cbf480a72f9c952af4ca77b159503dca44b
>>>>>>>   Author:     Joerg Roedel <joerg.roedel@amd.com>
>>>>>>>   AuthorDate: Wed Feb 13 18:58:47 2008 +0100
>>>>>>>
>>>>>>>     KVM: SVM: enable LBR virtualization
>>>>>>>
>>>>>>>     This patch implements the Last Branch Record Virtualization (LBRV) feature of
>>>>>>>     the AMD Barcelona and Phenom processors into the kvm-amd module. It will only
>>>>>>>     be enabled if the guest enables last branch recording in the DEBUG_CTL MSR. So
>>>>>>>     there is no increased world switch overhead when the guest doesn't use these
>>>>>>>     MSRs.
>>>>>>>
>>>>>>> but what it _doesn't_ say is what the world switch overhead is when LBRV is
>>>>>>> enabled.  If the overhead is small, e.g. 20 cycles?, then I see no reason to
>>>>>>> keep the dynamically toggling.
>>>>>>>
>>>>>>> And if we ditch the dynamic toggling, then this patch is unnecessary to fix the
>>>>>>> LBRV issue.  It _is_ necessary to actually let the guest use the LBRs, but that's
>>>>>>> a wildly different changelog and justification.
>>>>>>
>>>>>> The overhead might be less for legacy LBR. But upcoming hw also supports
>>>>>> LBR Stack Virtualization[1]. LBR Stack has total 34 MSRs (two control and
>>>>>> 16*2 stack). Also, Legacy and Stack LBR virtualization both are controlled
>>>>>> through the same VMCB bit. So I think I still need to keep the dynamic
>>>>>> toggling for LBR Stack virtualization.
>>>>>
>>>>> Please get performance number so that we can make an informed decision.  I don't
>>>>> want to carry complexity because we _think_ the overhead would be too high.
>>>>
>>>> LBR Virtualization overhead for guest entry + exit roundtrip is ~450 cycles* on
>>>
>>> Ouch.  Just to clearify, that's for LBR Stack Virtualization, correct?
>>
>> And they are all in the VMSA, triggered by LBR_CTL_ENABLE_MASK, for
>> non SEV-ES guests?
>>
>>> Anyways, I agree that we need to keep the dynamic toggling.
>>> But I still think we should delete the "lbrv" module param.  LBR Stack support has
>>> a CPUID feature flag, i.e. userspace can disable LBR support via CPUID in order
>>> to avoid the overhead on CPUs with LBR Stack.
>>
>> The "lbrv" module parameter is only there to test the logic for
>> processors (including nested virt) that don't have LBR virtualization.
>> But the only effect it has is to drop writes to
>> MSR_IA32_DEBUGCTL_MSR...
>>
>>>                 if (kvm_cpu_cap_has(X86_FEATURE_LBR_STACK) &&
>>>                     !guest_cpuid_has(vcpu, X86_FEATURE_LBR_STACK)) {
>>>                         kvm_pr_unimpl_wrmsr(vcpu, ecx, data);
>>>                         break;
>>>                 }
>>
>> ... and if you have this, adding an "!lbrv ||" is not a big deal, and
>> allows testing the code on machines without LBR stack.
> 
> Yeah, but keeping lbrv also requires tying KVM's X86_FEATURE_LBR_STACK capability
> to lbrv, i.e. KVM shouldn't advetise X86_FEATURE_LBR_STACK if lbrv=false.  And
> KVM needs to condition SEV-ES on lbrv=true.  Neither of those are difficult to
> handle, e.g. svm_set_cpu_caps() already checks plenty of module params, I'm just
> not convinced legacy LRB virtualization is interesting enough to warrant a module
> param.
> 
> That said, I'm ok keeping the param if folks prefer that approach.

Sure, will keep it. I'll respin with all these feedback addressed.

Thanks,
Ravi