Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <34b5dd08-edac-e32f-1884-c8f2b85f7971@gmail.com>
Date:   Wed, 19 Apr 2023 21:41:45 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
 Gecko/20100101 Thunderbird/102.10.0
Subject: Re: [PATCH 5/5] KVM: x86/pmu: Hide guest counter updates from the
 VMRUN instruction
Content-Language: en-US
To:     Sean Christopherson <seanjc@google.com>
Cc:     Paolo Bonzini <pbonzini@redhat.com>,
        Ravi Bangoria <ravi.bangoria@amd.com>, kvm@vger.kernel.org,
        linux-kernel@vger.kernel.org,
        "Bangoria, Ravikumar" <ravi.bangoria@amd.com>,
        Santosh Shukla <santosh.shukla@amd.com>,
        "Tom Lendacky (AMD)" <thomas.lendacky@amd.com>,
        Ananth Narayan <ananth.narayan@amd.com>
References: <20230310105346.12302-1-likexu@tencent.com>
 <20230310105346.12302-6-likexu@tencent.com> <ZC99f+AO1tZguu1I@google.com>
 <509b697f-4e60-94e5-f785-95f7f0a14006@gmail.com>
 <ZDAvDhV/bpPyt3oX@google.com>
From:   Like Xu <like.xu.linux@gmail.com>
In-Reply-To: <ZDAvDhV/bpPyt3oX@google.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

On 7/4/2023 10:56 pm, Sean Christopherson wrote:
> On Fri, Apr 07, 2023, Like Xu wrote:
>> On 7/4/2023 10:18 am, Sean Christopherson wrote:
>>> Wait, really?  VMRUN is counted if and only if it enters to a CPL0 guest?  Can
>>> someone from AMD confirm this?  I was going to say we should just treat this as
>>> "normal" behavior, but counting CPL0 but not CPL>0 is definitely quirky.
>>
>> VMRUN is only counted on a CPL0-target (branch) instruction counter.
> 
> Yes or no question: if KVM does VMRUN and a PMC is programmed to count _all_ taken
> branches, will the PMC count VMRUN as a branch if guest CPL>0 according to the VMCB?

YES, my quick tests (based on run_in_user() from KUT on Zen4) show:

EVENTSEL_GUESTONLY + EVENTSEL_ALL + VMRUN_to_USR -> AMD_ZEN_BR_RETIRED + 1
EVENTSEL_GUESTONLY + EVENTSEL_ALL + VMRUN_to_OS -> AMD_ZEN_BR_RETIRED + 1

EVENTSEL_GUESTONLY + EVENTSEL_USR + VMRUN_to_USR -> AMD_ZEN_BR_RETIRED + 1
EVENTSEL_GUESTONLY + EVENTSEL_OS + VMRUN_to_OS -> AMD_ZEN_BR_RETIRED + 1

VENTSEL_GUESTONLY + EVENTSEL_OS + VMRUN_to_USR -> No change
VENTSEL_GUESTONLY + EVENTSEL_USR + VMRUN_to_OS -> No change

I'm actually not surprised and related test would be posted later.

> 
>> This issue makes a guest CPL0-target instruction counter inexplicably
>> increase, as if it would have been under-counted before the virtualization
>> instructions were counted.
> 
> Heh, it's very much explicable, it's just not desirable, and you and I would argue
> that it's also incorrect.

This is completely inaccurate from the end guest pmu user's perspective.

I have a toy that looks like virtio-pmu, through which guest users can get 
hypervisor performance data.
But the side effect of letting the guest see the VMRUN instruction by default is 
unacceptable, isn't it ?

> 
> AMD folks, are there plans to document this as an erratum?  I agree with Like that
> counting VMRUN as a taken branch in guest context is a CPU bug, even if the behavior
> is known/expected.

+CC: Santosh, Tom, Ananth