Message-ID: <d80f6161-e327-f374-4caf-016de1a77cc3@gmail.com>
Date:   Tue, 28 Dec 2021 15:10:52 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.4.1
Content-Language: en-US
To:     Jim Mattson <jmattson@google.com>
Cc:     Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <seanjc@google.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Joerg Roedel <joro@8bytes.org>,
        Thomas Gleixner <tglx@linutronix.de>, x86@kernel.org,
        kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
        Like Xu <likexu@tencent.com>,
        Dongli Cao <caodongli@kingsoft.com>,
        Li RongQing <lirongqing@baidu.com>
References: <20211222133428.59977-1-likexu@tencent.com>
 <CALMp9eRNsKMB6onhc-nhW-aMb14gK1PCtQ_CxOoyZ5RvLHfvEQ@mail.gmail.com>
From:   Like Xu <like.xu.linux@gmail.com>
Organization: Tencent
Subject: Re: [PATCH v2] KVM: X86: Emulate APERF/MPERF to report actual vCPU
 frequency
In-Reply-To: <CALMp9eRNsKMB6onhc-nhW-aMb14gK1PCtQ_CxOoyZ5RvLHfvEQ@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hi Jim,

On 28/12/2021 2:33 am, Jim Mattson wrote:
> On Wed, Dec 22, 2021 at 5:34 AM Like Xu <like.xu.linux@gmail.com> wrote:
>>
>> From: Like Xu <likexu@tencent.com>
>>
>> The aperf/mperf are used to report current CPU frequency after 7d5905dc14a.
>> But guest kernel always reports a fixed vCPU frequency in the /proc/cpuinfo,
>> which may confuse users especially when turbo is enabled on the host or
>> when the vCPU has a noisy high power consumption neighbour task.
>>
>> Most guests such as Linux will only read accesses to AMPERF msrs, where
>> we can passthrough registers to the vcpu as the fast-path (a performance win)
>> and once any write accesses are trapped, the emulation will be switched to
>> slow-path, which emulates guest APERF/MPERF values based on host values.
>> In emulation mode, the returned MPERF msr value will be scaled according
>> to the TSCRatio value.
>>
>> As a minimum effort, KVM exposes the AMPERF feature when the host TSC
>> has CONSTANT and NONSTOP features, to avoid the need for more code
>> to cover various coner cases coming from host power throttling transitions.
>>
>> The slow path code reveals an opportunity to refactor update_vcpu_amperf()
>> and get_host_amperf() to be more flexible and generic, to cover more
>> power-related msrs.
>>
>> Requested-by: Dongli Cao <caodongli@kingsoft.com>
>> Requested-by: Li RongQing <lirongqing@baidu.com>
>> Signed-off-by: Like Xu <likexu@tencent.com>
> 
> I am not sure that it is necessary for kvm to get involved in the
> virtualization of APERF and MPERF at all, and I am highly skeptical of
> the need for passing through the hardware MSRs to a guest. Due to

The AMPERF is pass-through for read-only guest use cases.

> concerns over potential side-channel exploits a la Platypus

I agree that the enabling of AMPERF features increases the attack surface,
like any other upstreamed features (SGX), and they're not design flaw, are they?

As we know, KVM doesn't expose sufficient RAPL interface for Platypus. At least
the vendors has patched Platypus while the cat and mouse game will not end.

User space needs to choose whether to enable features based on the
guest's level of trust, rather than trying to prevent it from enablement.

> (https://platypusattack.com/), we are planning to provide only low
> fidelity APERF/MPERF virtualization from userspace, using the
> userspace MSR exiting mechanism. Of course, we should be able to do

It works for other non time-sensitive MSRs.

We have a long delay to walk the userspace MSR exiting mechanism
for both APERF msr and MPERF msr, which is almost intolerable for
frequent access guest reads. IMO, the low fidelity is not what the guest
user wants and it defeats the motivation for introducing amperf on host.

> that whether or not this change goes in, but I was wondering if you
> could provide some more details regarding your use case(s).

In addition to the advantages amperf brings in the kernel context
(e.g. smarter scheduler policies based on different power conditions),

Guest workload analysts are often curious about anomalous benchmark
scores under predictive CPU isolation guaranteed by service providers,
and they ask to look at actual vCPU frequencies to determine if the source
of performance noise is coming from neighboring hardware threads
particularly AVX or future AMX or other high power consumption neighbors.

This AMPERF data helps the customers to decide whether the back-end pCPU
is to be multiplexed or exclusive shared, or to upgrade to a faster HW model,
without being tricked by the guest CPUID.

IMO, this feature will be of value to most performance users. Any other comments?

Thanks,
Like Xu