MIME-Version: 1.0
References: <20211222133428.59977-1-likexu@tencent.com> <CALMp9eRNsKMB6onhc-nhW-aMb14gK1PCtQ_CxOoyZ5RvLHfvEQ@mail.gmail.com>
 <d80f6161-e327-f374-4caf-016de1a77cc3@gmail.com>
In-Reply-To: <d80f6161-e327-f374-4caf-016de1a77cc3@gmail.com>
From:   Jim Mattson <jmattson@google.com>
Date:   Tue, 28 Dec 2021 16:05:17 -0800
Message-ID: <CALMp9eTy-8CgoV1TJBQ=RON=k8bQ8SYR7xj00qipUEetR4Xofg@mail.gmail.com>
Subject: Re: [PATCH v2] KVM: X86: Emulate APERF/MPERF to report actual vCPU frequency
To:     Like Xu <like.xu.linux@gmail.com>
Cc:     Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <seanjc@google.com>,
        Wanpeng Li <wanpengli@tencent.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>,
        Joerg Roedel <joro@8bytes.org>,
        Thomas Gleixner <tglx@linutronix.de>, x86@kernel.org,
        kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
        Like Xu <likexu@tencent.com>,
        Dongli Cao <caodongli@kingsoft.com>,
        Li RongQing <lirongqing@baidu.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Mon, Dec 27, 2021 at 11:11 PM Like Xu <like.xu.linux@gmail.com> wrote:
>
> Hi Jim,
>
> On 28/12/2021 2:33 am, Jim Mattson wrote:
> > On Wed, Dec 22, 2021 at 5:34 AM Like Xu <like.xu.linux@gmail.com> wrote:
> >>
> >> From: Like Xu <likexu@tencent.com>
> >>
> >> The aperf/mperf are used to report current CPU frequency after 7d5905dc14a.
> >> But guest kernel always reports a fixed vCPU frequency in the /proc/cpuinfo,
> >> which may confuse users especially when turbo is enabled on the host or
> >> when the vCPU has a noisy high power consumption neighbour task.
> >>
> >> Most guests such as Linux will only read accesses to AMPERF msrs, where
> >> we can passthrough registers to the vcpu as the fast-path (a performance win)
> >> and once any write accesses are trapped, the emulation will be switched to
> >> slow-path, which emulates guest APERF/MPERF values based on host values.
> >> In emulation mode, the returned MPERF msr value will be scaled according
> >> to the TSCRatio value.
> >>
> >> As a minimum effort, KVM exposes the AMPERF feature when the host TSC
> >> has CONSTANT and NONSTOP features, to avoid the need for more code
> >> to cover various coner cases coming from host power throttling transitions.
> >>
> >> The slow path code reveals an opportunity to refactor update_vcpu_amperf()
> >> and get_host_amperf() to be more flexible and generic, to cover more
> >> power-related msrs.
> >>
> >> Requested-by: Dongli Cao <caodongli@kingsoft.com>
> >> Requested-by: Li RongQing <lirongqing@baidu.com>
> >> Signed-off-by: Like Xu <likexu@tencent.com>
> >
> > I am not sure that it is necessary for kvm to get involved in the
> > virtualization of APERF and MPERF at all, and I am highly skeptical of
> > the need for passing through the hardware MSRs to a guest. Due to
>
> The AMPERF is pass-through for read-only guest use cases.
>
> > concerns over potential side-channel exploits a la Platypus
>
> I agree that the enabling of AMPERF features increases the attack surface,
> like any other upstreamed features (SGX), and they're not design flaw, are they?
>
> As we know, KVM doesn't expose sufficient RAPL interface for Platypus. At least
> the vendors has patched Platypus while the cat and mouse game will not end.
>
> User space needs to choose whether to enable features based on the
> guest's level of trust, rather than trying to prevent it from enablement.
>
> > (https://platypusattack.com/), we are planning to provide only low
> > fidelity APERF/MPERF virtualization from userspace, using the
> > userspace MSR exiting mechanism. Of course, we should be able to do
>
> It works for other non time-sensitive MSRs.
>
> We have a long delay to walk the userspace MSR exiting mechanism
> for both APERF msr and MPERF msr, which is almost intolerable for
> frequent access guest reads. IMO, the low fidelity is not what the guest
> user wants and it defeats the motivation for introducing amperf on host.
>
> > that whether or not this change goes in, but I was wondering if you
> > could provide some more details regarding your use case(s).
>
> In addition to the advantages amperf brings in the kernel context
> (e.g. smarter scheduler policies based on different power conditions),
>
> Guest workload analysts are often curious about anomalous benchmark
> scores under predictive CPU isolation guaranteed by service providers,
> and they ask to look at actual vCPU frequencies to determine if the source
> of performance noise is coming from neighboring hardware threads
> particularly AVX or future AMX or other high power consumption neighbors.
>
> This AMPERF data helps the customers to decide whether the back-end pCPU
> is to be multiplexed or exclusive shared, or to upgrade to a faster HW model,
> without being tricked by the guest CPUID.
>
> IMO, this feature will be of value to most performance users. Any other comments?

If it's worth doing, it's worth doing well.

Let me go back and look in detail at the code, now that the question
of whether or not this is worth doing has been answered.

> Thanks,
> Like Xu