Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Tue, 26 Oct 2021 16:04:28 +0000
From:   Sean Christopherson <seanjc@google.com>
To:     zhenwei pi <pizhenwei@bytedance.com>
Cc:     Wanpeng Li <kernellwp@gmail.com>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Jonathan Corbet <corbet@lwn.net>,
        Wanpeng Li <wanpengli@tencent.com>,
        LKML <linux-kernel@vger.kernel.org>, linux-doc@vger.kernel.org
Subject: Re: [PATCH] x86/kvm: Introduce boot parameter no-kvm-pvipi
Message-ID: <YXgnDBOXh2v3gzU4@google.com>
References: <20211020120726.4022086-1-pizhenwei@bytedance.com>
 <CANRm+CxAVA-L0wjm72eohXXWvh9fS7wVFzfKHuEjrsiRFuk9fg@mail.gmail.com>
 <YXB4FHfzh99707EH@google.com>
 <08757159-1673-5c7b-3efc-e5b54e82d6c3@bytedance.com>
 <CANRm+CzcTUWYJeaj3eWKH84YZYgeMZz3kbpn13c8i97iYGGHFQ@mail.gmail.com>
 <5b718b32-cd92-920e-c474-27b9cafeec60@bytedance.com>
 <ec014e8d-eb5f-03cc-3ed1-da58039ef034@bytedance.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ec014e8d-eb5f-03cc-3ed1-da58039ef034@bytedance.com>
Precedence: bulk

On Mon, Oct 25, 2021, zhenwei pi wrote:
> Hi, Wanpeng & Sean
> 
> Also benchmark redis(by 127.0.0.1) in a guest(2vCPU), 'no-kvm-pvipi' gets
> better performance.
> 
> Test env:
> Host side: pin 2vCPU on 2core in a die.
> Guest side: run command:
>   taskset -c 1 ./redis-server --appendonly no
>   taskset -c 0 ./redis-benchmark -h 127.0.0.1 -d 1024 -n 10000000 -t get
> 
> 1> without no-kvm-pvipi:
> redis QPS: 193203.12 requests per second
> kvm_pv_send_ipi exit: ~18K/s
> 
> 2> with no-kvm-pvipi:
> redis QPS: 196028.47 requests per second
> avic_incomplete_ipi_interception exit: ~5K/s

Numbers look sane, but I don't think that adding a guest-side kernel param is
the correct "fix".  As evidenced by Wanpeng's tests, PV IPI can outperform AVIC
in overcommit scenarios, and there's also no guarantee that AVIC/APICv is even
supported/enabled.  In other words, blindly disabling PV IPIs from within the
guest makes sense if and only if the guest knows that AVIC is enabled and that
its vCPUs are pinned.  If the guest has that info, then the host also has that
info, in which case the correct way to handle this is to simply not advertise
KVM_FEATURE_PV_SEND_IPI to the guest in CPUID.