Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <1559799086-13912-1-git-send-email-wanpengli@tencent.com>
In-Reply-To: <1559799086-13912-1-git-send-email-wanpengli@tencent.com>
From:   Wanpeng Li <kernellwp@gmail.com>
Date:   Mon, 10 Jun 2019 12:32:33 +0800
Message-ID: <CANRm+CwUyX_7FafiWPCr4JgFetfOkttw2P0xS6yjOzKs-X3f9A@mail.gmail.com>
Subject: Re: [PATCH v2 0/3] KVM: LAPIC: Implement Exitless Timer
To:     LKML <linux-kernel@vger.kernel.org>, kvm <kvm@vger.kernel.org>
Cc:     Paolo Bonzini <pbonzini@redhat.com>,
        =?UTF-8?B?UmFkaW0gS3LEjW3DocWZ?= <rkrcmar@redhat.com>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Thu, 6 Jun 2019 at 13:31, Wanpeng Li <kernellwp@gmail.com> wrote:
>
> Dedicated instances are currently disturbed by unnecessary jitter due
> to the emulated lapic timers fire on the same pCPUs which vCPUs resident.
> There is no hardware virtual timer on Intel for guest like ARM. Both
> programming timer in guest and the emulated timer fires incur vmexits.
> This patchset tries to avoid vmexit which is incurred by the emulated
> timer fires in dedicated instance scenario.
>
> When nohz_full is enabled in dedicated instances scenario, the unpinned
> timer will be moved to the nearest busy housekeepers after commit 444969223c8
> ("sched/nohz: Fix affine unpinned timers mess"). However, KVM always makes
> lapic timer pinned to the pCPU which vCPU residents, the reason is explained
> by commit 61abdbe0 (kvm: x86: make lapic hrtimer pinned). Actually, these
> emulated timers can be offload to the housekeeping cpus since APICv
> is really common in recent years. The guest timer interrupt is injected by
> posted-interrupt which is delivered by housekeeping cpu once the emulated
> timer fires.
>
> The host admin should fine tuned, e.g. dedicated instances scenario w/
> nohz_full cover the pCPUs which vCPUs resident, several pCPUs surplus
> for housekeeping, disable mwait/hlt/pause vmexits to occupy the pCPUs,
> fortunately preemption timer is disabled after mwait is exposed to
> guest which makes emulated timer offload can be possible.
> 3%~5% redis performance benefit can be observed on Skylake server.

w/o patchset:

                             VM-EXIT    Samples  Samples%     Time%
Min Time    Max Time         Avg time

  EXTERNAL_INTERRUPT      42916    49.43%    39.30%      0.47us
106.09us      0.71us ( +-   1.09% )

w/ patchset:

                             VM-EXIT    Samples  Samples%     Time%
Min Time    Max Time         Avg time

  EXTERNAL_INTERRUPT       6871     9.29%     2.96%      0.44us
57.88us      0.72us ( +-   4.02% )

Regards,
Wanpeng Li