Received-SPF: pass (google.com: domain of linux-kernel+bounces-59203-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Precedence: bulk
MIME-Version: 1.0
References: <20240124113446.2977003-1-foxywang@tencent.com> <02fe988e-2b42-9610-6ab5-bd17b0d9fb80@oracle.com>
In-Reply-To: <02fe988e-2b42-9610-6ab5-bd17b0d9fb80@oracle.com>
From: Yi Wang <up2wing@gmail.com>
Date: Fri, 9 Feb 2024 18:17:56 +0800
Message-ID: <CAN35MuSanFT1JxM16usksSDjrLLsAAWs-kosJEd20sKckvwJfg@mail.gmail.com>
Subject: Re: [v3 0/3] KVM: irqchip: synchronize srcu only if needed
To: Dongli Zhang <dongli.zhang@oracle.com>
Cc: x86@kernel.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, 
	foxywang@tencent.com, seanjc@google.com, pbonzini@redhat.com, 
	mingo@redhat.com, tglx@linutronix.de, dave.hansen@linux.intel.com, 
	hpa@zytor.com, bp@alien8.de, wanpengli@tencent.com, oliver.upton@linux.dev, 
	anup@brainfault.org, frankja@linux.ibm.com, imbrenda@linux.ibm.com, 
	maz@kernel.org, atishp@atishpatra.org, borntraeger@linux.ibm.com
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi Dongli,

Thanks for the reply and Happy Spring Festival to all :)

On Fri, Feb 9, 2024 at 5:00=E2=80=AFPM Dongli Zhang <dongli.zhang@oracle.co=
m> wrote:
>
> Hi Yi,
>
> On 1/24/24 03:34, Yi Wang wrote:
> > From: Yi Wang <foxywang@tencent.com>
> >
> > We found that it may cost more than 20 milliseconds very accidentally
> > to enable cap of KVM_CAP_SPLIT_IRQCHIP on a host which has many vms
> > already.
>
> Would you mind explaining the reason that the *number of VMs* matters, as
> KVM_CAP_SPLIT_IRQCHIP is a per-VM cap?
>
> Or it meant it is more likely to have some VM workload impacted by the
> synchronize_srcu_expedited() as in prior discussion?
>
> https://lore.kernel.org/kvm/CAN35MuSkQf0XmBZ5ZXGhcpUCGD-kKoyTv9G7ya4QVD1x=
iqOxLg@mail.gmail.com/
>

The actual reason is might_sleep() and the kworker in
synchronize_srcu_expedited(),
which may cause some delay when there are pretty many threads in the host, =
so
"number of VMs" is just one of  the situations which can trigger the issue =
:)

> Thank you very much!
>
> Dongli Zhang
>
> >
> > The reason is that when vmm(qemu/CloudHypervisor) invokes
> > KVM_CAP_SPLIT_IRQCHIP kvm will call synchronize_srcu_expedited() and
> > might_sleep and kworker of srcu may cost some delay during this period.
> > One way makes sence is setup empty irq routing when creating vm and
> > so that x86/s390 don't need to setup empty/dummy irq routing.
> >
> > Note: I have no s390 machine so the s390 patch has not been tested.
> >
> > Changelog:
> > ----------
> > v3:
> >   - squash setup empty routing function and use of that into one commit
> >   - drop the comment in s390 part
> >
> > v2:
> >   - setup empty irq routing in kvm_create_vm
> >   - don't setup irq routing in x86 KVM_CAP_SPLIT_IRQCHIP
> >   - don't setup irq routing in s390 KVM_CREATE_IRQCHIP
> >
> > v1: https://urldefense.com/v3/__https://lore.kernel.org/kvm/20240112091=
128.3868059-1-foxywang@tencent.com/__;!!ACWV5N9M2RV99hQ!LjwKfBaGVl3u1l9YQSs=
kg_1RU6278h2-fYnYLsoihF9i43aq73eIDqolGzOmeRvO8UlPreQHLqXEL1bAuw$
> >
> > Yi Wang (3):
> >   KVM: setup empty irq routing when create vm
> >   KVM: x86: don't setup empty irq routing when KVM_CAP_SPLIT_IRQCHIP
> >   KVM: s390: don't setup dummy routing when KVM_CREATE_IRQCHIP
> >
> >  arch/s390/kvm/kvm-s390.c |  9 +--------
> >  arch/x86/kvm/irq.h       |  1 -
> >  arch/x86/kvm/irq_comm.c  |  5 -----
> >  arch/x86/kvm/x86.c       |  3 ---
> >  include/linux/kvm_host.h |  1 +
> >  virt/kvm/irqchip.c       | 19 +++++++++++++++++++
> >  virt/kvm/kvm_main.c      |  4 ++++
> >  7 files changed, 25 insertions(+), 17 deletions(-)
> >


--=20
---
Best wishes
Yi Wang