Received-SPF: pass (google.com: domain of linux-kernel+bounces-27599-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Date: Tue, 16 Jan 2024 08:58:10 -0800
In-Reply-To: <72edaedc-50d7-415e-9c45-f17ffe0c1c23@linux.ibm.com>
Precedence: bulk
Mime-Version: 1.0
References: <20240112091128.3868059-1-foxywang@tencent.com>
 <ZaFor2Lvdm4O2NWa@google.com> <CAN35MuSkQf0XmBZ5ZXGhcpUCGD-kKoyTv9G7ya4QVD1xiqOxLg@mail.gmail.com>
 <72edaedc-50d7-415e-9c45-f17ffe0c1c23@linux.ibm.com>
Message-ID: <Zaa1omCaDQOxxy2j@google.com>
Subject: Re: [PATCH] KVM: irqchip: synchronize srcu only if needed
From: Sean Christopherson <seanjc@google.com>
To: Christian Borntraeger <borntraeger@linux.ibm.com>
Cc: Yi Wang <up2wing@gmail.com>, pbonzini@redhat.com, tglx@linutronix.de, 
	mingo@redhat.com, bp@alien8.de, dave.hansen@linux.intel.com, x86@kernel.org, 
	hpa@zytor.com, kvm@vger.kernel.org, linux-kernel@vger.kernel.org, 
	wanpengli@tencent.com, Yi Wang <foxywang@tencent.com>, 
	Oliver Upton <oliver.upton@linux.dev>, Marc Zyngier <maz@kernel.org>, 
	Anup Patel <anup@brainfault.org>, Atish Patra <atishp@atishpatra.org>, 
	Janosch Frank <frankja@linux.ibm.com>, Claudio Imbrenda <imbrenda@linux.ibm.com>
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable

On Tue, Jan 16, 2024, Christian Borntraeger wrote:
>=20
>=20
> Am 15.01.24 um 17:01 schrieb Yi Wang:
> > Many thanks for your such kind and detailed reply, Sean!
> >=20
> > On Sat, Jan 13, 2024 at 12:28=E2=80=AFAM Sean Christopherson <seanjc@go=
ogle.com> wrote:
> > >=20
> > > +other KVM maintainers
> > >=20
> > > On Fri, Jan 12, 2024, Yi Wang wrote:
> > > > From: Yi Wang <foxywang@tencent.com>
> > > >=20
> > > > We found that it may cost more than 20 milliseconds very accidental=
ly
> > > > to enable cap of KVM_CAP_SPLIT_IRQCHIP on a host which has many vms
> > > > already.
> > > >=20
> > > > The reason is that when vmm(qemu/CloudHypervisor) invokes
> > > > KVM_CAP_SPLIT_IRQCHIP kvm will call synchronize_srcu_expedited() an=
d
> > > > might_sleep and kworker of srcu may cost some delay during this per=
iod.
> > >=20
> > > might_sleep() yielding is not justification for changing KVM.  That's=
 more or
> > > less saying "my task got preempted and took longer to run".  Well, ye=
ah.
> >=20
> > Agree. But I suppose it may be one of the reasons that makes  time of
> > KVM_CAP_SPLIT_IRQCHIP delayed, of course, the kworker has the biggest
> > suspicion :)
> >=20
> > >=20
> > > > Since this happens during creating vm, it's no need to synchronize =
srcu
> > > > now 'cause everything is not ready(vcpu/irqfd) and none uses irq_sr=
cu now.
> >=20
> > ....
> >=20
> > > And on x86, I'm pretty sure as of commit 654f1f13ea56 ("kvm: Check ir=
qchip mode
> > > before assign irqfd"), which added kvm_arch_irqfd_allowed(), it's imp=
ossible for
> > > kvm_irq_map_gsi() to encounter a NULL irq_routing _on x86_.
> > >=20
> > > But I strongly suspect other architectures can reach kvm_irq_map_gsi(=
) with a
> > > NULL irq_routing, e.g. RISC-V dynamically configures its interrupt co=
ntroller,
> > > yet doesn't implement kvm_arch_intc_initialized().
> > >=20
> > > So instead of special casing x86, what if we instead have KVM setup a=
n empty
> > > IRQ routing table during kvm_create_vm(), and then avoid this mess en=
tirely?
> > > That way x86 and s390 no longer need to set empty/dummy routing when =
creating
> > > an IRQCHIP, and the worst case scenario of userspace misusing an ioct=
l() is no
> > > longer a NULL pointer deref.
>=20
> Sounds like a good idea. This should also speedup guest creation on s390 =
since
> it would avoid one syncronize_irq.
> >=20
> > To setup an empty IRQ routing table during kvm_create_vm() sounds a goo=
d idea,
> > at this time vCPU have not been created and kvm->lock is held so skippi=
ng
> > synchronization is safe here.
> >=20
> > However, there is one drawback, if vmm wants to emulate irqchip itself,
> > e.g. qemu with command line '-machine kernel-irqchip=3Doff' may not nee=
d
> > irqchip in kernel. How do we handle this issue?
>=20
> I would be fine with wasted memory.

+1.  If we really, really want to avoid the negligible memory overhead, we =
could
pre-configure a static global table and directly use that as the dummy tabl=
e (and
exempt it from being freed by free_irq_routing_table()).

> The only question is does it have a functional impact or can we simply ig=
nore
> the dummy routing.

Given the lack of sanity checks on kvm->irq_routing, I'm pretty sure the on=
ly way
for there to be functional impact is if there's a latent NULL pointer deref=
 hiding
somewhere.