Received-SPF: pass (google.com: domain of linux-kernel+bounces-27592-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:4601:e00::3 as permitted sender) client-ip=2604:1380:4601:e00::3;
Message-ID: <72edaedc-50d7-415e-9c45-f17ffe0c1c23@linux.ibm.com>
Date: Tue, 16 Jan 2024 17:50:39 +0100
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH] KVM: irqchip: synchronize srcu only if needed
To: Yi Wang <up2wing@gmail.com>, Sean Christopherson <seanjc@google.com>
Cc: pbonzini@redhat.com, tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
        dave.hansen@linux.intel.com, x86@kernel.org, hpa@zytor.com,
        kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
        wanpengli@tencent.com, Yi Wang <foxywang@tencent.com>,
        Oliver Upton <oliver.upton@linux.dev>, Marc Zyngier <maz@kernel.org>,
        Anup Patel <anup@brainfault.org>, Atish Patra <atishp@atishpatra.org>,
        Janosch Frank <frankja@linux.ibm.com>,
        Claudio Imbrenda <imbrenda@linux.ibm.com>
References: <20240112091128.3868059-1-foxywang@tencent.com>
 <ZaFor2Lvdm4O2NWa@google.com>
 <CAN35MuSkQf0XmBZ5ZXGhcpUCGD-kKoyTv9G7ya4QVD1xiqOxLg@mail.gmail.com>
Content-Language: en-US
From: Christian Borntraeger <borntraeger@linux.ibm.com>
In-Reply-To: <CAN35MuSkQf0XmBZ5ZXGhcpUCGD-kKoyTv9G7ya4QVD1xiqOxLg@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit


Am 15.01.24 um 17:01 schrieb Yi Wang:
> Many thanks for your such kind and detailed reply, Sean!
> 
> On Sat, Jan 13, 2024 at 12:28 AM Sean Christopherson <seanjc@google.com> wrote:
>>
>> +other KVM maintainers
>>
>> On Fri, Jan 12, 2024, Yi Wang wrote:
>>> From: Yi Wang <foxywang@tencent.com>
>>>
>>> We found that it may cost more than 20 milliseconds very accidentally
>>> to enable cap of KVM_CAP_SPLIT_IRQCHIP on a host which has many vms
>>> already.
>>>
>>> The reason is that when vmm(qemu/CloudHypervisor) invokes
>>> KVM_CAP_SPLIT_IRQCHIP kvm will call synchronize_srcu_expedited() and
>>> might_sleep and kworker of srcu may cost some delay during this period.
>>
>> might_sleep() yielding is not justification for changing KVM.  That's more or
>> less saying "my task got preempted and took longer to run".  Well, yeah.
> 
> Agree. But I suppose it may be one of the reasons that makes  time of
> KVM_CAP_SPLIT_IRQCHIP delayed, of course, the kworker has the biggest
> suspicion :)
> 
>>
>>> Since this happens during creating vm, it's no need to synchronize srcu
>>> now 'cause everything is not ready(vcpu/irqfd) and none uses irq_srcu now.
> 
> ....
> 
>> And on x86, I'm pretty sure as of commit 654f1f13ea56 ("kvm: Check irqchip mode
>> before assign irqfd"), which added kvm_arch_irqfd_allowed(), it's impossible for
>> kvm_irq_map_gsi() to encounter a NULL irq_routing _on x86_.
>>
>> But I strongly suspect other architectures can reach kvm_irq_map_gsi() with a
>> NULL irq_routing, e.g. RISC-V dynamically configures its interrupt controller,
>> yet doesn't implement kvm_arch_intc_initialized().
>>
>> So instead of special casing x86, what if we instead have KVM setup an empty
>> IRQ routing table during kvm_create_vm(), and then avoid this mess entirely?
>> That way x86 and s390 no longer need to set empty/dummy routing when creating
>> an IRQCHIP, and the worst case scenario of userspace misusing an ioctl() is no
>> longer a NULL pointer deref.

Sounds like a good idea. This should also speedup guest creation on s390 since
it would avoid one syncronize_irq.
> 
> To setup an empty IRQ routing table during kvm_create_vm() sounds a good idea,
> at this time vCPU have not been created and kvm->lock is held so skipping
> synchronization is safe here.
> 
> However, there is one drawback, if vmm wants to emulate irqchip
> itself, e.g. qemu
> with command line '-machine kernel-irqchip=off' may not need irqchip
> in kernel. How
> do we handle this issue?

I would be fine with wasted memory. The only question is does it have a functional
impact or can we simply ignore the dummy routing.