Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Subject: Re: [QUESTION] Exception print when enabling GICv4
To:     Marc Zyngier <maz@kernel.org>
References: <6d6d61fb-6241-4e1e-ddff-8ae8be96f9ff@hisilicon.com>
 <87bktu1hfj.wl-maz@kernel.org>
CC:     Alex Williamson <alex.williamson@redhat.com>,
        <pbonzini@redhat.com>, <kvm@vger.kernel.org>,
        chenxiang via <qemu-devel@nongnu.org>,
        linux-kernel <linux-kernel@vger.kernel.org>
From:   "chenxiang (M)" <chenxiang66@hisilicon.com>
Message-ID: <13e4fde9-05e9-f492-a2b6-20d567eb2920@hisilicon.com>
Date:   Wed, 13 Jul 2022 14:02:10 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101
 Thunderbird/45.2.0
MIME-Version: 1.0
In-Reply-To: <87bktu1hfj.wl-maz@kernel.org>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk

Hi Marc,

Thank you for your reply.

在 2022/7/12 23:25, Marc Zyngier 写道:
> Hi Xiang,
>
> On Tue, 12 Jul 2022 13:55:16 +0100,
> "chenxiang (M)" <chenxiang66@hisilicon.com> wrote:
>> Hi,
>> I encounter a issue related to GICv4 enable on ARM64 platform (kernel
>> 5.19-rc4, qemu 6.2.0):
>> We have a accelaration module whose VF has 3 MSI interrupts, and we
>> passthrough it to virtual machine with following steps:
>>
>> echo 0000:79:00.1 > /sys/bus/pci/drivers/hisi_hpre/unbind
>> echo vfio-pci >
>> /sys/devices/pci0000\:78/0000\:78\:00.0/0000\:79\:00.1/driver_override
>> echo 0000:79:00.1 > /sys/bus/pci/drivers_probe
>>
>> Then we boot VM with "-device vfio-pci,host=79:00.1,id=net0 \".
>> When insmod the driver which registers 3 PCI MSI interrupts in VM,
>> some exception print occur as following:
>>
>> vfio-pci 0000:3a:00.1: irq bypass producer (token 000000008f08224d)
>> registration fails: 66311
>>
>> I find that bit[6:4] of register PCI_MSI_FLAGS is 2 (4 MSI interrupts)
>> though we only register 3 PCI MSI interrupt,
>>
>> and only 3 MSI interrupt is activated at last.
>> It allocates 4 vectors in function vfio_msi_enable() (qemu)  as it
>> reads the register PCI_MSI_FLAGS.
>> Later it will  call system call VFIO_DEVICE_SET_IRQS to set forwarding
>> for those interrupts
>> using function kvm_vgic_v4_set_forrwarding() as GICv4 is enabled. For
>> interrupt 0~2, it success to set forwarding as they are already
>> activated,
>> but for the 4th interrupt, it is not activated, so ite is not found in
>> function vgic_its_resolve_lpi(), so above printk occurs.
>>
>> It seems that we only allocate and activate 3 MSI interrupts in guest
>> while it tried to set forwarding for 4 MSI interrupts in host.
>> Do you have any idea about this issue?
> I have a hunch: QEMU cannot know that the guest is only using 3 MSIs
> out of the 4 that the device can use, and PCI/Multi-MSI only has a
> single enable bit for all MSIs. So it probably iterates over all
> possible MSIs and enable the forwarding. Since the guest has only
> created 3 mappings in the virtual ITS, the last call fails. I would
> expect the guest to still work properly though.

Yes, that's the reason of exception print.
Is it possible for QEMU to get the exact number of interrupts guest is 
using? It seems not.

>
> Thanks,
>
> 	M.
>