Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "chenxiang (M)" <chenxiang66@hisilicon.com>
To:     Alex Williamson <alex.williamson@redhat.com>, <maz@kernel.org>,
        <pbonzini@redhat.com>
CC:     <kvm@vger.kernel.org>, chenxiang via <qemu-devel@nongnu.org>,
        linux-kernel <linux-kernel@vger.kernel.org>
Subject: [QUESTION] Exception print when enabling GICv4
Message-ID: <6d6d61fb-6241-4e1e-ddff-8ae8be96f9ff@hisilicon.com>
Date:   Tue, 12 Jul 2022 20:55:16 +0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:45.0) Gecko/20100101
 Thunderbird/45.2.0
MIME-Version: 1.0
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hi,
I encounter a issue related to GICv4 enable on ARM64 platform (kernel 
5.19-rc4, qemu 6.2.0):
We have a accelaration module whose VF has 3 MSI interrupts, and we 
passthrough it to virtual machine with following steps:

echo 0000:79:00.1 > /sys/bus/pci/drivers/hisi_hpre/unbind
echo vfio-pci > 
/sys/devices/pci0000\:78/0000\:78\:00.0/0000\:79\:00.1/driver_override
echo 0000:79:00.1 > /sys/bus/pci/drivers_probe

Then we boot VM with "-device vfio-pci,host=79:00.1,id=net0 \".
When insmod the driver which registers 3 PCI MSI interrupts in VM,  some 
exception print occur as following:

vfio-pci 0000:3a:00.1: irq bypass producer (token 000000008f08224d) 
registration fails: 66311

I find that bit[6:4] of register PCI_MSI_FLAGS is 2 (4 MSI interrupts) 
though we only register 3 PCI MSI interrupt,

and only 3 MSI interrupt is activated at last.
It allocates 4 vectors in function vfio_msi_enable() (qemu)  as it reads 
the register PCI_MSI_FLAGS.
Later it will  call system call VFIO_DEVICE_SET_IRQS to set forwarding 
for those interrupts
using function kvm_vgic_v4_set_forrwarding() as GICv4 is enabled. For 
interrupt 0~2, it success to set forwarding as they are already activated,
but for the 4th interrupt, it is not activated, so ite is not found in 
function vgic_its_resolve_lpi(), so above printk occurs.

It seems that we only allocate and activate 3 MSI interrupts in guest 
while it tried to set forwarding for 4 MSI interrupts in host.
Do you have any idea about this issue?


Best regards,

Xiang Chen