Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Kishon Vijay Abraham I <kishon@ti.com>
To:     Thomas Gleixner <tglx@linutronix.de>, Marc Zyngier <maz@kernel.org>
CC:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "Vutla, Lokesh" <lokeshvutla@ti.com>,
        "Nori, Sekhar" <nsekhar@ti.com>
Subject: [QUERY] Using same ITS device ID for two PCI devices (two PCI
 Requestor ID)
Message-ID: <5f8f586b-4308-dad7-d87d-9a341a248680@ti.com>
Date:   Thu, 16 Sep 2021 18:32:58 +0530
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
MIME-Version: 1.0
Content-Type: text/plain; charset="utf-8"
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hi Marc, Thomas,

TI's K3 platforms use GIT ITS for PCIe MSI/MSI-X interrupts. It uses
*pre_its_window* as implemented by *its_enable_quirk_socionext_synquacer* in
irq-gic-v3-its.c. So PCIe controller instead of directly writing to
GITS_TRANSLATER, will write to a separate window and the device ID is taken from
the offset to which the PCIe device writes (instead of dedicated lines from PCIe
controller to GIC ITS). So every 4-byte register address maps in this window
maps to a unique ITS device id.

All of this is already implemented in Linux Kernel
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/drivers/irqchip/irq-gic-v3-its.c#n4645

Now TI's AM64 has an issue in that it doesn't trigger interrupt if the address
in the *pre_its_window* is not aligned to 8-bytes (this is due to an invalid
bridge configuration in HW).

This means there will not be interrupts for devices with PCIe requestor ID, 0x1,
0x3, 0x5..., as the address in the pre-ITS window would be 4 (1 << 2), 12 (3 <<
2), 20 (5 << 2) respectively.

So in order to provide 8 byte aligned address always, I mapped the PCIe
requestor ID to ITS device ID such that it always provides 8-byte aligned
address. The DT property like below helped me achieve that.

msi-map = <0x0 &gic_its 0x0 0x10000>;
msi-map-mask = <0xfffe>;

So this would result in creating one "struct its_device" for 2 PCIe devices and
the pre-ITS address will be aligned to 8-bytes.

However with this, its_alloc_device_irq() for the 2nd PCIe device is failing
since we create "struct its_device" with the number of interrupt vectors
requested by the 1st PCIe device.

Would like to get your opinion on what would be the best way to workaround this
for AM64.

One option would be to create a new compatible for AM64 ("ti,am64,gic-v3-its")
allocate a minimal number of interrupt vector while creating "struct
its_device". Would that be acceptable? Any other ideas?

Thanks,
Kishon