2020-05-12 04:10:03

by zhangfei

[permalink] [raw]
Subject: [PATCH 0/2] Let pci_fixup_final access iommu_fwnode

Some platform devices appear as PCI but are
actually on the AMBA bus, and they need fixup in
drivers/pci/quirks.c handling iommu_fwnode.
So calling pci_fixup_final after iommu_fwnode is allocated.

For example:
Hisilicon platform device need fixup in
drivers/pci/quirks.c

+static void quirk_huawei_pcie_sva(struct pci_dev *pdev)
+{
+ struct iommu_fwspec *fwspec;
+
+ pdev->eetlp_prefix_path = 1;
+ fwspec = dev_iommu_fwspec_get(&pdev->dev);
+ if (fwspec)
+ fwspec->can_stall = 1;
+}
+
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_HUAWEI, 0xa250, quirk_huawei_pcie_sva);
+DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_HUAWEI, 0xa251, quirk_huawei_pcie_sva);


Zhangfei Gao (2):
iommu/of: Let pci_fixup_final access iommu_fwnode
ACPI/IORT: Let pci_fixup_final access iommu_fwnode

drivers/acpi/arm64/iort.c | 1 +
drivers/iommu/of_iommu.c | 1 +
2 files changed, 2 insertions(+)

--
2.7.4


2020-05-12 04:11:30

by zhangfei

[permalink] [raw]
Subject: [PATCH 2/2] ACPI/IORT: Let pci_fixup_final access iommu_fwnode

Calling pci_fixup_final after iommu_fwspec_init, which alloc
iommu_fwnode. Some platform devices appear as PCI but are
actually on the AMBA bus, and they need fixup in
drivers/pci/quirks.c handling iommu_fwnode.
So calling pci_fixup_final after iommu_fwnode is allocated.

Signed-off-by: Zhangfei Gao <[email protected]>
---
drivers/acpi/arm64/iort.c | 1 +
1 file changed, 1 insertion(+)

diff --git a/drivers/acpi/arm64/iort.c b/drivers/acpi/arm64/iort.c
index 7d04424..02e361d 100644
--- a/drivers/acpi/arm64/iort.c
+++ b/drivers/acpi/arm64/iort.c
@@ -1027,6 +1027,7 @@ const struct iommu_ops *iort_iommu_configure(struct device *dev)
info.node = node;
err = pci_for_each_dma_alias(to_pci_dev(dev),
iort_pci_iommu_init, &info);
+ pci_fixup_device(pci_fixup_final, to_pci_dev(dev));

fwspec = dev_iommu_fwspec_get(dev);
if (fwspec && iort_pci_rc_supports_ats(node))
--
2.7.4

2020-05-22 02:14:21

by zhangfei

[permalink] [raw]
Subject: Re: [PATCH 0/2] Let pci_fixup_final access iommu_fwnode

Hi, Joerg

On 2020/5/12 下午12:08, Zhangfei Gao wrote:
> Some platform devices appear as PCI but are
> actually on the AMBA bus, and they need fixup in
> drivers/pci/quirks.c handling iommu_fwnode.
> So calling pci_fixup_final after iommu_fwnode is allocated.
>
> For example:
> Hisilicon platform device need fixup in
> drivers/pci/quirks.c
>
> +static void quirk_huawei_pcie_sva(struct pci_dev *pdev)
> +{
> + struct iommu_fwspec *fwspec;
> +
> + pdev->eetlp_prefix_path = 1;
> + fwspec = dev_iommu_fwspec_get(&pdev->dev);
> + if (fwspec)
> + fwspec->can_stall = 1;
> +}
> +
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_HUAWEI, 0xa250, quirk_huawei_pcie_sva);
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_HUAWEI, 0xa251, quirk_huawei_pcie_sva);
>
>
> Zhangfei Gao (2):
> iommu/of: Let pci_fixup_final access iommu_fwnode
> ACPI/IORT: Let pci_fixup_final access iommu_fwnode
>
> drivers/acpi/arm64/iort.c | 1 +
> drivers/iommu/of_iommu.c | 1 +
> 2 files changed, 2 insertions(+)
>
Would you mind give any suggestion?

We need access fwspec->can_stall describing the platform device (a fake
pcie) can support stall feature.
can_stall will be used arm_smmu_add_device [1].
And stall is not a pci feature, so no such member in struct pci_dev.

iommu_fwnode is allocated in iommu_fwspec_init, from of_pci_iommu_init
or iort_pci_iommu_init.
The pci_fixup_device(pci_fixup_final, dev) in pci_bus_add_device is too
early that  iommu_fwnode
is not allocated.
The pci_fixup_device(pci_fixup_enable, dev) in do_pci_enable_device is
too late after

arm_smmu_add_device.


So the idea here is calling pci_fixup_device(pci_fixup_final) after
of_pci_iommu_init and iort_pci_iommu_init, where iommu_fwnode is allocated.



[1] https://www.spinics.net/lists/linux-pci/msg94559.html

Thanks

2020-05-25 13:43:58

by Joerg Roedel

[permalink] [raw]
Subject: Re: [PATCH 0/2] Let pci_fixup_final access iommu_fwnode

On Tue, May 12, 2020 at 12:08:29PM +0800, Zhangfei Gao wrote:
> Some platform devices appear as PCI but are
> actually on the AMBA bus, and they need fixup in
> drivers/pci/quirks.c handling iommu_fwnode.
> So calling pci_fixup_final after iommu_fwnode is allocated.
>
> For example:
> Hisilicon platform device need fixup in
> drivers/pci/quirks.c
>
> +static void quirk_huawei_pcie_sva(struct pci_dev *pdev)
> +{
> + struct iommu_fwspec *fwspec;
> +
> + pdev->eetlp_prefix_path = 1;
> + fwspec = dev_iommu_fwspec_get(&pdev->dev);
> + if (fwspec)
> + fwspec->can_stall = 1;
> +}
> +
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_HUAWEI, 0xa250, quirk_huawei_pcie_sva);
> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_HUAWEI, 0xa251, quirk_huawei_pcie_sva);

I don't think it is a great idea to hook this into PCI_FIXUP_FINAL. The
fixup list needs to be processed for every device, which will slow down
probing.

So either we introduce something like PCI_FIXUP_IOMMU, if this is
entirely PCI specific. If it needs to be generic we need some fixup
infrastructure in the IOMMU code itself.

Regards,

Joerg

2020-05-26 13:01:46

by zhangfei

[permalink] [raw]
Subject: Re: [PATCH 0/2] Let pci_fixup_final access iommu_fwnode



On 2020/5/25 下午9:43, Joerg Roedel wrote:
> On Tue, May 12, 2020 at 12:08:29PM +0800, Zhangfei Gao wrote:
>> Some platform devices appear as PCI but are
>> actually on the AMBA bus, and they need fixup in
>> drivers/pci/quirks.c handling iommu_fwnode.
>> So calling pci_fixup_final after iommu_fwnode is allocated.
>>
>> For example:
>> Hisilicon platform device need fixup in
>> drivers/pci/quirks.c
>>
>> +static void quirk_huawei_pcie_sva(struct pci_dev *pdev)
>> +{
>> + struct iommu_fwspec *fwspec;
>> +
>> + pdev->eetlp_prefix_path = 1;
>> + fwspec = dev_iommu_fwspec_get(&pdev->dev);
>> + if (fwspec)
>> + fwspec->can_stall = 1;
>> +}
>> +
>> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_HUAWEI, 0xa250, quirk_huawei_pcie_sva);
>> +DECLARE_PCI_FIXUP_FINAL(PCI_VENDOR_ID_HUAWEI, 0xa251, quirk_huawei_pcie_sva);
> I don't think it is a great idea to hook this into PCI_FIXUP_FINAL. The
> fixup list needs to be processed for every device, which will slow down
> probing.
>
> So either we introduce something like PCI_FIXUP_IOMMU, if this is
> entirely PCI specific. If it needs to be generic we need some fixup
> infrastructure in the IOMMU code itself.

Thanks Joerg for the good suggestion.
I am trying to introduce PCI_FIXUP_IOMMU in
https://lkml.org/lkml/2020/5/26/366

Thanks