This series adds 'iommus' DT property for PCIe RC nodes so that IOMMU
can be used for address translation.
and setting pcie max link speed to gen4 that was linited to gen3 ealier.
Dependency for Patch 2
----------------------
Depends on:
https://lore.kernel.org/all/[email protected]/
https://lore.kernel.org/all/[email protected]/
Mrinmay Sarkar (2):
arm64: dts: qcom: sa8775p: Adding iommus property in pcie DT nodes
arm64: dts: qcom: sa8775p: Set max link speed to gen4 for ep pcie
arch/arm64/boot/dts/qcom/sa8775p.dtsi | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
--
2.7.4
Adding this change to set max link speed to gen4 as sa8775p supports
gen4 so that pcie link can be enumerated as gen4.
Signed-off-by: Mrinmay Sarkar <[email protected]>
---
arch/arm64/boot/dts/qcom/sa8775p.dtsi | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/arch/arm64/boot/dts/qcom/sa8775p.dtsi b/arch/arm64/boot/dts/qcom/sa8775p.dtsi
index 0c52180..aad2cd7 100644
--- a/arch/arm64/boot/dts/qcom/sa8775p.dtsi
+++ b/arch/arm64/boot/dts/qcom/sa8775p.dtsi
@@ -3730,7 +3730,7 @@
power-domains = <&gcc PCIE_0_GDSC>;
phys = <&pcie0_phy>;
phy-names = "pciephy";
- max-link-speed = <3>; /* FIXME: Limiting the Gen speed due to stability issues */
+ max-link-speed = <4>;
num-lanes = <2>;
status = "disabled";
@@ -3888,7 +3888,7 @@
power-domains = <&gcc PCIE_1_GDSC>;
phys = <&pcie1_phy>;
phy-names = "pciephy";
- max-link-speed = <3>; /* FIXME: Limiting the Gen speed due to stability issues */
+ max-link-speed = <4>;
num-lanes = <4>;
status = "disabled";
--
2.7.4
'iommus' is a list of phandle and IOMMU specifier pairs that describe
the IOMMU master interfaces of the device. Specified this property in
PCIe DT nodes so that IOMMU can be used for address translation.
Signed-off-by: Mrinmay Sarkar <[email protected]>
---
arch/arm64/boot/dts/qcom/sa8775p.dtsi | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/arm64/boot/dts/qcom/sa8775p.dtsi b/arch/arm64/boot/dts/qcom/sa8775p.dtsi
index 9065645..0c52180 100644
--- a/arch/arm64/boot/dts/qcom/sa8775p.dtsi
+++ b/arch/arm64/boot/dts/qcom/sa8775p.dtsi
@@ -3666,6 +3666,7 @@
<&gem_noc MASTER_APPSS_PROC 0 &config_noc SLAVE_PCIE_0 0>;
interconnect-names = "pcie-mem", "cpu-pcie";
+ iommus = <&pcie_smmu 0x0000 0x7f>;
iommu-map = <0x0 &pcie_smmu 0x0000 0x1>,
<0x100 &pcie_smmu 0x0001 0x1>;
@@ -3822,6 +3823,7 @@
<&gem_noc MASTER_APPSS_PROC 0 &config_noc SLAVE_PCIE_1 0>;
interconnect-names = "pcie-mem", "cpu-pcie";
+ iommus = <&pcie_smmu 0x0080 0x7f>;
iommu-map = <0x0 &pcie_smmu 0x0080 0x1>,
<0x100 &pcie_smmu 0x0081 0x1>;
--
2.7.4
On Tue, 30 Apr 2024 at 19:33, Mrinmay Sarkar <[email protected]> wrote:
>
> Adding this change to set max link speed to gen4 as sa8775p supports
> gen4 so that pcie link can be enumerated as gen4.
Previous patches mentioned stability issues. Were they solved?
>
> Signed-off-by: Mrinmay Sarkar <[email protected]>
> ---
> arch/arm64/boot/dts/qcom/sa8775p.dtsi | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/arch/arm64/boot/dts/qcom/sa8775p.dtsi b/arch/arm64/boot/dts/qcom/sa8775p.dtsi
> index 0c52180..aad2cd7 100644
> --- a/arch/arm64/boot/dts/qcom/sa8775p.dtsi
> +++ b/arch/arm64/boot/dts/qcom/sa8775p.dtsi
> @@ -3730,7 +3730,7 @@
> power-domains = <&gcc PCIE_0_GDSC>;
> phys = <&pcie0_phy>;
> phy-names = "pciephy";
> - max-link-speed = <3>; /* FIXME: Limiting the Gen speed due to stability issues */
> + max-link-speed = <4>;
> num-lanes = <2>;
>
> status = "disabled";
> @@ -3888,7 +3888,7 @@
> power-domains = <&gcc PCIE_1_GDSC>;
> phys = <&pcie1_phy>;
> phy-names = "pciephy";
> - max-link-speed = <3>; /* FIXME: Limiting the Gen speed due to stability issues */
I think you've just sent a patchset which adds this node. Is there any
reason for setting the max-link-speed to 3 just to change it to 4
immediately?
> + max-link-speed = <4>;
> num-lanes = <4>;
>
> status = "disabled";
> --
> 2.7.4
>
>
--
With best wishes
Dmitry
On 4/30/2024 11:20 PM, Dmitry Baryshkov wrote:
> On Tue, 30 Apr 2024 at 19:33, Mrinmay Sarkar <[email protected]> wrote:
>> Adding this change to set max link speed to gen4 as sa8775p supports
>> gen4 so that pcie link can be enumerated as gen4.
> Previous patches mentioned stability issues. Were they solved?
Hi Dmitry,
Thanks for review.
Actually earlier gen4 related equalization setting was missing in driver.
That's why gen4 was not stable and it was coming to gen3.
With this below driver change gen4 is stable now
https://lore.kernel.org/all/[email protected]/
>> Signed-off-by: Mrinmay Sarkar <[email protected]>
>> ---
>> arch/arm64/boot/dts/qcom/sa8775p.dtsi | 4 ++--
>> 1 file changed, 2 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/arm64/boot/dts/qcom/sa8775p.dtsi b/arch/arm64/boot/dts/qcom/sa8775p.dtsi
>> index 0c52180..aad2cd7 100644
>> --- a/arch/arm64/boot/dts/qcom/sa8775p.dtsi
>> +++ b/arch/arm64/boot/dts/qcom/sa8775p.dtsi
>> @@ -3730,7 +3730,7 @@
>> power-domains = <&gcc PCIE_0_GDSC>;
>> phys = <&pcie0_phy>;
>> phy-names = "pciephy";
>> - max-link-speed = <3>; /* FIXME: Limiting the Gen speed due to stability issues */
>> + max-link-speed = <4>;
>> num-lanes = <2>;
>>
>> status = "disabled";
>> @@ -3888,7 +3888,7 @@
>> power-domains = <&gcc PCIE_1_GDSC>;
>> phys = <&pcie1_phy>;
>> phy-names = "pciephy";
>> - max-link-speed = <3>; /* FIXME: Limiting the Gen speed due to stability issues */
> I think you've just sent a patchset which adds this node. Is there any
> reason for setting the max-link-speed to 3 just to change it to 4
> immediately?
Earlier we were not sure about the root cause of the issue
so limited the speed to gen3. Now as we have the solution
for this issue so moving to gen4.
> Thanks,
> Mrinmay
>> + max-link-speed = <4>;
>> num-lanes = <4>;
>>
>> status = "disabled";
>> --
>> 2.7.4
>>
>>
>
> --
> With best wishes
>
> Dmitry
On Tue, Apr 30, 2024 at 10:01:50PM +0530, Mrinmay Sarkar wrote:
> 'iommus' is a list of phandle and IOMMU specifier pairs that describe
> the IOMMU master interfaces of the device. Specified this property in
> PCIe DT nodes so that IOMMU can be used for address translation.
>
This patch description is heavily misleading. Even without the 'iommus'
property, there will be IOMMU translation because of 'iommu-map'. And I recently
got rid of 'iommus' property from all DTs because it is not really required for
the translation (it allows the host bridge to bind to IOMMU, but that's not what
we want).
This patch is intented to fix the IOMMU fault that occurs whenever the EP is
attached to the host. But you never described or even mentioned about the IOMMU
fault. Please describe the problem clearly and explain how the patch fixes that
in patch description.
Now for the IOMMU fault, I did some investigation and found that the fault is
happening due to some AER generated by the bridge whenever the device is
attached to the host. Interestingly, there was no AER IRQ received on the host.
But that can be expected due to the IOMMU fault as that could've blocked the AER
MSI from reaching the interrupt controller. And 'lspci' shows that the bridge
(even device) generated CE error (RxErr):
CESta: RxErr+ BadTLP- BadDLLP- Rollover- Timeout+ AdvNonFatalErr-
But I dont' know why the IOMMU fault occurs. I also tried to manually inject the
AER errors and I saw the AER IRQs are generated correctly. So this confirms that
there is no problem with AER itself.
For experimenting, I reduced the PCIe bandwidth to Gen 2, and the above error
was gone. So this hints that there could be something wrong with the PHY.
And yes, adding the 'iommus' property indeed makes the IOMMU fault go away, but
still I can see the AER error in lspci, but no actual IRQ received (weird). So
this patch is not really _fixing_ the issue, but just masking it in some form.
Please investigate on why the RxErr is being generated and how that ended up as
an IOMMU fault instead of an IRQ.
- Mani
> Signed-off-by: Mrinmay Sarkar <[email protected]>
> ---
> arch/arm64/boot/dts/qcom/sa8775p.dtsi | 2 ++
> 1 file changed, 2 insertions(+)
>
> diff --git a/arch/arm64/boot/dts/qcom/sa8775p.dtsi b/arch/arm64/boot/dts/qcom/sa8775p.dtsi
> index 9065645..0c52180 100644
> --- a/arch/arm64/boot/dts/qcom/sa8775p.dtsi
> +++ b/arch/arm64/boot/dts/qcom/sa8775p.dtsi
> @@ -3666,6 +3666,7 @@
> <&gem_noc MASTER_APPSS_PROC 0 &config_noc SLAVE_PCIE_0 0>;
> interconnect-names = "pcie-mem", "cpu-pcie";
>
> + iommus = <&pcie_smmu 0x0000 0x7f>;
> iommu-map = <0x0 &pcie_smmu 0x0000 0x1>,
> <0x100 &pcie_smmu 0x0001 0x1>;
>
> @@ -3822,6 +3823,7 @@
> <&gem_noc MASTER_APPSS_PROC 0 &config_noc SLAVE_PCIE_1 0>;
> interconnect-names = "pcie-mem", "cpu-pcie";
>
> + iommus = <&pcie_smmu 0x0080 0x7f>;
> iommu-map = <0x0 &pcie_smmu 0x0080 0x1>,
> <0x100 &pcie_smmu 0x0081 0x1>;
>
> --
> 2.7.4
>
--
மணிவண்ணன் சதாசிவம்