Replace dublicated MRRS limit quirks by mrrs_limit_quirk from core
* drivers/pci/controller/dwc/pci-keystone.c
* drivers/pci/controller/pci-loongson.c
Both ks_pcie_quirk loongson_mrrs_quirk was rewritten without any
functionality changes by one mrrs_limit_quirk
Added DesignWare PCI controller which need same quirk for
* drivers/pci/controller/dwc/pci-meson.c (PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3)
This quirk can solve some issue for Khadas VIM3/VIM3L(Amlogic)
with HDMI scrambled picture and nvme devices at intensive writing...
come from:
* https://lore.kernel.org/linux-pci/[email protected]/
Artem Lapkin (4):
PCI: move Keystone and Loongson device IDs to pci_ids
PCI: core: quirks: add mrrs_limit_quirk
PCI: keystone move mrrs quirk to core
PCI: loongson move mrrs quirk to core
--
2.25.1
[+cc Huacai]
On Sat, Jun 19, 2021 at 02:39:48PM +0800, Artem Lapkin wrote:
> Replace dublicated MRRS limit quirks by mrrs_limit_quirk from core
> * drivers/pci/controller/dwc/pci-keystone.c
> * drivers/pci/controller/pci-loongson.c
s/dublicated/duplicated/ (several occurrences)
Capitalize subject lines.
Use "git log --online" to learn conventions and follow them.
Add "()" after function names.
Capitalize acronyms appropriately (NVMe, MRRS, PCI, etc).
End sentences with periods.
A "move" patch must include both the removal and the addition and make
no changes to the code itself.
Amlogic appears without explanation in 2/4. Must be separate patch to
address only that specific issue. Should reference published erratum
if possible. "Solves some issue" is not a compelling justification.
The tree must be consistent and functionally the same or improved
after every patch.
Add to pci_ids.h only if symbol used more than one place.
See
https://lore.kernel.org/r/[email protected],
which looks similar. Combine efforts if possible and cc Huacai so
you're both aware of overlapping work.
More hints in case they're useful:
https://lore.kernel.org/linux-pci/[email protected]/
> Both ks_pcie_quirk loongson_mrrs_quirk was rewritten without any
> functionality changes by one mrrs_limit_quirk
>
> Added DesignWare PCI controller which need same quirk for
> * drivers/pci/controller/dwc/pci-meson.c (PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3)
>
> This quirk can solve some issue for Khadas VIM3/VIM3L(Amlogic)
> with HDMI scrambled picture and nvme devices at intensive writing...
>
> come from:
> * https://lore.kernel.org/linux-pci/[email protected]/
>
> Artem Lapkin (4):
> PCI: move Keystone and Loongson device IDs to pci_ids
> PCI: core: quirks: add mrrs_limit_quirk
> PCI: keystone move mrrs quirk to core
> PCI: loongson move mrrs quirk to core
>
> --
> 2.25.1
>
Hi, Bjorn,
> -----原始邮件-----
> 发件人: "Bjorn Helgaas" <[email protected]>
> 发送时间: 2021-07-01 23:46:34 (星期四)
> 收件人: "Artem Lapkin" <[email protected]>
> 抄送: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], "Huacai Chen" <[email protected]>
> 主题: Re: [PATCH 0/4] PCI: replace dublicated MRRS limit quirks
>
> [+cc Huacai]
>
> On Sat, Jun 19, 2021 at 02:39:48PM +0800, Artem Lapkin wrote:
> > Replace dublicated MRRS limit quirks by mrrs_limit_quirk from core
> > * drivers/pci/controller/dwc/pci-keystone.c
> > * drivers/pci/controller/pci-loongson.c
>
> s/dublicated/duplicated/ (several occurrences)
>
> Capitalize subject lines.
>
> Use "git log --online" to learn conventions and follow them.
>
> Add "()" after function names.
>
> Capitalize acronyms appropriately (NVMe, MRRS, PCI, etc).
>
> End sentences with periods.
>
> A "move" patch must include both the removal and the addition and make
> no changes to the code itself.
>
> Amlogic appears without explanation in 2/4. Must be separate patch to
> address only that specific issue. Should reference published erratum
> if possible. "Solves some issue" is not a compelling justification.
>
> The tree must be consistent and functionally the same or improved
> after every patch.
>
> Add to pci_ids.h only if symbol used more than one place.
>
> See
> https://lore.kernel.org/r/[email protected],
> which looks similar. Combine efforts if possible and cc Huacai so
> you're both aware of overlapping work.
>
> More hints in case they're useful:
> https://lore.kernel.org/linux-pci/[email protected]/
>
> > Both ks_pcie_quirk loongson_mrrs_quirk was rewritten without any
> > functionality changes by one mrrs_limit_quirk
Does that means keystone and Loongson has the same MRRS problem? And what should I do now?
Huacai
> >
> > Added DesignWare PCI controller which need same quirk for
> > * drivers/pci/controller/dwc/pci-meson.c (PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3)
> >
> > This quirk can solve some issue for Khadas VIM3/VIM3L(Amlogic)
> > with HDMI scrambled picture and nvme devices at intensive writing...
> >
> > come from:
> > * https://lore.kernel.org/linux-pci/[email protected]/
> >
> > Artem Lapkin (4):
> > PCI: move Keystone and Loongson device IDs to pci_ids
> > PCI: core: quirks: add mrrs_limit_quirk
> > PCI: keystone move mrrs quirk to core
> > PCI: loongson move mrrs quirk to core
> >
> > --
> > 2.25.1
> >
</[email protected]></[email protected]></[email protected]>
> Does that means keystone and Loongson has the same MRRS problem? And what should I do now?
Look like yes ! and amlogic has the same problem.
I think somebody need to rewrite it all to one common quirk for this problem.
If no one has any objection, I can try to remake it again.
On Fri, Jul 2, 2021 at 9:15 AM 陈华才 <[email protected]> wrote:
>
> Hi, Bjorn,
>
> > -----原始邮件-----
> > 发件人: "Bjorn Helgaas" <[email protected]>
> > 发送时间: 2021-07-01 23:46:34 (星期四)
> > 收件人: "Artem Lapkin" <[email protected]>
> > 抄送: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], "Huacai Chen" <[email protected]>
> > 主题: Re: [PATCH 0/4] PCI: replace dublicated MRRS limit quirks
> >
> > [+cc Huacai]
> >
> > On Sat, Jun 19, 2021 at 02:39:48PM +0800, Artem Lapkin wrote:
> > > Replace dublicated MRRS limit quirks by mrrs_limit_quirk from core
> > > * drivers/pci/controller/dwc/pci-keystone.c
> > > * drivers/pci/controller/pci-loongson.c
> >
> > s/dublicated/duplicated/ (several occurrences)
> >
> > Capitalize subject lines.
> >
> > Use "git log --online" to learn conventions and follow them.
> >
> > Add "()" after function names.
> >
> > Capitalize acronyms appropriately (NVMe, MRRS, PCI, etc).
> >
> > End sentences with periods.
> >
> > A "move" patch must include both the removal and the addition and make
> > no changes to the code itself.
> >
> > Amlogic appears without explanation in 2/4. Must be separate patch to
> > address only that specific issue. Should reference published erratum
> > if possible. "Solves some issue" is not a compelling justification.
> >
> > The tree must be consistent and functionally the same or improved
> > after every patch.
> >
> > Add to pci_ids.h only if symbol used more than one place.
> >
> > See
> > https://lore.kernel.org/r/[email protected],
> > which looks similar. Combine efforts if possible and cc Huacai so
> > you're both aware of overlapping work.
> >
> > More hints in case they're useful:
> > https://lore.kernel.org/linux-pci/[email protected]/
> >
> > > Both ks_pcie_quirk loongson_mrrs_quirk was rewritten without any
> > > functionality changes by one mrrs_limit_quirk
> Does that means keystone and Loongson has the same MRRS problem? And what should I do now?
>
> Huacai
> > >
> > > Added DesignWare PCI controller which need same quirk for
> > > * drivers/pci/controller/dwc/pci-meson.c (PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3)
> > >
> > > This quirk can solve some issue for Khadas VIM3/VIM3L(Amlogic)
> > > with HDMI scrambled picture and nvme devices at intensive writing...
> > >
> > > come from:
> > > * https://lore.kernel.org/linux-pci/[email protected]/
> > >
> > > Artem Lapkin (4):
> > > PCI: move Keystone and Loongson device IDs to pci_ids
> > > PCI: core: quirks: add mrrs_limit_quirk
> > > PCI: keystone move mrrs quirk to core
> > > PCI: loongson move mrrs quirk to core
> > >
> > > --
> > > 2.25.1
> > >
>
>
> </[email protected]></[email protected]></[email protected]>
Hi Art,
[...]
> If no one has any objection, I can try to remake it again.
I can't speak for everyone, of course, but I am sure that nobody would
object if you decide to work on v2, so that a proper fix can be applied,
and issues fixed long term.
Thank you for working on this! Much appreciated!
Krzysztof
Hi, Art,
On Mon, Jul 5, 2021 at 4:35 PM Art Nikpal <[email protected]> wrote:
>
> > Does that means keystone and Loongson has the same MRRS problem? And what should I do now?
>
> Look like yes ! and amlogic has the same problem.
> I think somebody need to rewrite it all to one common quirk for this problem.
>
> If no one has any objection, I can try to remake it again.
But, Loongson platform has newer revision of hardware, and the MRRS
quirk has changed, please see:
https://patchwork.kernel.org/project/linux-pci/list/?series=509497
Huacai
>
> On Fri, Jul 2, 2021 at 9:15 AM 陈华才 <[email protected]> wrote:
> >
> > Hi, Bjorn,
> >
> > > -----原始邮件-----
> > > 发件人: "Bjorn Helgaas" <[email protected]>
> > > 发送时间: 2021-07-01 23:46:34 (星期四)
> > > 收件人: "Artem Lapkin" <[email protected]>
> > > 抄送: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], "Huacai Chen" <[email protected]>
> > > 主题: Re: [PATCH 0/4] PCI: replace dublicated MRRS limit quirks
> > >
> > > [+cc Huacai]
> > >
> > > On Sat, Jun 19, 2021 at 02:39:48PM +0800, Artem Lapkin wrote:
> > > > Replace dublicated MRRS limit quirks by mrrs_limit_quirk from core
> > > > * drivers/pci/controller/dwc/pci-keystone.c
> > > > * drivers/pci/controller/pci-loongson.c
> > >
> > > s/dublicated/duplicated/ (several occurrences)
> > >
> > > Capitalize subject lines.
> > >
> > > Use "git log --online" to learn conventions and follow them.
> > >
> > > Add "()" after function names.
> > >
> > > Capitalize acronyms appropriately (NVMe, MRRS, PCI, etc).
> > >
> > > End sentences with periods.
> > >
> > > A "move" patch must include both the removal and the addition and make
> > > no changes to the code itself.
> > >
> > > Amlogic appears without explanation in 2/4. Must be separate patch to
> > > address only that specific issue. Should reference published erratum
> > > if possible. "Solves some issue" is not a compelling justification.
> > >
> > > The tree must be consistent and functionally the same or improved
> > > after every patch.
> > >
> > > Add to pci_ids.h only if symbol used more than one place.
> > >
> > > See
> > > https://lore.kernel.org/r/[email protected],
> > > which looks similar. Combine efforts if possible and cc Huacai so
> > > you're both aware of overlapping work.
> > >
> > > More hints in case they're useful:
> > > https://lore.kernel.org/linux-pci/[email protected]/
> > >
> > > > Both ks_pcie_quirk loongson_mrrs_quirk was rewritten without any
> > > > functionality changes by one mrrs_limit_quirk
> > Does that means keystone and Loongson has the same MRRS problem? And what should I do now?
> >
> > Huacai
> > > >
> > > > Added DesignWare PCI controller which need same quirk for
> > > > * drivers/pci/controller/dwc/pci-meson.c (PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3)
> > > >
> > > > This quirk can solve some issue for Khadas VIM3/VIM3L(Amlogic)
> > > > with HDMI scrambled picture and nvme devices at intensive writing...
> > > >
> > > > come from:
> > > > * https://lore.kernel.org/linux-pci/[email protected]/
> > > >
> > > > Artem Lapkin (4):
> > > > PCI: move Keystone and Loongson device IDs to pci_ids
> > > > PCI: core: quirks: add mrrs_limit_quirk
> > > > PCI: keystone move mrrs quirk to core
> > > > PCI: loongson move mrrs quirk to core
> > > >
> > > > --
> > > > 2.25.1
> > > >
> >
> >
> > </[email protected]></[email protected]></[email protected]>
>But, Loongson platform has newer revision of hardware, and the MRRS
> quirk has changed, please see:
> https://patchwork.kernel.org/project/linux-pci/list/?series=509497
> Huacai
OK! tnx for information ! maybe we can cooperate and make one
universal quirk for all
On Tue, Jul 6, 2021 at 9:36 AM Huacai Chen <[email protected]> wrote:
>
> Hi, Art,
>
> On Mon, Jul 5, 2021 at 4:35 PM Art Nikpal <[email protected]> wrote:
> >
> > > Does that means keystone and Loongson has the same MRRS problem? And what should I do now?
> >
> > Look like yes ! and amlogic has the same problem.
> > I think somebody need to rewrite it all to one common quirk for this problem.
> >
> > If no one has any objection, I can try to remake it again.
> But, Loongson platform has newer revision of hardware, and the MRRS
> quirk has changed, please see:
> https://patchwork.kernel.org/project/linux-pci/list/?series=509497
>
> Huacai
> >
> > On Fri, Jul 2, 2021 at 9:15 AM 陈华才 <[email protected]> wrote:
> > >
> > > Hi, Bjorn,
> > >
> > > > -----原始邮件-----
> > > > 发件人: "Bjorn Helgaas" <[email protected]>
> > > > 发送时间: 2021-07-01 23:46:34 (星期四)
> > > > 收件人: "Artem Lapkin" <[email protected]>
> > > > 抄送: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], "Huacai Chen" <[email protected]>
> > > > 主题: Re: [PATCH 0/4] PCI: replace dublicated MRRS limit quirks
> > > >
> > > > [+cc Huacai]
> > > >
> > > > On Sat, Jun 19, 2021 at 02:39:48PM +0800, Artem Lapkin wrote:
> > > > > Replace dublicated MRRS limit quirks by mrrs_limit_quirk from core
> > > > > * drivers/pci/controller/dwc/pci-keystone.c
> > > > > * drivers/pci/controller/pci-loongson.c
> > > >
> > > > s/dublicated/duplicated/ (several occurrences)
> > > >
> > > > Capitalize subject lines.
> > > >
> > > > Use "git log --online" to learn conventions and follow them.
> > > >
> > > > Add "()" after function names.
> > > >
> > > > Capitalize acronyms appropriately (NVMe, MRRS, PCI, etc).
> > > >
> > > > End sentences with periods.
> > > >
> > > > A "move" patch must include both the removal and the addition and make
> > > > no changes to the code itself.
> > > >
> > > > Amlogic appears without explanation in 2/4. Must be separate patch to
> > > > address only that specific issue. Should reference published erratum
> > > > if possible. "Solves some issue" is not a compelling justification.
> > > >
> > > > The tree must be consistent and functionally the same or improved
> > > > after every patch.
> > > >
> > > > Add to pci_ids.h only if symbol used more than one place.
> > > >
> > > > See
> > > > https://lore.kernel.org/r/[email protected],
> > > > which looks similar. Combine efforts if possible and cc Huacai so
> > > > you're both aware of overlapping work.
> > > >
> > > > More hints in case they're useful:
> > > > https://lore.kernel.org/linux-pci/[email protected]/
> > > >
> > > > > Both ks_pcie_quirk loongson_mrrs_quirk was rewritten without any
> > > > > functionality changes by one mrrs_limit_quirk
> > > Does that means keystone and Loongson has the same MRRS problem? And what should I do now?
> > >
> > > Huacai
> > > > >
> > > > > Added DesignWare PCI controller which need same quirk for
> > > > > * drivers/pci/controller/dwc/pci-meson.c (PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3)
> > > > >
> > > > > This quirk can solve some issue for Khadas VIM3/VIM3L(Amlogic)
> > > > > with HDMI scrambled picture and nvme devices at intensive writing...
> > > > >
> > > > > come from:
> > > > > * https://lore.kernel.org/linux-pci/[email protected]/
> > > > >
> > > > > Artem Lapkin (4):
> > > > > PCI: move Keystone and Loongson device IDs to pci_ids
> > > > > PCI: core: quirks: add mrrs_limit_quirk
> > > > > PCI: keystone move mrrs quirk to core
> > > > > PCI: loongson move mrrs quirk to core
> > > > >
> > > > > --
> > > > > 2.25.1
> > > > >
> > >
> > >
> > > </[email protected]></[email protected]></[email protected]>
Hi,
On 06/07/2021 08:06, Art Nikpal wrote:
>> But, Loongson platform has newer revision of hardware, and the MRRS
>> quirk has changed, please see:
>> https://patchwork.kernel.org/project/linux-pci/list/?series=509497
>> Huacai
>
> OK! tnx for information ! maybe we can cooperate and make one
> universal quirk for all
In their Designware PCIe controller driver, amlogic sets the Max_Payload_Size & Max_Read_Request_Size to 256:
https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L260
https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L276
in their root port PCIe Express Device Control Register.
Looking at the Synopsys DW-PCIe Databook, Max_Payload_Size & Max_Read_Request_Size are used to decompose into AXI burst,
but it seems the Max_Payload_Size & Max_Read_Request_Size are set by default to 512 but the internal Max_Payload_Size_Supported
is set to 256, thus changing these values to 256 at runtime to match and optimize bandwidth.
It's said, "Reducing Outbound Decomposition" :
- "Ensure that your application master does not generate bursts of size greater than or equal to Max_Payload_Size"
- "Program your PCIe system with a larger value of Max_Payload_Size without exceeding Max_Payload_Size_Supported"
- "Program your PCIe system with a larger value of Max_Read_Request without exceeding Max_Payload_Size_Supported:
So leaving 512 in Max_Payload_Size & Max_Read_Request leads to Outbound Decomposition which decreases PCIe link and degrades
the AXI bus by doubling the bursts, leading to this fix to avoid overflowing the AXI bus.
So it seems to be still needed, I assume this *should* be handled in the core somehow to propagate these settings to child endpoints to match
the root port Max_Payload_Size & Max_Read_Request sizes.
Maybe by adding a core function to set these values instead of using the dw_pcie_find_capability() & dw_pcie_write/readl_dbi() helpers
and set a state on the root port to propagate the value ?
Neil
>
> On Tue, Jul 6, 2021 at 9:36 AM Huacai Chen <[email protected]> wrote:
>>
>> Hi, Art,
>>
>> On Mon, Jul 5, 2021 at 4:35 PM Art Nikpal <[email protected]> wrote:
>>>
>>>> Does that means keystone and Loongson has the same MRRS problem? And what should I do now?
>>>
>>> Look like yes ! and amlogic has the same problem.
>>> I think somebody need to rewrite it all to one common quirk for this problem.
>>>
>>> If no one has any objection, I can try to remake it again.
>> But, Loongson platform has newer revision of hardware, and the MRRS
>> quirk has changed, please see:
>> https://patchwork.kernel.org/project/linux-pci/list/?series=509497
>>
>> Huacai
>>>
>>> On Fri, Jul 2, 2021 at 9:15 AM 陈华才 <[email protected]> wrote:
>>>>
>>>> Hi, Bjorn,
>>>>
>>>> > -----原始邮件-----
>>>> > 发件人: "Bjorn Helgaas" <[email protected]>
>>>> > 发送时间: 2021-07-01 23:46:34 (星期四)
>>>> > 收件人: "Artem Lapkin" <[email protected]>
>>>> > 抄送: [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], [email protected], "Huacai Chen" <[email protected]>
>>>> > 主题: Re: [PATCH 0/4] PCI: replace dublicated MRRS limit quirks
>>>> >
>>>> > [+cc Huacai]
>>>> >
>>>> > On Sat, Jun 19, 2021 at 02:39:48PM +0800, Artem Lapkin wrote:
>>>> > > Replace dublicated MRRS limit quirks by mrrs_limit_quirk from core
>>>> > > * drivers/pci/controller/dwc/pci-keystone.c
>>>> > > * drivers/pci/controller/pci-loongson.c
>>>> >
>>>> > s/dublicated/duplicated/ (several occurrences)
>>>> >
>>>> > Capitalize subject lines.
>>>> >
>>>> > Use "git log --online" to learn conventions and follow them.
>>>> >
>>>> > Add "()" after function names.
>>>> >
>>>> > Capitalize acronyms appropriately (NVMe, MRRS, PCI, etc).
>>>> >
>>>> > End sentences with periods.
>>>> >
>>>> > A "move" patch must include both the removal and the addition and make
>>>> > no changes to the code itself.
>>>> >
>>>> > Amlogic appears without explanation in 2/4. Must be separate patch to
>>>> > address only that specific issue. Should reference published erratum
>>>> > if possible. "Solves some issue" is not a compelling justification.
>>>> >
>>>> > The tree must be consistent and functionally the same or improved
>>>> > after every patch.
>>>> >
>>>> > Add to pci_ids.h only if symbol used more than one place.
>>>> >
>>>> > See
>>>> > https://lore.kernel.org/r/[email protected],
>>>> > which looks similar. Combine efforts if possible and cc Huacai so
>>>> > you're both aware of overlapping work.
>>>> >
>>>> > More hints in case they're useful:
>>>> > https://lore.kernel.org/linux-pci/[email protected]/
>>>> >
>>>> > > Both ks_pcie_quirk loongson_mrrs_quirk was rewritten without any
>>>> > > functionality changes by one mrrs_limit_quirk
>>>> Does that means keystone and Loongson has the same MRRS problem? And what should I do now?
>>>>
>>>> Huacai
>>>> > >
>>>> > > Added DesignWare PCI controller which need same quirk for
>>>> > > * drivers/pci/controller/dwc/pci-meson.c (PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3)
>>>> > >
>>>> > > This quirk can solve some issue for Khadas VIM3/VIM3L(Amlogic)
>>>> > > with HDMI scrambled picture and nvme devices at intensive writing...
>>>> > >
>>>> > > come from:
>>>> > > * https://lore.kernel.org/linux-pci/[email protected]/
>>>> > >
>>>> > > Artem Lapkin (4):
>>>> > > PCI: move Keystone and Loongson device IDs to pci_ids
>>>> > > PCI: core: quirks: add mrrs_limit_quirk
>>>> > > PCI: keystone move mrrs quirk to core
>>>> > > PCI: loongson move mrrs quirk to core
>>>> > >
>>>> > > --
>>>> > > 2.25.1
>>>> > >
>>>>
>>>>
>>>> </[email protected]></[email protected]></[email protected]>
On Tue, Jul 06, 2021 at 11:54:05AM +0200, Neil Armstrong wrote:
> In their Designware PCIe controller driver, amlogic sets the
> Max_Payload_Size & Max_Read_Request_Size to 256:
> https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L260
> https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L276
> in their root port PCIe Express Device Control Register.
>
> Looking at the Synopsys DW-PCIe Databook, Max_Payload_Size &
> Max_Read_Request_Size are used to decompose into AXI burst, but it
> seems the Max_Payload_Size & Max_Read_Request_Size are set by
> default to 512 but the internal Max_Payload_Size_Supported is set to
> 256, thus changing these values to 256 at runtime to match and
> optimize bandwidth.
>
> It's said, "Reducing Outbound Decomposition" :
> - "Ensure that your application master does not generate bursts of
> size greater than or equal to Max_Payload_Size"
>
> - "Program your PCIe system with a larger value of Max_Payload_Size
> without exceeding Max_Payload_Size_Supported"
>
> - "Program your PCIe system with a larger value of Max_Read_Request
> without exceeding Max_Payload_Size_Supported:
>
> So leaving 512 in Max_Payload_Size & Max_Read_Request leads to
> Outbound Decomposition which decreases PCIe link and degrades the
> AXI bus by doubling the bursts, leading to this fix to avoid
> overflowing the AXI bus.
>
> So it seems to be still needed, I assume this *should* be handled in
> the core somehow to propagate these settings to child endpoints to
> match the root port Max_Payload_Size & Max_Read_Request sizes.
>
> Maybe by adding a core function to set these values instead of using
> the dw_pcie_find_capability() & dw_pcie_write/readl_dbi() helpers
> and set a state on the root port to propagate the value ?
I don't have the Synopsys DW-PCIe Databook, so I'm lacking any
context. The above *seems* to say that MPS/MRRS settings affect AXI
bus usage.
The MPS and MRRS registers are defined to affect traffic on *PCIe*. If
a platform uses MPS and MRRS values to optimize transfers on non-PCIe
links, that's a problem because the PCI core code that manages MPS and
MRRS has no knowledge of those non-PCIe parts of the system.
You might be able to deal with this in Synopsys-specific code somehow,
but it's going to be a bit of a hassle because I don't want it to make
maintenance of the generic MPS/MRRS code harder.
Bjorn
Hi,
On 07/07/2021 17:54, Bjorn Helgaas wrote:
> On Tue, Jul 06, 2021 at 11:54:05AM +0200, Neil Armstrong wrote:
>> In their Designware PCIe controller driver, amlogic sets the
>> Max_Payload_Size & Max_Read_Request_Size to 256:
>> https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L260
>> https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L276
>> in their root port PCIe Express Device Control Register.
>>
>> Looking at the Synopsys DW-PCIe Databook, Max_Payload_Size &
>> Max_Read_Request_Size are used to decompose into AXI burst, but it
>> seems the Max_Payload_Size & Max_Read_Request_Size are set by
>> default to 512 but the internal Max_Payload_Size_Supported is set to
>> 256, thus changing these values to 256 at runtime to match and
>> optimize bandwidth.
>>
>> It's said, "Reducing Outbound Decomposition" :
>> - "Ensure that your application master does not generate bursts of
>> size greater than or equal to Max_Payload_Size"
>>
>> - "Program your PCIe system with a larger value of Max_Payload_Size
>> without exceeding Max_Payload_Size_Supported"
>>
>> - "Program your PCIe system with a larger value of Max_Read_Request
>> without exceeding Max_Payload_Size_Supported:
>>
>> So leaving 512 in Max_Payload_Size & Max_Read_Request leads to
>> Outbound Decomposition which decreases PCIe link and degrades the
>> AXI bus by doubling the bursts, leading to this fix to avoid
>> overflowing the AXI bus.
>>
>> So it seems to be still needed, I assume this *should* be handled in
>> the core somehow to propagate these settings to child endpoints to
>> match the root port Max_Payload_Size & Max_Read_Request sizes.
>>
>> Maybe by adding a core function to set these values instead of using
>> the dw_pcie_find_capability() & dw_pcie_write/readl_dbi() helpers
>> and set a state on the root port to propagate the value ?
>
> I don't have the Synopsys DW-PCIe Databook, so I'm lacking any
> context. The above *seems* to say that MPS/MRRS settings affect AXI
> bus usage.
It does when the TLPs are directed to the RC.
>
> The MPS and MRRS registers are defined to affect traffic on *PCIe*. If
> a platform uses MPS and MRRS values to optimize transfers on non-PCIe
> links, that's a problem because the PCI core code that manages MPS and
> MRRS has no knowledge of those non-PCIe parts of the system.
Yes and no, it only affects PCIe in P2P, in non-P2P is will certainly affect
transfers on the internal SoC/Processor/Chip internal bus/fabric.
> You might be able to deal with this in Synopsys-specific code somehow,
> but it's going to be a bit of a hassle because I don't want it to make
> maintenance of the generic MPS/MRRS code harder.
I understand, but this is why these quirks are currently implemented in the
controller driver and only applies when the controller has been probed
and to each endpoint detected on this particular controller.
So we may continue having separate quirks for each controller if the core
isn't the right place to handle MPS/MRRS.
Neil
> Bjorn
>
On Wed, Jul 07, 2021 at 06:43:13PM +0200, Neil Armstrong wrote:
> On 07/07/2021 17:54, Bjorn Helgaas wrote:
> > On Tue, Jul 06, 2021 at 11:54:05AM +0200, Neil Armstrong wrote:
> >> In their Designware PCIe controller driver, amlogic sets the
> >> Max_Payload_Size & Max_Read_Request_Size to 256:
> >> https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L260
> >> https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L276
> >> in their root port PCIe Express Device Control Register.
> >>
> >> Looking at the Synopsys DW-PCIe Databook, Max_Payload_Size &
> >> Max_Read_Request_Size are used to decompose into AXI burst, but it
> >> seems the Max_Payload_Size & Max_Read_Request_Size are set by
> >> default to 512 but the internal Max_Payload_Size_Supported is set to
> >> 256, thus changing these values to 256 at runtime to match and
> >> optimize bandwidth.
> >>
> >> It's said, "Reducing Outbound Decomposition" :
> >> - "Ensure that your application master does not generate bursts of
> >> size greater than or equal to Max_Payload_Size"
> >>
> >> - "Program your PCIe system with a larger value of Max_Payload_Size
> >> without exceeding Max_Payload_Size_Supported"
> >>
> >> - "Program your PCIe system with a larger value of Max_Read_Request
> >> without exceeding Max_Payload_Size_Supported:
> >>
> >> So leaving 512 in Max_Payload_Size & Max_Read_Request leads to
> >> Outbound Decomposition which decreases PCIe link and degrades the
> >> AXI bus by doubling the bursts, leading to this fix to avoid
> >> overflowing the AXI bus.
> >>
> >> So it seems to be still needed, I assume this *should* be handled in
> >> the core somehow to propagate these settings to child endpoints to
> >> match the root port Max_Payload_Size & Max_Read_Request sizes.
> >>
> >> Maybe by adding a core function to set these values instead of using
> >> the dw_pcie_find_capability() & dw_pcie_write/readl_dbi() helpers
> >> and set a state on the root port to propagate the value ?
> >
> > I don't have the Synopsys DW-PCIe Databook, so I'm lacking any
> > context. The above *seems* to say that MPS/MRRS settings affect AXI
> > bus usage.
>
> It does when the TLPs are directed to the RC.
That's a defect in the RC.
> > The MPS and MRRS registers are defined to affect traffic on *PCIe*. If
> > a platform uses MPS and MRRS values to optimize transfers on non-PCIe
> > links, that's a problem because the PCI core code that manages MPS and
> > MRRS has no knowledge of those non-PCIe parts of the system.
>
> Yes and no, it only affects PCIe in P2P, in non-P2P is will certainly affect
> transfers on the internal SoC/Processor/Chip internal bus/fabric.
>
> > You might be able to deal with this in Synopsys-specific code somehow,
> > but it's going to be a bit of a hassle because I don't want it to make
> > maintenance of the generic MPS/MRRS code harder.
>
> I understand, but this is why these quirks are currently implemented in the
> controller driver and only applies when the controller has been probed
> and to each endpoint detected on this particular controller.
>
> So we may continue having separate quirks for each controller if the core
> isn't the right place to handle MPS/MRRS.
The PCI core is the correct place to handle MPS/MRRS because their
behavior is defined by the PCIe spec.
Quirks are the way to work around this defect in the Synopsys PCIe IP.
Bjorn
On Wed, Jul 07, 2021 at 11:57:35AM -0500, Bjorn Helgaas wrote:
> On Wed, Jul 07, 2021 at 06:43:13PM +0200, Neil Armstrong wrote:
> > On 07/07/2021 17:54, Bjorn Helgaas wrote:
> > > On Tue, Jul 06, 2021 at 11:54:05AM +0200, Neil Armstrong wrote:
> > >> In their Designware PCIe controller driver, amlogic sets the
> > >> Max_Payload_Size & Max_Read_Request_Size to 256:
> > >> https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L260
> > >> https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L276
> > >> in their root port PCIe Express Device Control Register.
> > >>
> > >> Looking at the Synopsys DW-PCIe Databook, Max_Payload_Size &
> > >> Max_Read_Request_Size are used to decompose into AXI burst, but it
> > >> seems the Max_Payload_Size & Max_Read_Request_Size are set by
> > >> default to 512 but the internal Max_Payload_Size_Supported is set to
> > >> 256, thus changing these values to 256 at runtime to match and
> > >> optimize bandwidth.
> > >>
> > >> It's said, "Reducing Outbound Decomposition" :
> > >> - "Ensure that your application master does not generate bursts of
> > >> size greater than or equal to Max_Payload_Size"
> > >>
> > >> - "Program your PCIe system with a larger value of Max_Payload_Size
> > >> without exceeding Max_Payload_Size_Supported"
> > >>
> > >> - "Program your PCIe system with a larger value of Max_Read_Request
> > >> without exceeding Max_Payload_Size_Supported:
> > >>
> > >> So leaving 512 in Max_Payload_Size & Max_Read_Request leads to
> > >> Outbound Decomposition which decreases PCIe link and degrades the
> > >> AXI bus by doubling the bursts, leading to this fix to avoid
> > >> overflowing the AXI bus.
> > >>
> > >> So it seems to be still needed, I assume this *should* be handled in
> > >> the core somehow to propagate these settings to child endpoints to
> > >> match the root port Max_Payload_Size & Max_Read_Request sizes.
> > >>
> > >> Maybe by adding a core function to set these values instead of using
> > >> the dw_pcie_find_capability() & dw_pcie_write/readl_dbi() helpers
> > >> and set a state on the root port to propagate the value ?
> > >
> > > I don't have the Synopsys DW-PCIe Databook, so I'm lacking any
> > > context. The above *seems* to say that MPS/MRRS settings affect AXI
> > > bus usage.
> >
> > It does when the TLPs are directed to the RC.
>
> That's a defect in the RC.
I mean, it's OK if MPS affects the *performance* of traffic on AXI,
but the RC must work correctly for any MPS up to the MPSS (Max Payload
Size Supported) it advertises. And there's no PCIe mechanism for
managing the AXI performance impact, so I think it's a mistake if
Synopsys expects device-specific code in the MPS/MRRS configuration
path. That code should be device-independent.
> > > The MPS and MRRS registers are defined to affect traffic on *PCIe*. If
> > > a platform uses MPS and MRRS values to optimize transfers on non-PCIe
> > > links, that's a problem because the PCI core code that manages MPS and
> > > MRRS has no knowledge of those non-PCIe parts of the system.
> >
> > Yes and no, it only affects PCIe in P2P, in non-P2P is will certainly affect
> > transfers on the internal SoC/Processor/Chip internal bus/fabric.
> >
> > > You might be able to deal with this in Synopsys-specific code somehow,
> > > but it's going to be a bit of a hassle because I don't want it to make
> > > maintenance of the generic MPS/MRRS code harder.
> >
> > I understand, but this is why these quirks are currently implemented in the
> > controller driver and only applies when the controller has been probed
> > and to each endpoint detected on this particular controller.
> >
> > So we may continue having separate quirks for each controller if the core
> > isn't the right place to handle MPS/MRRS.
>
> The PCI core is the correct place to handle MPS/MRRS because their
> behavior is defined by the PCIe spec.
>
> Quirks are the way to work around this defect in the Synopsys PCIe IP.
>
> Bjorn
> The PCI core is the correct place to handle MPS/MRRS because their> behavior is defined by the PCIe spec.
>Quirks are the way to work around this defect in the Synopsys PCIe IP.
> don't want it to make
> maintenance of the generic MPS/MRRS code harder.
Trying summarize ( every one must use separate quirk )
which file is right place for for meson_mrrs_limit_quirk()
- pci/controller/dwc/pci-meson.c or
- pci/quirks.c
rewrited quirk just for meson:
static void meson_mrrs_limit_quirk(struct pci_dev *dev)
{
struct pci_bus *bus = dev->bus;
int mrrs, mrrs_limit = 256;
static const struct pci_device_id bridge_devids[] = {
{ PCI_DEVICE(PCI_VENDOR_ID_SYNOPSYS, PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3) },
{ 0, },
};
/* look for the matching bridge */
while (!pci_is_root_bus(bus)) {
/*
* 256 bytes maximum read request size. They can't handle
* anything larger than this. So force this limit on
* any devices attached under these ports.
*/
if (!pci_match_id(bridge_devids, bus->self)){
bus = bus->parent;
continue;
}
mrrs = pcie_get_readrq(dev);
if (mrrs > mrrs_limit) {
pci_info(dev, "limiting MRRS %d to %d\n", mrrs, mrrs_limit);
pcie_set_readrq(dev, mrrs_limit);
}
break;
}
}
DECLARE_PCI_FIXUP_ENABLE(PCI_ANY_ID, PCI_ANY_ID, meson_mrrs_limit_quirk);
On Thu, Jul 8, 2021 at 12:57 AM Bjorn Helgaas <[email protected]> wrote:
>
> On Wed, Jul 07, 2021 at 06:43:13PM +0200, Neil Armstrong wrote:
> > On 07/07/2021 17:54, Bjorn Helgaas wrote:
> > > On Tue, Jul 06, 2021 at 11:54:05AM +0200, Neil Armstrong wrote:
> > >> In their Designware PCIe controller driver, amlogic sets the
> > >> Max_Payload_Size & Max_Read_Request_Size to 256:
> > >> https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L260
> > >> https://elixir.bootlin.com/linux/latest/source/drivers/pci/controller/dwc/pci-meson.c#L276
> > >> in their root port PCIe Express Device Control Register.
> > >>
> > >> Looking at the Synopsys DW-PCIe Databook, Max_Payload_Size &
> > >> Max_Read_Request_Size are used to decompose into AXI burst, but it
> > >> seems the Max_Payload_Size & Max_Read_Request_Size are set by
> > >> default to 512 but the internal Max_Payload_Size_Supported is set to
> > >> 256, thus changing these values to 256 at runtime to match and
> > >> optimize bandwidth.
> > >>
> > >> It's said, "Reducing Outbound Decomposition" :
> > >> - "Ensure that your application master does not generate bursts of
> > >> size greater than or equal to Max_Payload_Size"
> > >>
> > >> - "Program your PCIe system with a larger value of Max_Payload_Size
> > >> without exceeding Max_Payload_Size_Supported"
> > >>
> > >> - "Program your PCIe system with a larger value of Max_Read_Request
> > >> without exceeding Max_Payload_Size_Supported:
> > >>
> > >> So leaving 512 in Max_Payload_Size & Max_Read_Request leads to
> > >> Outbound Decomposition which decreases PCIe link and degrades the
> > >> AXI bus by doubling the bursts, leading to this fix to avoid
> > >> overflowing the AXI bus.
> > >>
> > >> So it seems to be still needed, I assume this *should* be handled in
> > >> the core somehow to propagate these settings to child endpoints to
> > >> match the root port Max_Payload_Size & Max_Read_Request sizes.
> > >>
> > >> Maybe by adding a core function to set these values instead of using
> > >> the dw_pcie_find_capability() & dw_pcie_write/readl_dbi() helpers
> > >> and set a state on the root port to propagate the value ?
> > >
> > > I don't have the Synopsys DW-PCIe Databook, so I'm lacking any
> > > context. The above *seems* to say that MPS/MRRS settings affect AXI
> > > bus usage.
> >
> > It does when the TLPs are directed to the RC.
>
> That's a defect in the RC.
>
> > > The MPS and MRRS registers are defined to affect traffic on *PCIe*. If
> > > a platform uses MPS and MRRS values to optimize transfers on non-PCIe
> > > links, that's a problem because the PCI core code that manages MPS and
> > > MRRS has no knowledge of those non-PCIe parts of the system.
> >
> > Yes and no, it only affects PCIe in P2P, in non-P2P is will certainly affect
> > transfers on the internal SoC/Processor/Chip internal bus/fabric.
> >
> > > You might be able to deal with this in Synopsys-specific code somehow,
> > > but it's going to be a bit of a hassle because I don't want it to make
> > > maintenance of the generic MPS/MRRS code harder.
> >
> > I understand, but this is why these quirks are currently implemented in the
> > controller driver and only applies when the controller has been probed
> > and to each endpoint detected on this particular controller.
> >
> > So we may continue having separate quirks for each controller if the core
> > isn't the right place to handle MPS/MRRS.
>
> The PCI core is the correct place to handle MPS/MRRS because their
> behavior is defined by the PCIe spec.
>
> Quirks are the way to work around this defect in the Synopsys PCIe IP.
>
> Bjorn