Hello,
During the review of the patch that fixes DBI access in PCI EP, Rob
suggested [1] using a fixed interface for passing the events from EPC to
EPF instead of the in-kernel notifiers.
This series introduces a simple callback based mechanism for passing the
events from EPC to EPF. This interface is chosen for satisfying the below
requirements:
1. The notification has to reach the EPF drivers without any additional
latency.
2. The context of the caller (EPC) needs to be preserved while passing the
notifications.
With the existing notifier mechanism, the 1st case can be satisfied since
notifiers aren't adding any huge overhead. But the 2nd case is clearly not
satisfied, because the current atomic notifiers forces the EPF
notification context to be atomic even though the caller (EPC) may not be
in atomic context. In the notification function, the EPF drivers are
required to call several EPC APIs that might sleep and this triggers a
sleeping in atomic bug during runtime.
The above issue could be fixed by using a blocking notifier instead of
atomic, but that proposal was not accepted either [2].
So instead of working around the issues within the notifiers, let's get rid
of it and use the callback mechanism.
NOTE: DRA7xx and TEGRA194 drivers are only compile tested. Testing this series
on the real platforms is greatly appreciated.
Thanks,
Mani
[1] https://lore.kernel.org/all/20220802072426.GA2494@thinkpad/T/#mfa3a5b3a9694798a562c36b228f595b6a571477d
[2] https://lore.kernel.org/all/[email protected]
Changes in v4:
* Added check for the presence of event_ops before involing the callbacks (Kishon)
* Added return with IRQ_WAKE_THREAD when link_up event is found in the hard irq
handler of tegra194 driver (Vidya)
* Collected review tags
Changes in v3:
* As Kishon spotted, fixed the DRA7xx driver and also the TEGRA194 driver to
call the LINK_UP callback in threaded IRQ handler.
Changes in v2:
* Introduced a new "list_lock" for protecting the epc->pci_epf list and
used it in the callback mechanism.
Manivannan Sadhasivam (5):
PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
PCI: endpoint: Use callback mechanism for passing events from EPC to
EPF
PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
drivers/pci/controller/dwc/pci-dra7xx.c | 2 +-
drivers/pci/controller/dwc/pcie-tegra194.c | 9 ++++-
drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
drivers/pci/endpoint/pci-epc-core.c | 32 ++++++++++++----
include/linux/pci-epc.h | 10 +----
include/linux/pci-epf.h | 19 ++++++----
6 files changed, 59 insertions(+), 51 deletions(-)
--
2.25.1
Hi Kishon,
On Tue, Oct 25, 2022 at 08:20:56PM +0530, Manivannan Sadhasivam wrote:
> Hello,
>
> During the review of the patch that fixes DBI access in PCI EP, Rob
> suggested [1] using a fixed interface for passing the events from EPC to
> EPF instead of the in-kernel notifiers.
>
> This series introduces a simple callback based mechanism for passing the
> events from EPC to EPF. This interface is chosen for satisfying the below
> requirements:
>
> 1. The notification has to reach the EPF drivers without any additional
> latency.
> 2. The context of the caller (EPC) needs to be preserved while passing the
> notifications.
>
> With the existing notifier mechanism, the 1st case can be satisfied since
> notifiers aren't adding any huge overhead. But the 2nd case is clearly not
> satisfied, because the current atomic notifiers forces the EPF
> notification context to be atomic even though the caller (EPC) may not be
> in atomic context. In the notification function, the EPF drivers are
> required to call several EPC APIs that might sleep and this triggers a
> sleeping in atomic bug during runtime.
>
> The above issue could be fixed by using a blocking notifier instead of
> atomic, but that proposal was not accepted either [2].
>
> So instead of working around the issues within the notifiers, let's get rid
> of it and use the callback mechanism.
>
> NOTE: DRA7xx and TEGRA194 drivers are only compile tested. Testing this series
> on the real platforms is greatly appreciated.
>
Any feedback on this series?
Thanks,
Mani
> Thanks,
> Mani
>
> [1] https://lore.kernel.org/all/20220802072426.GA2494@thinkpad/T/#mfa3a5b3a9694798a562c36b228f595b6a571477d
> [2] https://lore.kernel.org/all/[email protected]
>
> Changes in v4:
>
> * Added check for the presence of event_ops before involing the callbacks (Kishon)
> * Added return with IRQ_WAKE_THREAD when link_up event is found in the hard irq
> handler of tegra194 driver (Vidya)
> * Collected review tags
>
> Changes in v3:
>
> * As Kishon spotted, fixed the DRA7xx driver and also the TEGRA194 driver to
> call the LINK_UP callback in threaded IRQ handler.
>
> Changes in v2:
>
> * Introduced a new "list_lock" for protecting the epc->pci_epf list and
> used it in the callback mechanism.
>
> Manivannan Sadhasivam (5):
> PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
> PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
> PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
> PCI: endpoint: Use callback mechanism for passing events from EPC to
> EPF
> PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
>
> drivers/pci/controller/dwc/pci-dra7xx.c | 2 +-
> drivers/pci/controller/dwc/pcie-tegra194.c | 9 ++++-
> drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
> drivers/pci/endpoint/pci-epc-core.c | 32 ++++++++++++----
> include/linux/pci-epc.h | 10 +----
> include/linux/pci-epf.h | 19 ++++++----
> 6 files changed, 59 insertions(+), 51 deletions(-)
>
> --
> 2.25.1
>
--
மணிவண்ணன் சதாசிவம்
On Tue, Oct 25, 2022 at 08:20:56PM +0530, Manivannan Sadhasivam wrote:
> Hello,
>
> During the review of the patch that fixes DBI access in PCI EP, Rob
> suggested [1] using a fixed interface for passing the events from EPC to
> EPF instead of the in-kernel notifiers.
> Manivannan Sadhasivam (5):
> PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
> PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
> PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
> PCI: endpoint: Use callback mechanism for passing events from EPC to
> EPF
> PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
>
> drivers/pci/controller/dwc/pci-dra7xx.c | 2 +-
> drivers/pci/controller/dwc/pcie-tegra194.c | 9 ++++-
> drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
> drivers/pci/endpoint/pci-epc-core.c | 32 ++++++++++++----
> include/linux/pci-epc.h | 10 +----
> include/linux/pci-epf.h | 19 ++++++----
> 6 files changed, 59 insertions(+), 51 deletions(-)
Doesn't apply cleanly on v6.1-rc1. Does it depend on something else?
On Mon, Nov 07, 2022 at 02:28:53PM -0600, Bjorn Helgaas wrote:
> On Tue, Oct 25, 2022 at 08:20:56PM +0530, Manivannan Sadhasivam wrote:
> > Hello,
> >
> > During the review of the patch that fixes DBI access in PCI EP, Rob
> > suggested [1] using a fixed interface for passing the events from EPC to
> > EPF instead of the in-kernel notifiers.
>
> > Manivannan Sadhasivam (5):
> > PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
> > PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
> > PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
> > PCI: endpoint: Use callback mechanism for passing events from EPC to
> > EPF
> > PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
> >
> > drivers/pci/controller/dwc/pci-dra7xx.c | 2 +-
> > drivers/pci/controller/dwc/pcie-tegra194.c | 9 ++++-
> > drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
> > drivers/pci/endpoint/pci-epc-core.c | 32 ++++++++++++----
> > include/linux/pci-epc.h | 10 +----
> > include/linux/pci-epf.h | 19 ++++++----
> > 6 files changed, 59 insertions(+), 51 deletions(-)
>
> Doesn't apply cleanly on v6.1-rc1. Does it depend on something else?
Yes, this patch:
https://lore.kernel.org/linux-pci/[email protected]/
Since this patch is already merged by Lorenzo, I based this series on top of
that. If that's not required, I can send a new version without that patch.
Let me know.
Thanks,
Mani
--
மணிவண்ணன் சதாசிவம்
On Tue, Nov 08, 2022 at 05:44:40PM +0530, Manivannan Sadhasivam wrote:
> On Mon, Nov 07, 2022 at 02:28:53PM -0600, Bjorn Helgaas wrote:
> > On Tue, Oct 25, 2022 at 08:20:56PM +0530, Manivannan Sadhasivam wrote:
> > > Hello,
> > >
> > > During the review of the patch that fixes DBI access in PCI EP, Rob
> > > suggested [1] using a fixed interface for passing the events from EPC to
> > > EPF instead of the in-kernel notifiers.
> >
> > > Manivannan Sadhasivam (5):
> > > PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
> > > PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
> > > PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
> > > PCI: endpoint: Use callback mechanism for passing events from EPC to
> > > EPF
> > > PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
> > >
> > > drivers/pci/controller/dwc/pci-dra7xx.c | 2 +-
> > > drivers/pci/controller/dwc/pcie-tegra194.c | 9 ++++-
> > > drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
> > > drivers/pci/endpoint/pci-epc-core.c | 32 ++++++++++++----
> > > include/linux/pci-epc.h | 10 +----
> > > include/linux/pci-epf.h | 19 ++++++----
> > > 6 files changed, 59 insertions(+), 51 deletions(-)
> >
> > Doesn't apply cleanly on v6.1-rc1. Does it depend on something else?
>
> Yes, this patch:
> https://lore.kernel.org/linux-pci/[email protected]/
>
> Since this patch is already merged by Lorenzo, I based this series on top of
> that. If that's not required, I can send a new version without that patch.
I think it's fine as-is.
I tried applying it on both v6.1-rc1 and my current "next" branch.
Both failed because I haven't merged Lorenzo's branch into "next" yet.
As long as Lorenzo merges this on the correct branch, there's no
problem.
Mentioning the dependency or what the patch is based on in the cover
letter is the easiest way to make this smoother.
Bjorn
On Tue, Oct 25, 2022 at 08:20:56PM +0530, Manivannan Sadhasivam wrote:
> Hello,
>
> During the review of the patch that fixes DBI access in PCI EP, Rob
> suggested [1] using a fixed interface for passing the events from EPC to
> EPF instead of the in-kernel notifiers.
>
> This series introduces a simple callback based mechanism for passing the
> events from EPC to EPF. This interface is chosen for satisfying the below
> requirements:
>
> 1. The notification has to reach the EPF drivers without any additional
> latency.
> 2. The context of the caller (EPC) needs to be preserved while passing the
> notifications.
>
> With the existing notifier mechanism, the 1st case can be satisfied since
> notifiers aren't adding any huge overhead. But the 2nd case is clearly not
> satisfied, because the current atomic notifiers forces the EPF
> notification context to be atomic even though the caller (EPC) may not be
> in atomic context. In the notification function, the EPF drivers are
> required to call several EPC APIs that might sleep and this triggers a
> sleeping in atomic bug during runtime.
>
> The above issue could be fixed by using a blocking notifier instead of
> atomic, but that proposal was not accepted either [2].
>
> So instead of working around the issues within the notifiers, let's get rid
> of it and use the callback mechanism.
>
> NOTE: DRA7xx and TEGRA194 drivers are only compile tested. Testing this series
> on the real platforms is greatly appreciated.
>
Lorenzo, can this series be merged for v6.2 since all the patches are reviewed
now?
Thanks,
Mani
> Thanks,
> Mani
>
> [1] https://lore.kernel.org/all/20220802072426.GA2494@thinkpad/T/#mfa3a5b3a9694798a562c36b228f595b6a571477d
> [2] https://lore.kernel.org/all/[email protected]
>
> Changes in v4:
>
> * Added check for the presence of event_ops before involing the callbacks (Kishon)
> * Added return with IRQ_WAKE_THREAD when link_up event is found in the hard irq
> handler of tegra194 driver (Vidya)
> * Collected review tags
>
> Changes in v3:
>
> * As Kishon spotted, fixed the DRA7xx driver and also the TEGRA194 driver to
> call the LINK_UP callback in threaded IRQ handler.
>
> Changes in v2:
>
> * Introduced a new "list_lock" for protecting the epc->pci_epf list and
> used it in the callback mechanism.
>
> Manivannan Sadhasivam (5):
> PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
> PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
> PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
> PCI: endpoint: Use callback mechanism for passing events from EPC to
> EPF
> PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
>
> drivers/pci/controller/dwc/pci-dra7xx.c | 2 +-
> drivers/pci/controller/dwc/pcie-tegra194.c | 9 ++++-
> drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
> drivers/pci/endpoint/pci-epc-core.c | 32 ++++++++++++----
> include/linux/pci-epc.h | 10 +----
> include/linux/pci-epf.h | 19 ++++++----
> 6 files changed, 59 insertions(+), 51 deletions(-)
>
> --
> 2.25.1
>
--
மணிவண்ணன் சதாசிவம்
On Mon, Nov 14, 2022 at 01:03:16PM +0530, Manivannan Sadhasivam wrote:
> On Tue, Oct 25, 2022 at 08:20:56PM +0530, Manivannan Sadhasivam wrote:
> > Hello,
> >
> > During the review of the patch that fixes DBI access in PCI EP, Rob
> > suggested [1] using a fixed interface for passing the events from EPC to
> > EPF instead of the in-kernel notifiers.
> >
> > This series introduces a simple callback based mechanism for passing the
> > events from EPC to EPF. This interface is chosen for satisfying the below
> > requirements:
> >
> > 1. The notification has to reach the EPF drivers without any additional
> > latency.
> > 2. The context of the caller (EPC) needs to be preserved while passing the
> > notifications.
> >
> > With the existing notifier mechanism, the 1st case can be satisfied since
> > notifiers aren't adding any huge overhead. But the 2nd case is clearly not
> > satisfied, because the current atomic notifiers forces the EPF
> > notification context to be atomic even though the caller (EPC) may not be
> > in atomic context. In the notification function, the EPF drivers are
> > required to call several EPC APIs that might sleep and this triggers a
> > sleeping in atomic bug during runtime.
> >
> > The above issue could be fixed by using a blocking notifier instead of
> > atomic, but that proposal was not accepted either [2].
> >
> > So instead of working around the issues within the notifiers, let's get rid
> > of it and use the callback mechanism.
> >
> > NOTE: DRA7xx and TEGRA194 drivers are only compile tested. Testing this series
> > on the real platforms is greatly appreciated.
> >
>
> Lorenzo, can this series be merged for v6.2 since all the patches are reviewed
> now?
Patch (2) isn't (or I missed something) - we should be looking for
review/testing on it.
Thanks,
Lorenzo
> Thanks,
> Mani
>
> > Thanks,
> > Mani
> >
> > [1] https://lore.kernel.org/all/20220802072426.GA2494@thinkpad/T/#mfa3a5b3a9694798a562c36b228f595b6a571477d
> > [2] https://lore.kernel.org/all/[email protected]
> >
> > Changes in v4:
> >
> > * Added check for the presence of event_ops before involing the callbacks (Kishon)
> > * Added return with IRQ_WAKE_THREAD when link_up event is found in the hard irq
> > handler of tegra194 driver (Vidya)
> > * Collected review tags
> >
> > Changes in v3:
> >
> > * As Kishon spotted, fixed the DRA7xx driver and also the TEGRA194 driver to
> > call the LINK_UP callback in threaded IRQ handler.
> >
> > Changes in v2:
> >
> > * Introduced a new "list_lock" for protecting the epc->pci_epf list and
> > used it in the callback mechanism.
> >
> > Manivannan Sadhasivam (5):
> > PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
> > PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
> > PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
> > PCI: endpoint: Use callback mechanism for passing events from EPC to
> > EPF
> > PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
> >
> > drivers/pci/controller/dwc/pci-dra7xx.c | 2 +-
> > drivers/pci/controller/dwc/pcie-tegra194.c | 9 ++++-
> > drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
> > drivers/pci/endpoint/pci-epc-core.c | 32 ++++++++++++----
> > include/linux/pci-epc.h | 10 +----
> > include/linux/pci-epf.h | 19 ++++++----
> > 6 files changed, 59 insertions(+), 51 deletions(-)
> >
> > --
> > 2.25.1
> >
>
> --
> மணிவண்ணன் சதாசிவம்
On Mon, Nov 14, 2022 at 11:05:44AM +0100, Lorenzo Pieralisi wrote:
> On Mon, Nov 14, 2022 at 01:03:16PM +0530, Manivannan Sadhasivam wrote:
> > On Tue, Oct 25, 2022 at 08:20:56PM +0530, Manivannan Sadhasivam wrote:
> > > Hello,
> > >
> > > During the review of the patch that fixes DBI access in PCI EP, Rob
> > > suggested [1] using a fixed interface for passing the events from EPC to
> > > EPF instead of the in-kernel notifiers.
> > >
> > > This series introduces a simple callback based mechanism for passing the
> > > events from EPC to EPF. This interface is chosen for satisfying the below
> > > requirements:
> > >
> > > 1. The notification has to reach the EPF drivers without any additional
> > > latency.
> > > 2. The context of the caller (EPC) needs to be preserved while passing the
> > > notifications.
> > >
> > > With the existing notifier mechanism, the 1st case can be satisfied since
> > > notifiers aren't adding any huge overhead. But the 2nd case is clearly not
> > > satisfied, because the current atomic notifiers forces the EPF
> > > notification context to be atomic even though the caller (EPC) may not be
> > > in atomic context. In the notification function, the EPF drivers are
> > > required to call several EPC APIs that might sleep and this triggers a
> > > sleeping in atomic bug during runtime.
> > >
> > > The above issue could be fixed by using a blocking notifier instead of
> > > atomic, but that proposal was not accepted either [2].
> > >
> > > So instead of working around the issues within the notifiers, let's get rid
> > > of it and use the callback mechanism.
> > >
> > > NOTE: DRA7xx and TEGRA194 drivers are only compile tested. Testing this series
> > > on the real platforms is greatly appreciated.
> > >
> >
> > Lorenzo, can this series be merged for v6.2 since all the patches are reviewed
> > now?
>
> Patch (2) isn't (or I missed something) - we should be looking for
> review/testing on it.
>
Yes, 2/5 doesn't have a review tag yet. But as per comments from Vidya on v3
[1], I believe it is okay to get merged.
But I'll ping on that patch anyway.
Thanks,
Mani
[1] https://lore.kernel.org/lkml/[email protected]/
> Thanks,
> Lorenzo
>
> > Thanks,
> > Mani
> >
> > > Thanks,
> > > Mani
> > >
> > > [1] https://lore.kernel.org/all/20220802072426.GA2494@thinkpad/T/#mfa3a5b3a9694798a562c36b228f595b6a571477d
> > > [2] https://lore.kernel.org/all/[email protected]
> > >
> > > Changes in v4:
> > >
> > > * Added check for the presence of event_ops before involing the callbacks (Kishon)
> > > * Added return with IRQ_WAKE_THREAD when link_up event is found in the hard irq
> > > handler of tegra194 driver (Vidya)
> > > * Collected review tags
> > >
> > > Changes in v3:
> > >
> > > * As Kishon spotted, fixed the DRA7xx driver and also the TEGRA194 driver to
> > > call the LINK_UP callback in threaded IRQ handler.
> > >
> > > Changes in v2:
> > >
> > > * Introduced a new "list_lock" for protecting the epc->pci_epf list and
> > > used it in the callback mechanism.
> > >
> > > Manivannan Sadhasivam (5):
> > > PCI: dra7xx: Use threaded IRQ handler for "dra7xx-pcie-main" IRQ
> > > PCI: tegra194: Move dw_pcie_ep_linkup() to threaded IRQ handler
> > > PCI: endpoint: Use a separate lock for protecting epc->pci_epf list
> > > PCI: endpoint: Use callback mechanism for passing events from EPC to
> > > EPF
> > > PCI: endpoint: Use link_up() callback in place of LINK_UP notifier
> > >
> > > drivers/pci/controller/dwc/pci-dra7xx.c | 2 +-
> > > drivers/pci/controller/dwc/pcie-tegra194.c | 9 ++++-
> > > drivers/pci/endpoint/functions/pci-epf-test.c | 38 ++++++-------------
> > > drivers/pci/endpoint/pci-epc-core.c | 32 ++++++++++++----
> > > include/linux/pci-epc.h | 10 +----
> > > include/linux/pci-epf.h | 19 ++++++----
> > > 6 files changed, 59 insertions(+), 51 deletions(-)
> > >
> > > --
> > > 2.25.1
> > >
> >
> > --
> > மணிவண்ணன் சதாசிவம்
--
மணிவண்ணன் சதாசிவம்