Hi,
I was experimenting with a TotalCompute2022 (TC2) [1] reference design
sporting MHUv2 (v2.x) mailboxes and running on top of it an SCMI stack
configured platform-side to use multiple per-protocol channels with
multiple rx interrupts but NOT using a combined interrupt.
(it is not even wired at the hw-level on this platform anyway...)
This configuration seemed not supported by the current driver which
assumes that you have only one rx interrupt, possibly a combined one.
Looking at the MHUv2 spec ([2]/[3] "Channels") seems indeed that
per-channel rx-interrupts are indeed a possibility, beside the combined
one.
This small seriers aims to add such support by simply allowing to
specify multiple rx interrupts in the DT node and taking care to
register the same orginal ISR for all.
On the tx path, instead, I left things unchanged and stuck with the
current approach that just uses one single combined interrupt for the
clear channel interrupt. (if supported at all)
I tested on a TC2 model configured alternatively with MHU v2.1 and v2.0.
Series based on v6.3-rc4.
Any feedback welcome.
Thanks,
Cristian
----
[1]: https://gitlab.arm.com/arm-reference-solutions/arm-reference-solutions-docs/-/blob/master/docs/totalcompute/tc2/tc2_sw_stack.rst
[2]: https://developer.arm.com/documentation/102342/0000/Message-Handling-Unit/About-the-Message-Handling-Unit/Channels?lang=en
[3]: https://developer.arm.com/documentation/101418/0100/Message-Handling-Unit/Message-Handling-Unit-v2/Channel?lang=en
Cristian Marussi (2):
dt-bindings: mailbox : arm,mhuv2: Allow for more RX interrupts
mailbox: arm_mhuv2: Add support for multiple rx interrupt
.../bindings/mailbox/arm,mhuv2.yaml | 13 +++++---
drivers/mailbox/arm_mhuv2.c | 30 ++++++++++---------
2 files changed, 25 insertions(+), 18 deletions(-)
--
2.34.1
ARM MHUv2 can be configured to receive multiple interrupt related to the
receiver block, up to the maximum number of available channels, and not
necessarily grouped into a single combined interrupt.
Allow to register more interrupt for the RX block up to the maximum number
of interrupts supported by an AMBA device.
Signed-off-by: Cristian Marussi <[email protected]>
---
drivers/mailbox/arm_mhuv2.c | 30 ++++++++++++++++--------------
1 file changed, 16 insertions(+), 14 deletions(-)
diff --git a/drivers/mailbox/arm_mhuv2.c b/drivers/mailbox/arm_mhuv2.c
index c6d4957c4da8..89060bee1fb0 100644
--- a/drivers/mailbox/arm_mhuv2.c
+++ b/drivers/mailbox/arm_mhuv2.c
@@ -163,7 +163,6 @@ enum mhuv2_frame {
* @send: Base address of the register mapping region.
* @recv: Base address of the register mapping region.
* @frame: Frame type: RECEIVER_FRAME or SENDER_FRAME.
- * @irq: Interrupt.
* @windows: Channel windows implemented by the platform.
* @minor: Minor version of the controller.
* @length: Length of the protocols array in bytes.
@@ -178,7 +177,6 @@ struct mhuv2 {
struct mhu2_recv_frame_reg __iomem *recv;
};
enum mhuv2_frame frame;
- unsigned int irq;
unsigned int windows;
unsigned int minor;
unsigned int length;
@@ -991,7 +989,6 @@ static int mhuv2_tx_init(struct amba_device *adev, struct mhuv2 *mhu,
} else {
mhu->mbox.txdone_irq = true;
mhu->mbox.txdone_poll = false;
- mhu->irq = adev->irq[0];
writel_relaxed_bitfield(1, &mhu->send->int_en, struct int_en_t, chcomb);
@@ -1029,18 +1026,23 @@ static int mhuv2_rx_init(struct amba_device *adev, struct mhuv2 *mhu,
mhu->windows = readl_relaxed_bitfield(&mhu->recv->mhu_cfg, struct mhu_cfg_t, num_ch);
mhu->minor = readl_relaxed_bitfield(&mhu->recv->aidr, struct aidr_t, arch_minor_rev);
- mhu->irq = adev->irq[0];
- if (!mhu->irq) {
- dev_err(dev, "Missing receiver IRQ\n");
- return -EINVAL;
- }
+ for (i = 0; i < min_t(unsigned int, mhu->windows, AMBA_NR_IRQS); i++) {
+ if (!adev->irq[i]) {
+ /* At least one receiver IRQ is needed */
+ if (i == 0) {
+ dev_err(dev, "Missing receiver IRQ\n");
+ return -EINVAL;
+ }
+ continue;
+ }
- ret = devm_request_threaded_irq(dev, mhu->irq, NULL,
- mhuv2_receiver_interrupt, IRQF_ONESHOT,
- "mhuv2-rx", mhu);
- if (ret) {
- dev_err(dev, "Failed to request rx IRQ\n");
- return ret;
+ ret = devm_request_threaded_irq(dev, adev->irq[i], NULL,
+ mhuv2_receiver_interrupt, IRQF_ONESHOT,
+ "mhuv2-rx", mhu);
+ if (ret) {
+ dev_err(dev, "Failed to request rx IRQ\n");
+ return ret;
+ }
}
/* Mask all the channel windows */
--
2.34.1
The ARM MHUv2 Receiver block can indeed support more interrupts, up to the
maximum number of available channels, but anyway no more than the maximum
number of supported interrupt for an AMBA device.
Signed-off-by: Cristian Marussi <[email protected]>
---
Cc: Rob Herring <[email protected]>
Cc: Krzysztof Kozlowski <[email protected]>
Cc: [email protected]
.../devicetree/bindings/mailbox/arm,mhuv2.yaml | 13 +++++++++----
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml b/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
index a4f1fe63659a..5a57f4e2a623 100644
--- a/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
+++ b/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
@@ -69,10 +69,15 @@ properties:
interrupts:
description: |
- The MHUv2 controller always implements an interrupt in the "receiver"
- mode, while the interrupt in the "sender" mode was not available in the
- version MHUv2.0, but the later versions do have it.
- maxItems: 1
+ The MHUv2 controller always implements at least an interrupt in the
+ "receiver" mode, while the interrupt in the "sender" mode was not
+ available in the version MHUv2.0, but the later versions do have it.
+ In "receiver" mode, beside a single combined interrupt, there could be
+ multiple interrupts, up to the number of implemented channels but anyway
+ no more than the maximum number of interrupts potentially supported by
+ AMBA.
+ minItems: 1
+ maxItems: 9
clocks:
maxItems: 1
--
2.34.1
On Wed, Mar 29, 2023 at 04:39:35PM +0100, Cristian Marussi wrote:
> The ARM MHUv2 Receiver block can indeed support more interrupts, up to the
> maximum number of available channels, but anyway no more than the maximum
> number of supported interrupt for an AMBA device.
>
> Signed-off-by: Cristian Marussi <[email protected]>
> ---
> Cc: Rob Herring <[email protected]>
> Cc: Krzysztof Kozlowski <[email protected]>
> Cc: [email protected]
>
> .../devicetree/bindings/mailbox/arm,mhuv2.yaml | 13 +++++++++----
> 1 file changed, 9 insertions(+), 4 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml b/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> index a4f1fe63659a..5a57f4e2a623 100644
> --- a/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> +++ b/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> @@ -69,10 +69,15 @@ properties:
>
> interrupts:
> description: |
> - The MHUv2 controller always implements an interrupt in the "receiver"
> - mode, while the interrupt in the "sender" mode was not available in the
> - version MHUv2.0, but the later versions do have it.
> - maxItems: 1
> + The MHUv2 controller always implements at least an interrupt in the
> + "receiver" mode, while the interrupt in the "sender" mode was not
> + available in the version MHUv2.0, but the later versions do have it.
> + In "receiver" mode, beside a single combined interrupt, there could be
> + multiple interrupts, up to the number of implemented channels but anyway
> + no more than the maximum number of interrupts potentially supported by
> + AMBA.
> + minItems: 1
> + maxItems: 9
I am not sure 9 is the correct value here. IIUC it is just what Linux defines
as AMBA_NR_IRQS. Looking at the history it was bumped from 2 to 9 for use
by PL330 DMA driver. I couldn't find anything to relate this 9 in any
AMBA or other related specification.
Ideally I would say we don't know what the max here. We just have a platform
implementing 2 interrupts now. Do we for with 2 for now and change it if some
new users require more in the future ?
I will leave that to the DT maintainers but 9 is simply random based on Linux
code so I would rather choose some other random number with a better reasoning
than 9 as AMBA code in the kernel is limiting it to 9.
--
Regards,
Sudeep
On 29/03/2023 17:39, Cristian Marussi wrote:
> The ARM MHUv2 Receiver block can indeed support more interrupts, up to the
> maximum number of available channels, but anyway no more than the maximum
> number of supported interrupt for an AMBA device.
>
> Signed-off-by: Cristian Marussi <[email protected]>
> ---
> Cc: Rob Herring <[email protected]>
> Cc: Krzysztof Kozlowski <[email protected]>
> Cc: [email protected]
>
> .../devicetree/bindings/mailbox/arm,mhuv2.yaml | 13 +++++++++----
> 1 file changed, 9 insertions(+), 4 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml b/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> index a4f1fe63659a..5a57f4e2a623 100644
> --- a/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> +++ b/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> @@ -69,10 +69,15 @@ properties:
>
> interrupts:
> description: |
> - The MHUv2 controller always implements an interrupt in the "receiver"
> - mode, while the interrupt in the "sender" mode was not available in the
> - version MHUv2.0, but the later versions do have it.
> - maxItems: 1
> + The MHUv2 controller always implements at least an interrupt in the
> + "receiver" mode, while the interrupt in the "sender" mode was not
> + available in the version MHUv2.0, but the later versions do have it.
> + In "receiver" mode, beside a single combined interrupt, there could be
> + multiple interrupts, up to the number of implemented channels but anyway
> + no more than the maximum number of interrupts potentially supported by
> + AMBA.
Last sentence indicates that TX mode has something else, e.g. max 1
interrupt. Either correct the sentence or add if:then: narrowing it for TX.
Best regards,
Krzysztof
On 29/03/2023 17:39, Cristian Marussi wrote:
> The ARM MHUv2 Receiver block can indeed support more interrupts, up to the
> maximum number of available channels, but anyway no more than the maximum
> number of supported interrupt for an AMBA device.
Subject: no spaces before colon.
>
> Signed-off-by: Cristian Marussi <[email protected]>
> ---
> Cc: Rob Herring <[email protected]>
> Cc: Krzysztof Kozlowski <[email protected]>
> Cc: [email protected]
Best regards,
Krzysztof
On Wed, Mar 29, 2023 at 06:44:31PM +0100, Sudeep Holla wrote:
> On Wed, Mar 29, 2023 at 04:39:35PM +0100, Cristian Marussi wrote:
> > The ARM MHUv2 Receiver block can indeed support more interrupts, up to the
> > maximum number of available channels, but anyway no more than the maximum
> > number of supported interrupt for an AMBA device.
> >
> > Signed-off-by: Cristian Marussi <[email protected]>
> > ---
> > Cc: Rob Herring <[email protected]>
> > Cc: Krzysztof Kozlowski <[email protected]>
> > Cc: [email protected]
> >
> > .../devicetree/bindings/mailbox/arm,mhuv2.yaml | 13 +++++++++----
> > 1 file changed, 9 insertions(+), 4 deletions(-)
> >
> > diff --git a/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml b/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> > index a4f1fe63659a..5a57f4e2a623 100644
> > --- a/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> > +++ b/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> > @@ -69,10 +69,15 @@ properties:
> >
> > interrupts:
> > description: |
> > - The MHUv2 controller always implements an interrupt in the "receiver"
> > - mode, while the interrupt in the "sender" mode was not available in the
> > - version MHUv2.0, but the later versions do have it.
> > - maxItems: 1
> > + The MHUv2 controller always implements at least an interrupt in the
> > + "receiver" mode, while the interrupt in the "sender" mode was not
> > + available in the version MHUv2.0, but the later versions do have it.
> > + In "receiver" mode, beside a single combined interrupt, there could be
> > + multiple interrupts, up to the number of implemented channels but anyway
> > + no more than the maximum number of interrupts potentially supported by
> > + AMBA.
> > + minItems: 1
> > + maxItems: 9
>
Hi,
> I am not sure 9 is the correct value here. IIUC it is just what Linux defines
> as AMBA_NR_IRQS. Looking at the history it was bumped from 2 to 9 for use
> by PL330 DMA driver. I couldn't find anything to relate this 9 in any
> AMBA or other related specification.
>
Yes, I could not find either where the 9 comes from, but it is what
currently each amba device is limited to, at the software level, in terms of
interrupts that can be detected.
> Ideally I would say we don't know what the max here. We just have a platform
> implementing 2 interrupts now. Do we for with 2 for now and change it if some
> new users require more in the future ?
>
By the spec seems to me that the maximum number of interrupts are equal to
the maximum possible channels (124), or one combined interrupt.
But these in turn, as said, are capped by the AMBA_NR_IRQS and I have
only seen one system using 2. (for which I need this series to work)
> I will leave that to the DT maintainers but 9 is simply random based on Linux
> code so I would rather choose some other random number with a better reasoning
> than 9 as AMBA code in the kernel is limiting it to 9.
>
Agreed. Aiming to describe any possible hw in the DT, I would say 124 at
this point. (even though implausible not to use the combined interrupt
at that point...)
Thanks,
Cristian
On Thu, Mar 30, 2023 at 09:36:06AM +0200, Krzysztof Kozlowski wrote:
> On 29/03/2023 17:39, Cristian Marussi wrote:
> > The ARM MHUv2 Receiver block can indeed support more interrupts, up to the
> > maximum number of available channels, but anyway no more than the maximum
> > number of supported interrupt for an AMBA device.
> >
> > Signed-off-by: Cristian Marussi <[email protected]>
> > ---
> > Cc: Rob Herring <[email protected]>
> > Cc: Krzysztof Kozlowski <[email protected]>
> > Cc: [email protected]
> >
> > .../devicetree/bindings/mailbox/arm,mhuv2.yaml | 13 +++++++++----
> > 1 file changed, 9 insertions(+), 4 deletions(-)
> >
> > diff --git a/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml b/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> > index a4f1fe63659a..5a57f4e2a623 100644
> > --- a/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> > +++ b/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> > @@ -69,10 +69,15 @@ properties:
> >
> > interrupts:
> > description: |
> > - The MHUv2 controller always implements an interrupt in the "receiver"
> > - mode, while the interrupt in the "sender" mode was not available in the
> > - version MHUv2.0, but the later versions do have it.
> > - maxItems: 1
> > + The MHUv2 controller always implements at least an interrupt in the
> > + "receiver" mode, while the interrupt in the "sender" mode was not
> > + available in the version MHUv2.0, but the later versions do have it.
> > + In "receiver" mode, beside a single combined interrupt, there could be
> > + multiple interrupts, up to the number of implemented channels but anyway
> > + no more than the maximum number of interrupts potentially supported by
> > + AMBA.
>
Hi,
> Last sentence indicates that TX mode has something else, e.g. max 1
> interrupt. Either correct the sentence or add if:then: narrowing it for TX.
>
By the spec really you can have up to 124 rx interrupt (one per channel) and
optionally 124 tx interrupt too. At least one RX is mandatory, while the TX
clear channel iterrupt is optional (and not supported at all for spec < 2.1)
In both cases you could just have one single combined interrupt, though,
and this is what the driver did, and still do (I have noot changed this), on
the TX side: it just supports one single combined tx interrupt.
So on the TX side, at the HW level, there could be really 124 interrupts
BUT the driver still only support a single combined one.
So I think my statement above is anyway ambiguos and I'll fix it, but how to
fix it, really depends if we want to describe fully what the HW potentially
supports OR what the driver really can cope with as of now.
Thanks,
Cristian
On Thu, Mar 30, 2023 at 09:29:23AM +0100, Cristian Marussi wrote:
> On Wed, Mar 29, 2023 at 06:44:31PM +0100, Sudeep Holla wrote:
> > On Wed, Mar 29, 2023 at 04:39:35PM +0100, Cristian Marussi wrote:
> > > The ARM MHUv2 Receiver block can indeed support more interrupts, up to the
> > > maximum number of available channels, but anyway no more than the maximum
> > > number of supported interrupt for an AMBA device.
> > >
> > > Signed-off-by: Cristian Marussi <[email protected]>
> > > ---
> > > Cc: Rob Herring <[email protected]>
> > > Cc: Krzysztof Kozlowski <[email protected]>
> > > Cc: [email protected]
> > >
> > > .../devicetree/bindings/mailbox/arm,mhuv2.yaml | 13 +++++++++----
> > > 1 file changed, 9 insertions(+), 4 deletions(-)
> > >
> > > diff --git a/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml b/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> > > index a4f1fe63659a..5a57f4e2a623 100644
> > > --- a/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> > > +++ b/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> > > @@ -69,10 +69,15 @@ properties:
> > >
> > > interrupts:
> > > description: |
> > > - The MHUv2 controller always implements an interrupt in the "receiver"
> > > - mode, while the interrupt in the "sender" mode was not available in the
> > > - version MHUv2.0, but the later versions do have it.
> > > - maxItems: 1
> > > + The MHUv2 controller always implements at least an interrupt in the
> > > + "receiver" mode, while the interrupt in the "sender" mode was not
> > > + available in the version MHUv2.0, but the later versions do have it.
> > > + In "receiver" mode, beside a single combined interrupt, there could be
> > > + multiple interrupts, up to the number of implemented channels but anyway
> > > + no more than the maximum number of interrupts potentially supported by
> > > + AMBA.
> > > + minItems: 1
> > > + maxItems: 9
> >
>
> Hi,
>
> > I am not sure 9 is the correct value here. IIUC it is just what Linux defines
> > as AMBA_NR_IRQS. Looking at the history it was bumped from 2 to 9 for use
> > by PL330 DMA driver. I couldn't find anything to relate this 9 in any
> > AMBA or other related specification.
> >
>
> Yes, I could not find either where the 9 comes from, but it is what
> currently each amba device is limited to, at the software level, in terms of
> interrupts that can be detected.
IIRC, the PL330 can have an interrupt per context with up to 8 contexts
and then 1 global interrupt.
>
> > Ideally I would say we don't know what the max here. We just have a platform
> > implementing 2 interrupts now. Do we for with 2 for now and change it if some
> > new users require more in the future ?
> >
>
> By the spec seems to me that the maximum number of interrupts are equal to
> the maximum possible channels (124), or one combined interrupt.
>
> But these in turn, as said, are capped by the AMBA_NR_IRQS and I have
> only seen one system using 2. (for which I need this series to work)
>
> > I will leave that to the DT maintainers but 9 is simply random based on Linux
> > code so I would rather choose some other random number with a better reasoning
> > than 9 as AMBA code in the kernel is limiting it to 9.
> >
>
> Agreed. Aiming to describe any possible hw in the DT, I would say 124 at
> this point. (even though implausible not to use the combined interrupt
> at that point...)
Then use 124, but please describe how you get that in the description.
Rob
On Wed, Apr 12, 2023 at 08:15:21AM -0500, Rob Herring wrote:
> On Thu, Mar 30, 2023 at 09:29:23AM +0100, Cristian Marussi wrote:
> > On Wed, Mar 29, 2023 at 06:44:31PM +0100, Sudeep Holla wrote:
> > > On Wed, Mar 29, 2023 at 04:39:35PM +0100, Cristian Marussi wrote:
> > > > The ARM MHUv2 Receiver block can indeed support more interrupts, up to the
> > > > maximum number of available channels, but anyway no more than the maximum
> > > > number of supported interrupt for an AMBA device.
> > > >
> > > > Signed-off-by: Cristian Marussi <[email protected]>
> > > > ---
> > > > Cc: Rob Herring <[email protected]>
> > > > Cc: Krzysztof Kozlowski <[email protected]>
> > > > Cc: [email protected]
> > > >
> > > > .../devicetree/bindings/mailbox/arm,mhuv2.yaml | 13 +++++++++----
> > > > 1 file changed, 9 insertions(+), 4 deletions(-)
> > > >
> > > > diff --git a/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml b/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> > > > index a4f1fe63659a..5a57f4e2a623 100644
> > > > --- a/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> > > > +++ b/Documentation/devicetree/bindings/mailbox/arm,mhuv2.yaml
> > > > @@ -69,10 +69,15 @@ properties:
> > > >
> > > > interrupts:
> > > > description: |
> > > > - The MHUv2 controller always implements an interrupt in the "receiver"
> > > > - mode, while the interrupt in the "sender" mode was not available in the
> > > > - version MHUv2.0, but the later versions do have it.
> > > > - maxItems: 1
> > > > + The MHUv2 controller always implements at least an interrupt in the
> > > > + "receiver" mode, while the interrupt in the "sender" mode was not
> > > > + available in the version MHUv2.0, but the later versions do have it.
> > > > + In "receiver" mode, beside a single combined interrupt, there could be
> > > > + multiple interrupts, up to the number of implemented channels but anyway
> > > > + no more than the maximum number of interrupts potentially supported by
> > > > + AMBA.
> > > > + minItems: 1
> > > > + maxItems: 9
> > >
> >
> > Hi,
> >
> > > I am not sure 9 is the correct value here. IIUC it is just what Linux defines
> > > as AMBA_NR_IRQS. Looking at the history it was bumped from 2 to 9 for use
> > > by PL330 DMA driver. I couldn't find anything to relate this 9 in any
> > > AMBA or other related specification.
> > >
> >
> > Yes, I could not find either where the 9 comes from, but it is what
> > currently each amba device is limited to, at the software level, in terms of
> > interrupts that can be detected.
>
> IIRC, the PL330 can have an interrupt per context with up to 8 contexts
> and then 1 global interrupt.
>
> >
> > > Ideally I would say we don't know what the max here. We just have a platform
> > > implementing 2 interrupts now. Do we for with 2 for now and change it if some
> > > new users require more in the future ?
> > >
> >
> > By the spec seems to me that the maximum number of interrupts are equal to
> > the maximum possible channels (124), or one combined interrupt.
> >
> > But these in turn, as said, are capped by the AMBA_NR_IRQS and I have
> > only seen one system using 2. (for which I need this series to work)
> >
> > > I will leave that to the DT maintainers but 9 is simply random based on Linux
> > > code so I would rather choose some other random number with a better reasoning
> > > than 9 as AMBA code in the kernel is limiting it to 9.
> > >
> >
> > Agreed. Aiming to describe any possible hw in the DT, I would say 124 at
> > this point. (even though implausible not to use the combined interrupt
> > at that point...)
>
> Then use 124, but please describe how you get that in the description.
>
Ok, thanks, I'll do.
Cristian