When num-channels and qcom,num-ees is not provided in devicetree, the
driver will try to read these values from the registers during probe but
this fails if the interconnect is not on and then crashes the system.
So we can provide these properties in devicetree (queried after patching
BAM driver to enable the necessary interconnect) so we can probe
cryptobam without reading registers and then also use the QCE as
expected.
Fixes: 4d29db204361 ("arm64: dts: qcom: sm8350: fix BAM DMA crash and reboot")
Fixes: f1040a7fe8f0 ("arm64: dts: qcom: sm8350: Add Crypto Engine support")
Signed-off-by: Luca Weiss <[email protected]>
---
Not tested myself, but David Heidelberg was so nice and ran it on a
SM8350 board in a test farm and it seems to be working as expected.
But still please test it on some other boards so make sure it actually
works as expected there also.
---
arch/arm64/boot/dts/qcom/sm8350.dtsi | 6 ++----
1 file changed, 2 insertions(+), 4 deletions(-)
diff --git a/arch/arm64/boot/dts/qcom/sm8350.dtsi b/arch/arm64/boot/dts/qcom/sm8350.dtsi
index b46236235b7f..3cd75ab552c5 100644
--- a/arch/arm64/boot/dts/qcom/sm8350.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8350.dtsi
@@ -1754,10 +1754,10 @@ cryptobam: dma-controller@1dc4000 {
#dma-cells = <1>;
qcom,ee = <0>;
qcom,controlled-remotely;
+ num-channels = <16>;
+ qcom,num-ees = <4>;
iommus = <&apps_smmu 0x594 0x0011>,
<&apps_smmu 0x596 0x0011>;
- /* FIXME: Probing BAM DMA causes some abort and system hang */
- status = "fail";
};
crypto: crypto@1dfa000 {
@@ -1769,8 +1769,6 @@ crypto: crypto@1dfa000 {
<&apps_smmu 0x596 0x0011>;
interconnects = <&aggre2_noc MASTER_CRYPTO 0 &mc_virt SLAVE_EBI1 0>;
interconnect-names = "memory";
- /* FIXME: dependency BAM DMA is disabled */
- status = "disabled";
};
ipa: ipa@1e40000 {
---
base-commit: 0dd3ee31125508cd67f7e7172247f05b7fd1753a
change-id: 20240108-sm8350-qce-6ada49f90657
Best regards,
--
Luca Weiss <[email protected]>
On 8.01.2024 14:49, Luca Weiss wrote:
> When num-channels and qcom,num-ees is not provided in devicetree, the
> driver will try to read these values from the registers during probe but
> this fails if the interconnect is not on and then crashes the system.
>
> So we can provide these properties in devicetree (queried after patching
> BAM driver to enable the necessary interconnect) so we can probe
> cryptobam without reading registers and then also use the QCE as
> expected.
This really feels a bit backwards.. Enable the resource to query the
hardware for numbers, so that said resource can be enabled, but
slightly later :/
Konrad
On Mon Jan 8, 2024 at 3:18 PM CET, Konrad Dybcio wrote:
> On 8.01.2024 14:49, Luca Weiss wrote:
> > When num-channels and qcom,num-ees is not provided in devicetree, the
> > driver will try to read these values from the registers during probe but
> > this fails if the interconnect is not on and then crashes the system.
> >
> > So we can provide these properties in devicetree (queried after patching
> > BAM driver to enable the necessary interconnect) so we can probe
> > cryptobam without reading registers and then also use the QCE as
> > expected.
>
> This really feels a bit backwards.. Enable the resource to query the
> hardware for numbers, so that said resource can be enabled, but
> slightly later :/
If you think adding interconnect support to driver and dtsi is better,
let me know.
Stephan (+CC) mentioned it should be okay like this *shrug*
For the record, this is the same way I got the values for sc7280[0] and
sm6350[1].
[0] https://lore.kernel.org/linux-arm-msm/[email protected]/
[1] https://lore.kernel.org/linux-arm-msm/[email protected]/
diff --git a/arch/arm64/boot/dts/qcom/sm8350.dtsi b/arch/arm64/boot/dts/qcom/sm8350.dtsi
index b46236235b7f..cd4dd9852d9e 100644
--- a/arch/arm64/boot/dts/qcom/sm8350.dtsi
+++ b/arch/arm64/boot/dts/qcom/sm8350.dtsi
@@ -1756,8 +1756,8 @@ cryptobam: dma-controller@1dc4000 {
qcom,controlled-remotely;
iommus = <&apps_smmu 0x594 0x0011>,
<&apps_smmu 0x596 0x0011>;
- /* FIXME: Probing BAM DMA causes some abort and system hang */
- status = "fail";
+ interconnects = <&aggre2_noc MASTER_CRYPTO 0 &mc_virt SLAVE_EBI1 0>;
+ interconnect-names = "memory";
};
crypto: crypto@1dfa000 {
diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c
index 5e7d332731e0..9de28f615639 100644
--- a/drivers/dma/qcom/bam_dma.c
+++ b/drivers/dma/qcom/bam_dma.c
@@ -40,6 +40,7 @@
#include <linux/circ_buf.h>
#include <linux/clk.h>
#include <linux/dmaengine.h>
+#include <linux/interconnect.h>
#include <linux/pm_runtime.h>
#include "../dmaengine.h"
@@ -394,6 +395,7 @@ struct bam_device {
const struct reg_offset_data *layout;
struct clk *bamclk;
+ struct icc_path *mem_path;
int irq;
/* dma start transaction tasklet */
@@ -1206,6 +1208,7 @@ static int bam_init(struct bam_device *bdev)
bdev->num_channels = val & BAM_NUM_PIPES_MASK;
}
+ printk(KERN_ERR "%s:%d DBG num_ees=%u num_channels=%u\n", __func__, __LINE__, bdev->num_ees, bdev->num_channels);
/* Reset BAM now if fully controlled locally */
if (!bdev->controlled_remotely && !bdev->powered_remotely)
bam_reset(bdev);
@@ -1298,6 +1301,14 @@ static int bam_dma_probe(struct platform_device *pdev)
return ret;
}
+ bdev->mem_path = devm_of_icc_get(bdev->dev, "memory");
+ if (IS_ERR(bdev->mem_path))
+ return PTR_ERR(bdev->mem_path);
+
+ ret = icc_set_bw(bdev->mem_path, 1, 1);
+ if (ret)
+ return ret;
+
ret = bam_init(bdev);
if (ret)
goto err_disable_clk;
On Mon, 8 Jan 2024 at 16:23, Luca Weiss <[email protected]> wrote:
>
> On Mon Jan 8, 2024 at 3:18 PM CET, Konrad Dybcio wrote:
> > On 8.01.2024 14:49, Luca Weiss wrote:
> > > When num-channels and qcom,num-ees is not provided in devicetree, the
> > > driver will try to read these values from the registers during probe but
> > > this fails if the interconnect is not on and then crashes the system.
> > >
> > > So we can provide these properties in devicetree (queried after patching
> > > BAM driver to enable the necessary interconnect) so we can probe
> > > cryptobam without reading registers and then also use the QCE as
> > > expected.
> >
> > This really feels a bit backwards.. Enable the resource to query the
> > hardware for numbers, so that said resource can be enabled, but
> > slightly later :/
>
> If you think adding interconnect support to driver and dtsi is better,
> let me know.
I'd say, adding the proper interconnect is a better option. Otherwise
we just depend on the QCE itself to set up the vote for us.
>
> Stephan (+CC) mentioned it should be okay like this *shrug*
>
> For the record, this is the same way I got the values for sc7280[0] and
> sm6350[1].
>
> [0] https://lore.kernel.org/linux-arm-msm/[email protected]/
> [1] https://lore.kernel.org/linux-arm-msm/[email protected]/
>
> diff --git a/arch/arm64/boot/dts/qcom/sm8350.dtsi b/arch/arm64/boot/dts/qcom/sm8350.dtsi
> index b46236235b7f..cd4dd9852d9e 100644
> --- a/arch/arm64/boot/dts/qcom/sm8350.dtsi
> +++ b/arch/arm64/boot/dts/qcom/sm8350.dtsi
> @@ -1756,8 +1756,8 @@ cryptobam: dma-controller@1dc4000 {
> qcom,controlled-remotely;
> iommus = <&apps_smmu 0x594 0x0011>,
> <&apps_smmu 0x596 0x0011>;
> - /* FIXME: Probing BAM DMA causes some abort and system hang */
> - status = "fail";
> + interconnects = <&aggre2_noc MASTER_CRYPTO 0 &mc_virt SLAVE_EBI1 0>;
> + interconnect-names = "memory";
> };
>
> crypto: crypto@1dfa000 {
> diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c
> index 5e7d332731e0..9de28f615639 100644
> --- a/drivers/dma/qcom/bam_dma.c
> +++ b/drivers/dma/qcom/bam_dma.c
> @@ -40,6 +40,7 @@
> #include <linux/circ_buf.h>
> #include <linux/clk.h>
> #include <linux/dmaengine.h>
> +#include <linux/interconnect.h>
> #include <linux/pm_runtime.h>
>
> #include "../dmaengine.h"
> @@ -394,6 +395,7 @@ struct bam_device {
> const struct reg_offset_data *layout;
>
> struct clk *bamclk;
> + struct icc_path *mem_path;
> int irq;
>
> /* dma start transaction tasklet */
> @@ -1206,6 +1208,7 @@ static int bam_init(struct bam_device *bdev)
> bdev->num_channels = val & BAM_NUM_PIPES_MASK;
> }
>
> + printk(KERN_ERR "%s:%d DBG num_ees=%u num_channels=%u\n", __func__, __LINE__, bdev->num_ees, bdev->num_channels);
> /* Reset BAM now if fully controlled locally */
> if (!bdev->controlled_remotely && !bdev->powered_remotely)
> bam_reset(bdev);
> @@ -1298,6 +1301,14 @@ static int bam_dma_probe(struct platform_device *pdev)
> return ret;
> }
>
> + bdev->mem_path = devm_of_icc_get(bdev->dev, "memory");
> + if (IS_ERR(bdev->mem_path))
> + return PTR_ERR(bdev->mem_path);
> +
> + ret = icc_set_bw(bdev->mem_path, 1, 1);
Probably this needs some more sensible value.
> + if (ret)
> + return ret;
> +
> ret = bam_init(bdev);
> if (ret)
> goto err_disable_clk;
>
--
With best wishes
Dmitry
On Mon Jan 8, 2024 at 11:45 PM CET, Dmitry Baryshkov wrote:
> On Mon, 8 Jan 2024 at 16:23, Luca Weiss <[email protected]> wrote:
> >
> > On Mon Jan 8, 2024 at 3:18 PM CET, Konrad Dybcio wrote:
> > > On 8.01.2024 14:49, Luca Weiss wrote:
> > > > When num-channels and qcom,num-ees is not provided in devicetree, the
> > > > driver will try to read these values from the registers during probe but
> > > > this fails if the interconnect is not on and then crashes the system.
> > > >
> > > > So we can provide these properties in devicetree (queried after patching
> > > > BAM driver to enable the necessary interconnect) so we can probe
> > > > cryptobam without reading registers and then also use the QCE as
> > > > expected.
> > >
> > > This really feels a bit backwards.. Enable the resource to query the
> > > hardware for numbers, so that said resource can be enabled, but
> > > slightly later :/
> >
> > If you think adding interconnect support to driver and dtsi is better,
> > let me know.
>
> I'd say, adding the proper interconnect is a better option. Otherwise
> we just depend on the QCE itself to set up the vote for us.
Yes, currently we depend on that.
>
> >
> > Stephan (+CC) mentioned it should be okay like this *shrug*
> >
> > For the record, this is the same way I got the values for sc7280[0] and
> > sm6350[1].
> >
> > [0] https://lore.kernel.org/linux-arm-msm/[email protected]/
> > [1] https://lore.kernel.org/linux-arm-msm/[email protected]/
> >
> > diff --git a/arch/arm64/boot/dts/qcom/sm8350.dtsi b/arch/arm64/boot/dts/qcom/sm8350.dtsi
> > index b46236235b7f..cd4dd9852d9e 100644
> > --- a/arch/arm64/boot/dts/qcom/sm8350.dtsi
> > +++ b/arch/arm64/boot/dts/qcom/sm8350.dtsi
> > @@ -1756,8 +1756,8 @@ cryptobam: dma-controller@1dc4000 {
> > qcom,controlled-remotely;
> > iommus = <&apps_smmu 0x594 0x0011>,
> > <&apps_smmu 0x596 0x0011>;
> > - /* FIXME: Probing BAM DMA causes some abort and system hang */
> > - status = "fail";
> > + interconnects = <&aggre2_noc MASTER_CRYPTO 0 &mc_virt SLAVE_EBI1 0>;
> > + interconnect-names = "memory";
> > };
> >
> > crypto: crypto@1dfa000 {
> > diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c
> > index 5e7d332731e0..9de28f615639 100644
> > --- a/drivers/dma/qcom/bam_dma.c
> > +++ b/drivers/dma/qcom/bam_dma.c
> > @@ -40,6 +40,7 @@
> > #include <linux/circ_buf.h>
> > #include <linux/clk.h>
> > #include <linux/dmaengine.h>
> > +#include <linux/interconnect.h>
> > #include <linux/pm_runtime.h>
> >
> > #include "../dmaengine.h"
> > @@ -394,6 +395,7 @@ struct bam_device {
> > const struct reg_offset_data *layout;
> >
> > struct clk *bamclk;
> > + struct icc_path *mem_path;
> > int irq;
> >
> > /* dma start transaction tasklet */
> > @@ -1206,6 +1208,7 @@ static int bam_init(struct bam_device *bdev)
> > bdev->num_channels = val & BAM_NUM_PIPES_MASK;
> > }
> >
> > + printk(KERN_ERR "%s:%d DBG num_ees=%u num_channels=%u\n", __func__, __LINE__, bdev->num_ees, bdev->num_channels);
> > /* Reset BAM now if fully controlled locally */
> > if (!bdev->controlled_remotely && !bdev->powered_remotely)
> > bam_reset(bdev);
> > @@ -1298,6 +1301,14 @@ static int bam_dma_probe(struct platform_device *pdev)
> > return ret;
> > }
> >
> > + bdev->mem_path = devm_of_icc_get(bdev->dev, "memory");
> > + if (IS_ERR(bdev->mem_path))
> > + return PTR_ERR(bdev->mem_path);
> > +
> > + ret = icc_set_bw(bdev->mem_path, 1, 1);
>
> Probably this needs some more sensible value.
So downstream qcedev driver uses 384 for the interconnect. But this is
crypto-specific and probably different BAMs have different minimum
requirements?
#define CRYPTO_AVG_BW 384
#define CRYPTO_PEAK_BW 384
https://github.com/xiaomi-sm8450-kernel/android_kernel_platform_msm-kernel/blob/lineage-20/drivers/crypto/msm/qce.h#L57
Do you have any suggestion what to use here?
Also I'd assume that with pm_runtime suspended we'd need to clear the
votes in the driver so we don't keep the interconnect alive
unnecessarily?
If someone wants to pick up that patch, I'd be very glad since
especially for sm8350 this is just a drive-by, I don't care too much
about the SoC myself ;)
Regards
Luca
>
> > + if (ret)
> > + return ret;
> > +
> > ret = bam_init(bdev);
> > if (ret)
> > goto err_disable_clk;
> >
On 16.02.2024 11:36 AM, Luca Weiss wrote:
> On Mon Jan 8, 2024 at 11:45 PM CET, Dmitry Baryshkov wrote:
>> On Mon, 8 Jan 2024 at 16:23, Luca Weiss <[email protected]> wrote:
>>>
>>> On Mon Jan 8, 2024 at 3:18 PM CET, Konrad Dybcio wrote:
>>>> On 8.01.2024 14:49, Luca Weiss wrote:
>>>>> When num-channels and qcom,num-ees is not provided in devicetree, the
>>>>> driver will try to read these values from the registers during probe but
>>>>> this fails if the interconnect is not on and then crashes the system.
>>>>>
>>>>> So we can provide these properties in devicetree (queried after patching
>>>>> BAM driver to enable the necessary interconnect) so we can probe
>>>>> cryptobam without reading registers and then also use the QCE as
>>>>> expected.
>>>>
>>>> This really feels a bit backwards.. Enable the resource to query the
>>>> hardware for numbers, so that said resource can be enabled, but
>>>> slightly later :/
>>>
>>> If you think adding interconnect support to driver and dtsi is better,
>>> let me know.
>>
>> I'd say, adding the proper interconnect is a better option. Otherwise
>> we just depend on the QCE itself to set up the vote for us.
>
> Yes, currently we depend on that.
>
>>
>>>
>>> Stephan (+CC) mentioned it should be okay like this *shrug*
>>>
>>> For the record, this is the same way I got the values for sc7280[0] and
>>> sm6350[1].
>>>
>>> [0] https://lore.kernel.org/linux-arm-msm/[email protected]/
>>> [1] https://lore.kernel.org/linux-arm-msm/[email protected]/
>>>
>>> diff --git a/arch/arm64/boot/dts/qcom/sm8350.dtsi b/arch/arm64/boot/dts/qcom/sm8350.dtsi
>>> index b46236235b7f..cd4dd9852d9e 100644
>>> --- a/arch/arm64/boot/dts/qcom/sm8350.dtsi
>>> +++ b/arch/arm64/boot/dts/qcom/sm8350.dtsi
>>> @@ -1756,8 +1756,8 @@ cryptobam: dma-controller@1dc4000 {
>>> qcom,controlled-remotely;
>>> iommus = <&apps_smmu 0x594 0x0011>,
>>> <&apps_smmu 0x596 0x0011>;
>>> - /* FIXME: Probing BAM DMA causes some abort and system hang */
>>> - status = "fail";
>>> + interconnects = <&aggre2_noc MASTER_CRYPTO 0 &mc_virt SLAVE_EBI1 0>;
>>> + interconnect-names = "memory";
>>> };
>>>
>>> crypto: crypto@1dfa000 {
>>> diff --git a/drivers/dma/qcom/bam_dma.c b/drivers/dma/qcom/bam_dma.c
>>> index 5e7d332731e0..9de28f615639 100644
>>> --- a/drivers/dma/qcom/bam_dma.c
>>> +++ b/drivers/dma/qcom/bam_dma.c
>>> @@ -40,6 +40,7 @@
>>> #include <linux/circ_buf.h>
>>> #include <linux/clk.h>
>>> #include <linux/dmaengine.h>
>>> +#include <linux/interconnect.h>
>>> #include <linux/pm_runtime.h>
>>>
>>> #include "../dmaengine.h"
>>> @@ -394,6 +395,7 @@ struct bam_device {
>>> const struct reg_offset_data *layout;
>>>
>>> struct clk *bamclk;
>>> + struct icc_path *mem_path;
>>> int irq;
>>>
>>> /* dma start transaction tasklet */
>>> @@ -1206,6 +1208,7 @@ static int bam_init(struct bam_device *bdev)
>>> bdev->num_channels = val & BAM_NUM_PIPES_MASK;
>>> }
>>>
>>> + printk(KERN_ERR "%s:%d DBG num_ees=%u num_channels=%u\n", __func__, __LINE__, bdev->num_ees, bdev->num_channels);
>>> /* Reset BAM now if fully controlled locally */
>>> if (!bdev->controlled_remotely && !bdev->powered_remotely)
>>> bam_reset(bdev);
>>> @@ -1298,6 +1301,14 @@ static int bam_dma_probe(struct platform_device *pdev)
>>> return ret;
>>> }
>>>
>>> + bdev->mem_path = devm_of_icc_get(bdev->dev, "memory");
>>> + if (IS_ERR(bdev->mem_path))
>>> + return PTR_ERR(bdev->mem_path);
>>> +
>>> + ret = icc_set_bw(bdev->mem_path, 1, 1);
>>
>> Probably this needs some more sensible value.
>
> So downstream qcedev driver uses 384 for the interconnect. But this is
> crypto-specific and probably different BAMs have different minimum
> requirements?
>
> #define CRYPTO_AVG_BW 384
> #define CRYPTO_PEAK_BW 384
> https://github.com/xiaomi-sm8450-kernel/android_kernel_platform_msm-kernel/blob/lineage-20/drivers/crypto/msm/qce.h#L57
>
> Do you have any suggestion what to use here?
I'dve expected this to mean anything, but apparently not.
My immediate guess is that the 384 was the lowest magic value that didn't
result in the bus getting kicked offline.. 1 should be fine upstream due
to commit 91e045b93db7 ("interconnect: qcom: Fix small BW votes being
truncated to zero").
>
> Also I'd assume that with pm_runtime suspended we'd need to clear the
> votes in the driver so we don't keep the interconnect alive
> unnecessarily?
My naive understanding is that the power should only be necessary when
the thing is in use, so early probe and pm-active sounds about sane..
Konrad