Hello!
This patchset expand the funcionality of the Amlogic
crypto driver by adding support for more SoC families:
AXG, G12A, G12B, SM1, A1, S4.
Also specify and enable crypto node in device tree
for reference Amlogic devices.
Tested on GXL, AXG, G12A/B, SM1, A1 and S4 devices via
custom tests [1] and tcrypt module.
---
Changes V1 -> V2 [2]:
- Rebased over linux-next.
- Adjusted device tree bindings description.
- A1 and S4 dts use their own compatible, which is a G12 fallback.
Changes V2 -> V3 [3]:
- Fix errors in dt-bindings and device tree.
- Add new field in platform data, which determines
whether clock controller should be used for crypto IP.
- Place back MODULE_DEVICE_TABLE.
- Correct commit messages.
Changes V3 -> V4 [4]:
- Update dt-bindings as per Krzysztof Kozlowski comments.
- Fix bisection: get rid of compiler errors in some patches.
Changes V4 -> V5 [5]:
- Tested on GXL board:
1. Fix panic detected by Corentin Labbe [6].
2. Disable hasher backend for GXL: in its current realization
is doesn't work. And there are no examples or docs in the
vendor SDK.
- Fix AES-CTR realization: legacy boards (gxl, g12, axg) requires
inversion of the keyiv at keys setup stage.
- A1 now uses its own compatible string.
- S4 uses A1 compatible as fallback.
- Code fixes based on comments Neil Atrmstrong and Rob Herring.
- Style fixes (set correct indentations)
Links:
- [1] https://gist.github.com/mRrvz/3fb8943a7487ab7b943ec140706995e7
- [2] https://lore.kernel.org/all/[email protected]/
- [3] https://lore.kernel.org/all/[email protected]/
- [4] https://lore.kernel.org/all/[email protected]/
- [5] https://lore.kernel.org/all/[email protected]/
- [6] https://lore.kernel.org/all/ZcsYaPIUrBSg8iXu@Red/
Alexey Romanov (21):
drivers: crypto: meson: don't hardcode IRQ count
drviers: crypto: meson: add platform data
drivers: crypto: meson: make CLK controller optional
drivers: crypto: meson: add MMIO helpers
drivers: crypto: meson: move get_engine_number()
drivers: crypto: meson: drop status field from meson_flow
drivers: crypto: meson: move algs definition and cipher API to
cipher.c
drivers: crypto: meson: cleanup defines
drivers: crypto: meson: process more than MAXDESCS descriptors
drivers: crypto: meson: avoid kzalloc in engine thread
drivers: crypto: meson: introduce hasher
drivers: crypto: meson: add support for AES-CTR
drivers: crypto: meson: use fallback for 192-bit keys
drivers: crypto: meson: add support for G12-series
drivers: crypto: meson: add support for AXG-series
drivers: crypto: meson: add support for A1-series
dt-bindings: crypto: meson: support new SoC's
arch: arm64: dts: meson: a1: add crypto node
arch: arm64: dts: meson: s4: add crypto node
arch: arm64: dts: meson: g12: add crypto node
arch: arm64: dts: meson: axg: add crypto node
.../bindings/crypto/amlogic,gxl-crypto.yaml | 36 +-
arch/arm64/boot/dts/amlogic/meson-a1.dtsi | 7 +
arch/arm64/boot/dts/amlogic/meson-axg.dtsi | 6 +
.../boot/dts/amlogic/meson-g12-common.dtsi | 6 +
arch/arm64/boot/dts/amlogic/meson-s4.dtsi | 6 +
drivers/crypto/amlogic/Makefile | 2 +-
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 615 ++++++++++++------
drivers/crypto/amlogic/amlogic-gxl-core.c | 290 +++++----
drivers/crypto/amlogic/amlogic-gxl-hasher.c | 460 +++++++++++++
drivers/crypto/amlogic/amlogic-gxl.h | 118 +++-
10 files changed, 1205 insertions(+), 341 deletions(-)
create mode 100644 drivers/crypto/amlogic/amlogic-gxl-hasher.c
--
2.34.1
IRQ count is no longer hardcoded, and make it part of
struct meson_flow. We need this for extend driver support for
other Amlogic SoC's.
Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 2 +-
drivers/crypto/amlogic/amlogic-gxl-core.c | 47 ++++++++++++---------
drivers/crypto/amlogic/amlogic-gxl.h | 8 ++--
3 files changed, 31 insertions(+), 26 deletions(-)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index 29048da6f50a..b19032f92415 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -19,7 +19,7 @@
static int get_engine_number(struct meson_dev *mc)
{
- return atomic_inc_return(&mc->flow) % MAXFLOW;
+ return atomic_inc_return(&mc->flow) % mc->flow_cnt;
}
static bool meson_cipher_need_fallback(struct skcipher_request *areq)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index f54ab0d0b1e8..35ec64df5b3a 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -28,8 +28,8 @@ static irqreturn_t meson_irq_handler(int irq, void *data)
int flow;
u32 p;
- for (flow = 0; flow < MAXFLOW; flow++) {
- if (mc->irqs[flow] == irq) {
+ for (flow = 0; flow < mc->flow_cnt; flow++) {
+ if (mc->chanlist[flow].irq == irq) {
p = readl(mc->base + ((0x04 + flow) << 2));
if (p) {
writel_relaxed(0xF, mc->base + ((0x4 + flow) << 2));
@@ -110,7 +110,7 @@ static int meson_debugfs_show(struct seq_file *seq, void *v)
struct meson_dev *mc __maybe_unused = seq->private;
int i;
- for (i = 0; i < MAXFLOW; i++)
+ for (i = 0; i < mc->flow_cnt; i++)
seq_printf(seq, "Channel %d: nreq %lu\n", i,
#ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
mc->chanlist[i].stat_req);
@@ -153,14 +153,32 @@ static void meson_free_chanlist(struct meson_dev *mc, int i)
*/
static int meson_allocate_chanlist(struct meson_dev *mc)
{
+ struct platform_device *pdev = to_platform_device(mc->dev);
int i, err;
- mc->chanlist = devm_kcalloc(mc->dev, MAXFLOW,
+ mc->flow_cnt = platform_irq_count(pdev);
+ if (mc->flow_cnt <= 0) {
+ dev_err(mc->dev, "No IRQs defined\n");
+ return -ENODEV;
+ }
+
+ mc->chanlist = devm_kcalloc(mc->dev, mc->flow_cnt,
sizeof(struct meson_flow), GFP_KERNEL);
if (!mc->chanlist)
return -ENOMEM;
- for (i = 0; i < MAXFLOW; i++) {
+ for (i = 0; i < mc->flow_cnt; i++) {
+ mc->chanlist[i].irq = platform_get_irq(pdev, i);
+ if (mc->chanlist[i].irq < 0)
+ return mc->chanlist[i].irq;
+
+ err = devm_request_irq(mc->dev, mc->chanlist[i].irq,
+ meson_irq_handler, 0, "aml-crypto", mc);
+ if (err < 0) {
+ dev_err(mc->dev, "Cannot request IRQ for flow %d\n", i);
+ return err;
+ }
+
init_completion(&mc->chanlist[i].complete);
mc->chanlist[i].engine = crypto_engine_alloc_init(mc->dev, true);
@@ -230,7 +248,7 @@ static void meson_unregister_algs(struct meson_dev *mc)
static int meson_crypto_probe(struct platform_device *pdev)
{
struct meson_dev *mc;
- int err, i;
+ int err;
mc = devm_kzalloc(&pdev->dev, sizeof(*mc), GFP_KERNEL);
if (!mc)
@@ -252,19 +270,6 @@ static int meson_crypto_probe(struct platform_device *pdev)
return err;
}
- for (i = 0; i < MAXFLOW; i++) {
- mc->irqs[i] = platform_get_irq(pdev, i);
- if (mc->irqs[i] < 0)
- return mc->irqs[i];
-
- err = devm_request_irq(&pdev->dev, mc->irqs[i], meson_irq_handler, 0,
- "gxl-crypto", mc);
- if (err < 0) {
- dev_err(mc->dev, "Cannot request IRQ for flow %d\n", i);
- return err;
- }
- }
-
err = clk_prepare_enable(mc->busclk);
if (err != 0) {
dev_err(&pdev->dev, "Cannot prepare_enable busclk\n");
@@ -294,7 +299,7 @@ static int meson_crypto_probe(struct platform_device *pdev)
error_alg:
meson_unregister_algs(mc);
error_flow:
- meson_free_chanlist(mc, MAXFLOW - 1);
+ meson_free_chanlist(mc, mc->flow_cnt - 1);
clk_disable_unprepare(mc->busclk);
return err;
}
@@ -309,7 +314,7 @@ static void meson_crypto_remove(struct platform_device *pdev)
meson_unregister_algs(mc);
- meson_free_chanlist(mc, MAXFLOW - 1);
+ meson_free_chanlist(mc, mc->flow_cnt - 1);
clk_disable_unprepare(mc->busclk);
}
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index 1013a666c932..79177cfa8b88 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -22,8 +22,6 @@
#define MESON_OPMODE_ECB 0
#define MESON_OPMODE_CBC 1
-#define MAXFLOW 2
-
#define MAXDESC 64
#define DESC_LAST BIT(18)
@@ -62,6 +60,7 @@ struct meson_desc {
* @keylen: keylen for this flow operation
* @complete: completion for the current task on this flow
* @status: set to 1 by interrupt if task is done
+ * @irq: IRQ number for amlogic-crypto
* @t_phy: Physical address of task
* @tl: pointer to the current ce_task for this flow
* @stat_req: number of request done by this flow
@@ -70,6 +69,7 @@ struct meson_flow {
struct crypto_engine *engine;
struct completion complete;
int status;
+ int irq;
unsigned int keylen;
dma_addr_t t_phy;
struct meson_desc *tl;
@@ -85,7 +85,7 @@ struct meson_flow {
* @dev: the platform device
* @chanlist: array of all flow
* @flow: flow to use in next request
- * @irqs: IRQ numbers for amlogic-crypto
+ * @flow_cnt: flow count for amlogic-crypto
* @dbgfs_dir: Debugfs dentry for statistic directory
* @dbgfs_stats: Debugfs dentry for statistic counters
*/
@@ -95,7 +95,7 @@ struct meson_dev {
struct device *dev;
struct meson_flow *chanlist;
atomic_t flow;
- int irqs[MAXFLOW];
+ int flow_cnt;
#ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
struct dentry *dbgfs_dir;
#endif
--
2.34.1
Move get_engine_number() function from cipher.c to core.c
Signed-off-by: Alexey Romanov <[email protected]>
Reviewed-by: Neil Armstrong <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 9 ++-------
drivers/crypto/amlogic/amlogic-gxl-core.c | 5 +++++
drivers/crypto/amlogic/amlogic-gxl.h | 2 ++
3 files changed, 9 insertions(+), 7 deletions(-)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index 1fe916b0a138..18e9e2d39b1f 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -17,11 +17,6 @@
#include <crypto/internal/skcipher.h>
#include "amlogic-gxl.h"
-static int get_engine_number(struct meson_dev *mc)
-{
- return atomic_inc_return(&mc->flow) % mc->flow_cnt;
-}
-
static bool meson_cipher_need_fallback(struct skcipher_request *areq)
{
struct scatterlist *src_sg = areq->src;
@@ -282,7 +277,7 @@ int meson_skdecrypt(struct skcipher_request *areq)
rctx->op_dir = MESON_DECRYPT;
if (meson_cipher_need_fallback(areq))
return meson_cipher_do_fallback(areq);
- e = get_engine_number(op->mc);
+ e = meson_get_engine_number(op->mc);
engine = op->mc->chanlist[e].engine;
rctx->flow = e;
@@ -300,7 +295,7 @@ int meson_skencrypt(struct skcipher_request *areq)
rctx->op_dir = MESON_ENCRYPT;
if (meson_cipher_need_fallback(areq))
return meson_cipher_do_fallback(areq);
- e = get_engine_number(op->mc);
+ e = meson_get_engine_number(op->mc);
engine = op->mc->chanlist[e].engine;
rctx->flow = e;
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 70c2433c56da..467468a8a05e 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -23,6 +23,11 @@
#include "amlogic-gxl.h"
+int meson_get_engine_number(struct meson_dev *mc)
+{
+ return atomic_inc_return(&mc->flow) % mc->flow_cnt;
+}
+
void meson_dma_start(struct meson_dev *mc, int flow)
{
u32 offset = (mc->pdata->descs_reg + flow) << 2;
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index 8670f7ebcdda..dc54bd533a2d 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -161,6 +161,8 @@ struct meson_alg_template {
#endif
};
+int meson_get_engine_number(struct meson_dev *mc);
+
void meson_dma_start(struct meson_dev *mc, int flow);
int meson_enqueue(struct crypto_async_request *areq, u32 type);
--
2.34.1
This field is used only to check for timeout. But there is more
convenient way to achive the same goal.
Signed-off-by: Alexey Romanov <[email protected]>
Reviewed-by: Neil Armstrong <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 13 +++++++++----
drivers/crypto/amlogic/amlogic-gxl-core.c | 1 -
drivers/crypto/amlogic/amlogic-gxl.h | 2 --
3 files changed, 9 insertions(+), 7 deletions(-)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index 18e9e2d39b1f..dc0b100c5de2 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -219,13 +219,18 @@ static int meson_cipher(struct skcipher_request *areq)
}
reinit_completion(&mc->chanlist[flow].complete);
- mc->chanlist[flow].status = 0;
meson_dma_start(mc, flow);
- wait_for_completion_interruptible_timeout(&mc->chanlist[flow].complete,
- msecs_to_jiffies(500));
- if (mc->chanlist[flow].status == 0) {
+
+ err = wait_for_completion_interruptible_timeout(&mc->chanlist[flow].complete,
+ msecs_to_jiffies(500));
+ if (err == 0) {
dev_err(mc->dev, "DMA timeout for flow %d\n", flow);
err = -EINVAL;
+ } else if (err < 0) {
+ dev_err(mc->dev, "Waiting for DMA completion is failed (%d)\n", err);
+ } else {
+ /* No error */
+ err = 0;
}
dma_unmap_single(mc->dev, phykeyiv, keyivlen, DMA_TO_DEVICE);
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 467468a8a05e..882482438ba2 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -54,7 +54,6 @@ static irqreturn_t meson_irq_handler(int irq, void *data)
for (flow = 0; flow < mc->flow_cnt; flow++) {
if (mc->chanlist[flow].irq == irq) {
if (meson_dma_ready(mc, flow)) {
- mc->chanlist[flow].status = 1;
complete(&mc->chanlist[flow].complete);
return IRQ_HANDLED;
}
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index dc54bd533a2d..4d60a0cc2dca 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -59,7 +59,6 @@ struct meson_desc {
* @engine: ptr to the crypto_engine for this flow
* @keylen: keylen for this flow operation
* @complete: completion for the current task on this flow
- * @status: set to 1 by interrupt if task is done
* @irq: IRQ number for amlogic-crypto
* @t_phy: Physical address of task
* @tl: pointer to the current ce_task for this flow
@@ -68,7 +67,6 @@ struct meson_desc {
struct meson_flow {
struct crypto_engine *engine;
struct completion complete;
- int status;
int irq;
unsigned int keylen;
dma_addr_t t_phy;
--
2.34.1
To support other Amlogic SoC's we have to
use platform data: descriptors and status registers
offsets are individual for each SoC series.
Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 2 +-
drivers/crypto/amlogic/amlogic-gxl-core.c | 22 +++++++++++++++++----
drivers/crypto/amlogic/amlogic-gxl.h | 11 +++++++++++
3 files changed, 30 insertions(+), 5 deletions(-)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index b19032f92415..7eff3ae7356f 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -225,7 +225,7 @@ static int meson_cipher(struct skcipher_request *areq)
reinit_completion(&mc->chanlist[flow].complete);
mc->chanlist[flow].status = 0;
- writel(mc->chanlist[flow].t_phy | 2, mc->base + (flow << 2));
+ writel(mc->chanlist[flow].t_phy | 2, mc->base + ((mc->pdata->descs_reg + flow) << 2));
wait_for_completion_interruptible_timeout(&mc->chanlist[flow].complete,
msecs_to_jiffies(500));
if (mc->chanlist[flow].status == 0) {
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 35ec64df5b3a..e9e733ed98e0 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -18,6 +18,7 @@
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/of.h>
+#include <linux/of_device.h>
#include <linux/platform_device.h>
#include "amlogic-gxl.h"
@@ -30,9 +31,10 @@ static irqreturn_t meson_irq_handler(int irq, void *data)
for (flow = 0; flow < mc->flow_cnt; flow++) {
if (mc->chanlist[flow].irq == irq) {
- p = readl(mc->base + ((0x04 + flow) << 2));
+ p = readl(mc->base + ((mc->pdata->status_reg + flow) << 2));
if (p) {
- writel_relaxed(0xF, mc->base + ((0x4 + flow) << 2));
+ writel_relaxed(0xF, mc->base +
+ ((mc->pdata->status_reg + flow) << 2));
mc->chanlist[flow].status = 1;
complete(&mc->chanlist[flow].complete);
return IRQ_HANDLED;
@@ -254,6 +256,10 @@ static int meson_crypto_probe(struct platform_device *pdev)
if (!mc)
return -ENOMEM;
+ mc->pdata = of_device_get_match_data(&pdev->dev);
+ if (!mc->pdata)
+ return -EINVAL;
+
mc->dev = &pdev->dev;
platform_set_drvdata(pdev, mc);
@@ -319,9 +325,17 @@ static void meson_crypto_remove(struct platform_device *pdev)
clk_disable_unprepare(mc->busclk);
}
+static const struct meson_pdata meson_gxl_pdata = {
+ .descs_reg = 0x0,
+ .status_reg = 0x4,
+};
+
static const struct of_device_id meson_crypto_of_match_table[] = {
- { .compatible = "amlogic,gxl-crypto", },
- {}
+ {
+ .compatible = "amlogic,gxl-crypto",
+ .data = &meson_gxl_pdata,
+ },
+ {},
};
MODULE_DEVICE_TABLE(of, meson_crypto_of_match_table);
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index 79177cfa8b88..9ad75da214ff 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -78,6 +78,16 @@ struct meson_flow {
#endif
};
+/*
+ * struct meson_pdata - SoC series dependent data.
+ * @reg_descs: offset to descriptors register
+ * @reg_status: offset to status register
+ */
+struct meson_pdata {
+ u32 descs_reg;
+ u32 status_reg;
+};
+
/*
* struct meson_dev - main container for all this driver information
* @base: base address of amlogic-crypto
@@ -93,6 +103,7 @@ struct meson_dev {
void __iomem *base;
struct clk *busclk;
struct device *dev;
+ const struct meson_pdata *pdata;
struct meson_flow *chanlist;
atomic_t flow;
int flow_cnt;
--
2.34.1
Because that is proper place for them. In particular,
it takes less of exported symbol between compiling entities.
Signed-off-by: Alexey Romanov <[email protected]>
Reviewed-by: Neil Armstrong <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 98 +++++++++++++++--
drivers/crypto/amlogic/amlogic-gxl-core.c | 110 ++++----------------
drivers/crypto/amlogic/amlogic-gxl.h | 14 +--
3 files changed, 119 insertions(+), 103 deletions(-)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index dc0b100c5de2..bc3092a8a2c2 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -271,7 +271,7 @@ int meson_handle_cipher_request(struct crypto_engine *engine, void *areq)
return 0;
}
-int meson_skdecrypt(struct skcipher_request *areq)
+static int meson_skdecrypt(struct skcipher_request *areq)
{
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
@@ -289,7 +289,7 @@ int meson_skdecrypt(struct skcipher_request *areq)
return crypto_transfer_skcipher_request_to_engine(engine, areq);
}
-int meson_skencrypt(struct skcipher_request *areq)
+static int meson_skencrypt(struct skcipher_request *areq)
{
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
@@ -307,7 +307,7 @@ int meson_skencrypt(struct skcipher_request *areq)
return crypto_transfer_skcipher_request_to_engine(engine, areq);
}
-int meson_cipher_init(struct crypto_tfm *tfm)
+static int meson_cipher_init(struct crypto_tfm *tfm)
{
struct meson_cipher_tfm_ctx *op = crypto_tfm_ctx(tfm);
struct meson_alg_template *algt;
@@ -333,7 +333,7 @@ int meson_cipher_init(struct crypto_tfm *tfm)
return 0;
}
-void meson_cipher_exit(struct crypto_tfm *tfm)
+static void meson_cipher_exit(struct crypto_tfm *tfm)
{
struct meson_cipher_tfm_ctx *op = crypto_tfm_ctx(tfm);
@@ -341,8 +341,8 @@ void meson_cipher_exit(struct crypto_tfm *tfm)
crypto_free_skcipher(op->fallback_tfm);
}
-int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
- unsigned int keylen)
+static int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
+ unsigned int keylen)
{
struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
struct meson_dev *mc = op->mc;
@@ -369,3 +369,89 @@ int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
return crypto_skcipher_setkey(op->fallback_tfm, key, keylen);
}
+
+static struct meson_alg_template algs[] = {
+{
+ .type = CRYPTO_ALG_TYPE_SKCIPHER,
+ .blockmode = MESON_OPMODE_CBC,
+ .alg.skcipher.base = {
+ .base = {
+ .cra_name = "cbc(aes)",
+ .cra_driver_name = "cbc-aes-gxl",
+ .cra_priority = 400,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |
+ CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY |
+ CRYPTO_ALG_NEED_FALLBACK,
+ .cra_ctxsize = sizeof(struct meson_cipher_tfm_ctx),
+ .cra_module = THIS_MODULE,
+ .cra_alignmask = 0xf,
+ .cra_init = meson_cipher_init,
+ .cra_exit = meson_cipher_exit,
+ },
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .ivsize = AES_BLOCK_SIZE,
+ .setkey = meson_aes_setkey,
+ .encrypt = meson_skencrypt,
+ .decrypt = meson_skdecrypt,
+ },
+ .alg.skcipher.op = {
+ .do_one_request = meson_handle_cipher_request,
+ },
+},
+{
+ .type = CRYPTO_ALG_TYPE_SKCIPHER,
+ .blockmode = MESON_OPMODE_ECB,
+ .alg.skcipher.base = {
+ .base = {
+ .cra_name = "ecb(aes)",
+ .cra_driver_name = "ecb-aes-gxl",
+ .cra_priority = 400,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |
+ CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY |
+ CRYPTO_ALG_NEED_FALLBACK,
+ .cra_ctxsize = sizeof(struct meson_cipher_tfm_ctx),
+ .cra_module = THIS_MODULE,
+ .cra_alignmask = 0xf,
+ .cra_init = meson_cipher_init,
+ .cra_exit = meson_cipher_exit,
+ },
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .setkey = meson_aes_setkey,
+ .encrypt = meson_skencrypt,
+ .decrypt = meson_skdecrypt,
+ },
+ .alg.skcipher.op = {
+ .do_one_request = meson_handle_cipher_request,
+ },
+},
+};
+
+int meson_cipher_register(struct meson_dev *mc)
+{
+ return meson_register_algs(mc, algs, ARRAY_SIZE(algs));
+}
+
+void meson_cipher_unregister(struct meson_dev *mc)
+{
+ meson_unregister_algs(mc, algs, ARRAY_SIZE(algs));
+}
+
+void meson_cipher_debugfs_show(struct seq_file *seq, void *v)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(algs); i++) {
+ seq_printf(seq, "%s %s %lu %lu\n",
+ algs[i].alg.skcipher.base.base.cra_driver_name,
+ algs[i].alg.skcipher.base.base.cra_name,
+#ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
+ algs[i].stat_req, algs[i].stat_fb);
+#else
+ 0ul, 0ul);
+#endif
+ }
+}
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 882482438ba2..95443945bdc1 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -65,66 +65,6 @@ static irqreturn_t meson_irq_handler(int irq, void *data)
return IRQ_HANDLED;
}
-static struct meson_alg_template mc_algs[] = {
-{
- .type = CRYPTO_ALG_TYPE_SKCIPHER,
- .blockmode = MESON_OPMODE_CBC,
- .alg.skcipher.base = {
- .base = {
- .cra_name = "cbc(aes)",
- .cra_driver_name = "cbc-aes-gxl",
- .cra_priority = 400,
- .cra_blocksize = AES_BLOCK_SIZE,
- .cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |
- CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY |
- CRYPTO_ALG_NEED_FALLBACK,
- .cra_ctxsize = sizeof(struct meson_cipher_tfm_ctx),
- .cra_module = THIS_MODULE,
- .cra_alignmask = 0xf,
- .cra_init = meson_cipher_init,
- .cra_exit = meson_cipher_exit,
- },
- .min_keysize = AES_MIN_KEY_SIZE,
- .max_keysize = AES_MAX_KEY_SIZE,
- .ivsize = AES_BLOCK_SIZE,
- .setkey = meson_aes_setkey,
- .encrypt = meson_skencrypt,
- .decrypt = meson_skdecrypt,
- },
- .alg.skcipher.op = {
- .do_one_request = meson_handle_cipher_request,
- },
-},
-{
- .type = CRYPTO_ALG_TYPE_SKCIPHER,
- .blockmode = MESON_OPMODE_ECB,
- .alg.skcipher.base = {
- .base = {
- .cra_name = "ecb(aes)",
- .cra_driver_name = "ecb-aes-gxl",
- .cra_priority = 400,
- .cra_blocksize = AES_BLOCK_SIZE,
- .cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |
- CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY |
- CRYPTO_ALG_NEED_FALLBACK,
- .cra_ctxsize = sizeof(struct meson_cipher_tfm_ctx),
- .cra_module = THIS_MODULE,
- .cra_alignmask = 0xf,
- .cra_init = meson_cipher_init,
- .cra_exit = meson_cipher_exit,
- },
- .min_keysize = AES_MIN_KEY_SIZE,
- .max_keysize = AES_MAX_KEY_SIZE,
- .setkey = meson_aes_setkey,
- .encrypt = meson_skencrypt,
- .decrypt = meson_skdecrypt,
- },
- .alg.skcipher.op = {
- .do_one_request = meson_handle_cipher_request,
- },
-},
-};
-
static int meson_debugfs_show(struct seq_file *seq, void *v)
{
struct meson_dev *mc __maybe_unused = seq->private;
@@ -138,20 +78,8 @@ static int meson_debugfs_show(struct seq_file *seq, void *v)
0ul);
#endif
- for (i = 0; i < ARRAY_SIZE(mc_algs); i++) {
- switch (mc_algs[i].type) {
- case CRYPTO_ALG_TYPE_SKCIPHER:
- seq_printf(seq, "%s %s %lu %lu\n",
- mc_algs[i].alg.skcipher.base.base.cra_driver_name,
- mc_algs[i].alg.skcipher.base.base.cra_name,
-#ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
- mc_algs[i].stat_req, mc_algs[i].stat_fb);
-#else
- 0ul, 0ul);
-#endif
- break;
- }
- }
+ meson_cipher_debugfs_show(seq, v);
+
return 0;
}
DEFINE_SHOW_ATTRIBUTE(meson_debugfs);
@@ -228,19 +156,20 @@ static int meson_allocate_chanlist(struct meson_dev *mc)
return err;
}
-static int meson_register_algs(struct meson_dev *mc)
+int meson_register_algs(struct meson_dev *mc, struct meson_alg_template *algs,
+ unsigned int count)
{
int err, i;
- for (i = 0; i < ARRAY_SIZE(mc_algs); i++) {
- mc_algs[i].mc = mc;
- switch (mc_algs[i].type) {
+ for (i = 0; i < count; i++) {
+ algs[i].mc = mc;
+ switch (algs[i].type) {
case CRYPTO_ALG_TYPE_SKCIPHER:
- err = crypto_engine_register_skcipher(&mc_algs[i].alg.skcipher);
+ err = crypto_engine_register_skcipher(&algs[i].alg.skcipher);
if (err) {
dev_err(mc->dev, "Fail to register %s\n",
- mc_algs[i].alg.skcipher.base.base.cra_name);
- mc_algs[i].mc = NULL;
+ algs[i].alg.skcipher.base.base.cra_name);
+ meson_unregister_algs(mc, algs, count);
return err;
}
break;
@@ -250,16 +179,17 @@ static int meson_register_algs(struct meson_dev *mc)
return 0;
}
-static void meson_unregister_algs(struct meson_dev *mc)
+void meson_unregister_algs(struct meson_dev *mc, struct meson_alg_template *algs,
+ unsigned int count)
{
int i;
- for (i = 0; i < ARRAY_SIZE(mc_algs); i++) {
- if (!mc_algs[i].mc)
+ for (i = 0; i < count; i++) {
+ if (!algs[i].mc)
continue;
- switch (mc_algs[i].type) {
+ switch (algs[i].type) {
case CRYPTO_ALG_TYPE_SKCIPHER:
- crypto_engine_unregister_skcipher(&mc_algs[i].alg.skcipher);
+ crypto_engine_unregister_skcipher(&algs[i].alg.skcipher);
break;
}
}
@@ -299,9 +229,9 @@ static int meson_crypto_probe(struct platform_device *pdev)
if (err)
goto error_flow;
- err = meson_register_algs(mc);
+ err = meson_cipher_register(mc);
if (err)
- goto error_alg;
+ goto error_flow;
if (IS_ENABLED(CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG)) {
struct dentry *dbgfs_dir;
@@ -315,8 +245,6 @@ static int meson_crypto_probe(struct platform_device *pdev)
}
return 0;
-error_alg:
- meson_unregister_algs(mc);
error_flow:
meson_free_chanlist(mc, mc->flow_cnt - 1);
return err;
@@ -330,7 +258,7 @@ static void meson_crypto_remove(struct platform_device *pdev)
debugfs_remove_recursive(mc->dbgfs_dir);
#endif
- meson_unregister_algs(mc);
+ meson_cipher_unregister(mc);
meson_free_chanlist(mc, mc->flow_cnt - 1);
}
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index 4d60a0cc2dca..9d66903aa73d 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -165,10 +165,12 @@ void meson_dma_start(struct meson_dev *mc, int flow);
int meson_enqueue(struct crypto_async_request *areq, u32 type);
-int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
- unsigned int keylen);
-int meson_cipher_init(struct crypto_tfm *tfm);
-void meson_cipher_exit(struct crypto_tfm *tfm);
-int meson_skdecrypt(struct skcipher_request *areq);
-int meson_skencrypt(struct skcipher_request *areq);
+int meson_register_algs(struct meson_dev *mc, struct meson_alg_template *algs,
+ unsigned int count);
+void meson_unregister_algs(struct meson_dev *mc, struct meson_alg_template *algs,
+ unsigned int count);
+
+int meson_cipher_register(struct meson_dev *mc);
+void meson_cipher_unregister(struct meson_dev *mc);
+void meson_cipher_debugfs_show(struct seq_file *seq, void *v);
int meson_handle_cipher_request(struct crypto_engine *engine, void *areq);
--
2.34.1
Add MMIO access helpers: meson_dma_start() and meson_dma_ready().
Signed-off-by: Alexey Romanov <[email protected]>
Reviewed-by: Neil Armstrong <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 2 +-
drivers/crypto/amlogic/amlogic-gxl-core.c | 24 ++++++++++++++++-----
drivers/crypto/amlogic/amlogic-gxl.h | 2 ++
3 files changed, 22 insertions(+), 6 deletions(-)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index 7eff3ae7356f..1fe916b0a138 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -225,7 +225,7 @@ static int meson_cipher(struct skcipher_request *areq)
reinit_completion(&mc->chanlist[flow].complete);
mc->chanlist[flow].status = 0;
- writel(mc->chanlist[flow].t_phy | 2, mc->base + ((mc->pdata->descs_reg + flow) << 2));
+ meson_dma_start(mc, flow);
wait_for_completion_interruptible_timeout(&mc->chanlist[flow].complete,
msecs_to_jiffies(500));
if (mc->chanlist[flow].status == 0) {
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index a3a69a59f476..70c2433c56da 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -23,18 +23,32 @@
#include "amlogic-gxl.h"
+void meson_dma_start(struct meson_dev *mc, int flow)
+{
+ u32 offset = (mc->pdata->descs_reg + flow) << 2;
+
+ writel(mc->chanlist[flow].t_phy | 2, mc->base + offset);
+}
+
+static bool meson_dma_ready(struct meson_dev *mc, int flow)
+{
+ u32 offset = (mc->pdata->status_reg + flow) << 2;
+ u32 data = readl(mc->base + offset);
+
+ if (data)
+ writel_relaxed(0xF, mc->base + offset);
+
+ return data;
+}
+
static irqreturn_t meson_irq_handler(int irq, void *data)
{
struct meson_dev *mc = (struct meson_dev *)data;
int flow;
- u32 p;
for (flow = 0; flow < mc->flow_cnt; flow++) {
if (mc->chanlist[flow].irq == irq) {
- p = readl(mc->base + ((mc->pdata->status_reg + flow) << 2));
- if (p) {
- writel_relaxed(0xF, mc->base +
- ((mc->pdata->status_reg + flow) << 2));
+ if (meson_dma_ready(mc, flow)) {
mc->chanlist[flow].status = 1;
complete(&mc->chanlist[flow].complete);
return IRQ_HANDLED;
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index 9ad75da214ff..8670f7ebcdda 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -161,6 +161,8 @@ struct meson_alg_template {
#endif
};
+void meson_dma_start(struct meson_dev *mc, int flow);
+
int meson_enqueue(struct crypto_async_request *areq, u32 type);
int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
--
2.34.1
It is bad to use hardcoded values directly in the code.
Signed-off-by: Alexey Romanov <[email protected]>
Reviewed-by: Neil Armstrong <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 24 ++++++++++-----------
drivers/crypto/amlogic/amlogic-gxl.h | 16 ++++++++------
2 files changed, 21 insertions(+), 19 deletions(-)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index bc3092a8a2c2..c662c4b86e97 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -141,8 +141,8 @@ static int meson_cipher(struct skcipher_request *areq)
ivsize, 0);
}
}
- if (keyivlen == 24)
- keyivlen = 32;
+ if (keyivlen == AES_KEYSIZE_192)
+ keyivlen = AES_MAX_KEY_SIZE;
phykeyiv = dma_map_single(mc->dev, bkeyiv, keyivlen,
DMA_TO_DEVICE);
@@ -161,7 +161,7 @@ static int meson_cipher(struct skcipher_request *areq)
todo = min(keyivlen - eat, 16u);
desc->t_src = cpu_to_le32(phykeyiv + i * 16);
desc->t_dst = cpu_to_le32(i * 16);
- v = (MODE_KEY << 20) | DESC_OWN | 16;
+ v = DESC_MODE_KEY | DESC_OWN | 16;
desc->t_status = cpu_to_le32(v);
eat += todo;
@@ -205,7 +205,7 @@ static int meson_cipher(struct skcipher_request *areq)
desc->t_src = cpu_to_le32(sg_dma_address(src_sg));
desc->t_dst = cpu_to_le32(sg_dma_address(dst_sg));
todo = min(len, sg_dma_len(src_sg));
- v = (op->keymode << 20) | DESC_OWN | todo | (algt->blockmode << 26);
+ v = op->keymode | DESC_OWN | todo | algt->blockmode;
if (rctx->op_dir)
v |= DESC_ENCRYPTION;
len -= todo;
@@ -348,14 +348,14 @@ static int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
struct meson_dev *mc = op->mc;
switch (keylen) {
- case 128 / 8:
- op->keymode = MODE_AES_128;
+ case AES_KEYSIZE_128:
+ op->keymode = DESC_MODE_AES_128;
break;
- case 192 / 8:
- op->keymode = MODE_AES_192;
+ case AES_KEYSIZE_192:
+ op->keymode = DESC_MODE_AES_192;
break;
- case 256 / 8:
- op->keymode = MODE_AES_256;
+ case AES_KEYSIZE_256:
+ op->keymode = DESC_MODE_AES_256;
break;
default:
dev_dbg(mc->dev, "ERROR: Invalid keylen %u\n", keylen);
@@ -373,7 +373,7 @@ static int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
static struct meson_alg_template algs[] = {
{
.type = CRYPTO_ALG_TYPE_SKCIPHER,
- .blockmode = MESON_OPMODE_CBC,
+ .blockmode = DESC_OPMODE_CBC,
.alg.skcipher.base = {
.base = {
.cra_name = "cbc(aes)",
@@ -402,7 +402,7 @@ static struct meson_alg_template algs[] = {
},
{
.type = CRYPTO_ALG_TYPE_SKCIPHER,
- .blockmode = MESON_OPMODE_ECB,
+ .blockmode = DESC_OPMODE_ECB,
.alg.skcipher.base = {
.base = {
.cra_name = "ecb(aes)",
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index 9d66903aa73d..1ab3462dea42 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -11,19 +11,21 @@
#include <linux/crypto.h>
#include <linux/scatterlist.h>
-#define MODE_KEY 1
-#define MODE_AES_128 0x8
-#define MODE_AES_192 0x9
-#define MODE_AES_256 0xa
-
#define MESON_DECRYPT 0
#define MESON_ENCRYPT 1
-#define MESON_OPMODE_ECB 0
-#define MESON_OPMODE_CBC 1
+#define DESC_MODE_KEY (0x1 << 20)
+#define DESC_MODE_AES_128 (0x8 << 20)
+#define DESC_MODE_AES_192 (0x9 << 20)
+#define DESC_MODE_AES_256 (0xa << 20)
#define MAXDESC 64
+#define DESC_OPMODE_ECB (0 << 26)
+#define DESC_OPMODE_CBC (1 << 26)
+
+#define DESC_MAXLEN GENMASK(16, 0)
+
#define DESC_LAST BIT(18)
#define DESC_ENCRYPTION BIT(28)
#define DESC_OWN BIT(31)
--
2.34.1
1. The old alhorithm was not designed to process a large
amount of memory, and therefore gave incorrect results.
2. Not all Amlogic SoC's use 3 KEY/IV descriptors.
Add keyiv descriptors count parameter to platform data.
Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 441 ++++++++++++--------
drivers/crypto/amlogic/amlogic-gxl-core.c | 1 +
drivers/crypto/amlogic/amlogic-gxl.h | 2 +
3 files changed, 280 insertions(+), 164 deletions(-)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index c662c4b86e97..a71b133a9e9a 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -17,35 +17,41 @@
#include <crypto/internal/skcipher.h>
#include "amlogic-gxl.h"
-static bool meson_cipher_need_fallback(struct skcipher_request *areq)
+static bool meson_cipher_need_fallback_sg(struct skcipher_request *areq,
+ struct scatterlist *sg)
{
- struct scatterlist *src_sg = areq->src;
- struct scatterlist *dst_sg = areq->dst;
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
+ unsigned int blocksize = crypto_skcipher_blocksize(tfm);
+ unsigned int cryptlen = areq->cryptlen;
+
+ while (cryptlen) {
+ unsigned int len = min(cryptlen, sg->length);
+
+ if (!IS_ALIGNED(sg->offset, sizeof(u32)))
+ return true;
+ if (len % blocksize != 0)
+ return true;
+
+ cryptlen -= len;
+ sg = sg_next(sg);
+ }
+
+ return false;
+}
+static bool meson_cipher_need_fallback(struct skcipher_request *areq)
+{
if (areq->cryptlen == 0)
return true;
- if (sg_nents(src_sg) != sg_nents(dst_sg))
+ if (meson_cipher_need_fallback_sg(areq, areq->src))
return true;
- /* KEY/IV descriptors use 3 desc */
- if (sg_nents(src_sg) > MAXDESC - 3 || sg_nents(dst_sg) > MAXDESC - 3)
- return true;
+ if (areq->dst == areq->src)
+ return false;
- while (src_sg && dst_sg) {
- if ((src_sg->length % 16) != 0)
- return true;
- if ((dst_sg->length % 16) != 0)
- return true;
- if (src_sg->length != dst_sg->length)
- return true;
- if (!IS_ALIGNED(src_sg->offset, sizeof(u32)))
- return true;
- if (!IS_ALIGNED(dst_sg->offset, sizeof(u32)))
- return true;
- src_sg = sg_next(src_sg);
- dst_sg = sg_next(dst_sg);
- }
+ if (meson_cipher_need_fallback_sg(areq, areq->dst))
+ return true;
return false;
}
@@ -76,184 +82,291 @@ static int meson_cipher_do_fallback(struct skcipher_request *areq)
return err;
}
-static int meson_cipher(struct skcipher_request *areq)
+struct cipher_ctx {
+ struct {
+ dma_addr_t addr;
+ unsigned int len;
+ } keyiv;
+
+ struct skcipher_request *areq;
+ struct scatterlist *src_sg;
+ struct scatterlist *dst_sg;
+ void *bkeyiv;
+
+ unsigned int src_offset;
+ unsigned int dst_offset;
+ unsigned int cryptlen;
+ unsigned int tloffset;
+};
+
+static int meson_map_scatterlist(struct skcipher_request *areq, struct meson_dev *mc)
{
- struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
- struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
- struct meson_cipher_req_ctx *rctx = skcipher_request_ctx(areq);
- struct meson_dev *mc = op->mc;
- struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
- struct meson_alg_template *algt;
- int flow = rctx->flow;
- unsigned int todo, eat, len;
- struct scatterlist *src_sg = areq->src;
- struct scatterlist *dst_sg = areq->dst;
- struct meson_desc *desc;
int nr_sgs, nr_sgd;
- int i, err = 0;
- unsigned int keyivlen, ivsize, offset, tloffset;
- dma_addr_t phykeyiv;
- void *backup_iv = NULL, *bkeyiv;
- u32 v;
-
- algt = container_of(alg, struct meson_alg_template, alg.skcipher.base);
-
- dev_dbg(mc->dev, "%s %s %u %x IV(%u) key=%u flow=%d\n", __func__,
- crypto_tfm_alg_name(areq->base.tfm),
- areq->cryptlen,
- rctx->op_dir, crypto_skcipher_ivsize(tfm),
- op->keylen, flow);
-
-#ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
- algt->stat_req++;
- mc->chanlist[flow].stat_req++;
-#endif
-
- /*
- * The hardware expect a list of meson_desc structures.
- * The 2 first structures store key
- * The third stores IV
- */
- bkeyiv = kzalloc(48, GFP_KERNEL | GFP_DMA);
- if (!bkeyiv)
- return -ENOMEM;
-
- memcpy(bkeyiv, op->key, op->keylen);
- keyivlen = op->keylen;
-
- ivsize = crypto_skcipher_ivsize(tfm);
- if (areq->iv && ivsize > 0) {
- if (ivsize > areq->cryptlen) {
- dev_err(mc->dev, "invalid ivsize=%d vs len=%d\n", ivsize, areq->cryptlen);
- err = -EINVAL;
- goto theend;
- }
- memcpy(bkeyiv + 32, areq->iv, ivsize);
- keyivlen = 48;
- if (rctx->op_dir == MESON_DECRYPT) {
- backup_iv = kzalloc(ivsize, GFP_KERNEL);
- if (!backup_iv) {
- err = -ENOMEM;
- goto theend;
- }
- offset = areq->cryptlen - ivsize;
- scatterwalk_map_and_copy(backup_iv, areq->src, offset,
- ivsize, 0);
- }
- }
- if (keyivlen == AES_KEYSIZE_192)
- keyivlen = AES_MAX_KEY_SIZE;
-
- phykeyiv = dma_map_single(mc->dev, bkeyiv, keyivlen,
- DMA_TO_DEVICE);
- err = dma_mapping_error(mc->dev, phykeyiv);
- if (err) {
- dev_err(mc->dev, "Cannot DMA MAP KEY IV\n");
- goto theend;
- }
-
- tloffset = 0;
- eat = 0;
- i = 0;
- while (keyivlen > eat) {
- desc = &mc->chanlist[flow].tl[tloffset];
- memset(desc, 0, sizeof(struct meson_desc));
- todo = min(keyivlen - eat, 16u);
- desc->t_src = cpu_to_le32(phykeyiv + i * 16);
- desc->t_dst = cpu_to_le32(i * 16);
- v = DESC_MODE_KEY | DESC_OWN | 16;
- desc->t_status = cpu_to_le32(v);
-
- eat += todo;
- i++;
- tloffset++;
- }
if (areq->src == areq->dst) {
nr_sgs = dma_map_sg(mc->dev, areq->src, sg_nents(areq->src),
DMA_BIDIRECTIONAL);
if (!nr_sgs) {
dev_err(mc->dev, "Invalid SG count %d\n", nr_sgs);
- err = -EINVAL;
- goto theend;
+ return -EINVAL;
}
- nr_sgd = nr_sgs;
} else {
nr_sgs = dma_map_sg(mc->dev, areq->src, sg_nents(areq->src),
DMA_TO_DEVICE);
- if (!nr_sgs || nr_sgs > MAXDESC - 3) {
+ if (!nr_sgs) {
dev_err(mc->dev, "Invalid SG count %d\n", nr_sgs);
- err = -EINVAL;
- goto theend;
+ return -EINVAL;
}
+
nr_sgd = dma_map_sg(mc->dev, areq->dst, sg_nents(areq->dst),
DMA_FROM_DEVICE);
- if (!nr_sgd || nr_sgd > MAXDESC - 3) {
+ if (!nr_sgd) {
dev_err(mc->dev, "Invalid SG count %d\n", nr_sgd);
- err = -EINVAL;
- goto theend;
+ return -EINVAL;
}
}
- src_sg = areq->src;
- dst_sg = areq->dst;
- len = areq->cryptlen;
- while (src_sg) {
- desc = &mc->chanlist[flow].tl[tloffset];
- memset(desc, 0, sizeof(struct meson_desc));
-
- desc->t_src = cpu_to_le32(sg_dma_address(src_sg));
- desc->t_dst = cpu_to_le32(sg_dma_address(dst_sg));
- todo = min(len, sg_dma_len(src_sg));
- v = op->keymode | DESC_OWN | todo | algt->blockmode;
- if (rctx->op_dir)
- v |= DESC_ENCRYPTION;
- len -= todo;
-
- if (!sg_next(src_sg))
- v |= DESC_LAST;
- desc->t_status = cpu_to_le32(v);
- tloffset++;
- src_sg = sg_next(src_sg);
- dst_sg = sg_next(dst_sg);
+ return 0;
+}
+
+static void meson_unmap_scatterlist(struct skcipher_request *areq, struct meson_dev *mc)
+{
+ if (areq->src == areq->dst) {
+ dma_unmap_sg(mc->dev, areq->src, sg_nents(areq->src), DMA_BIDIRECTIONAL);
+ } else {
+ dma_unmap_sg(mc->dev, areq->src, sg_nents(areq->src), DMA_TO_DEVICE);
+ dma_unmap_sg(mc->dev, areq->dst, sg_nents(areq->dst), DMA_FROM_DEVICE);
+ }
+}
+
+static void meson_setup_keyiv_descs(struct cipher_ctx *ctx)
+{
+ struct meson_cipher_req_ctx *rctx = skcipher_request_ctx(ctx->areq);
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(ctx->areq);
+ struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
+ struct meson_alg_template *algt = container_of(alg,
+ struct meson_alg_template, alg.skcipher.base);
+ struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
+ struct meson_dev *mc = op->mc;
+ unsigned int ivsize = crypto_skcipher_ivsize(tfm);
+ unsigned int blockmode = algt->blockmode;
+ int i;
+
+ if (ctx->tloffset)
+ return;
+
+ if (blockmode == DESC_OPMODE_CBC) {
+ memcpy(ctx->bkeyiv + AES_MAX_KEY_SIZE, ctx->areq->iv, ivsize);
+ ctx->keyiv.len = AES_MAX_KEY_SIZE + ivsize;
+ dma_sync_single_for_device(mc->dev, ctx->keyiv.addr,
+ ctx->keyiv.len, DMA_TO_DEVICE);
}
- reinit_completion(&mc->chanlist[flow].complete);
- meson_dma_start(mc, flow);
+ for (i = 0; i < mc->pdata->setup_desc_cnt; i++) {
+ struct meson_desc *desc =
+ &mc->chanlist[rctx->flow].tl[ctx->tloffset];
+ int offset = i * 16;
+
+ desc->t_src = cpu_to_le32(ctx->keyiv.addr + offset);
+ desc->t_dst = cpu_to_le32(offset);
+ desc->t_status = cpu_to_le32(DESC_OWN | DESC_MODE_KEY | ctx->keyiv.len);
+
+ ctx->tloffset++;
+ }
+}
- err = wait_for_completion_interruptible_timeout(&mc->chanlist[flow].complete,
+static bool meson_setup_data_descs(struct cipher_ctx *ctx)
+{
+ struct meson_cipher_req_ctx *rctx = skcipher_request_ctx(ctx->areq);
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(ctx->areq);
+ struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
+ struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
+ struct meson_alg_template *algt = container_of(alg,
+ struct meson_alg_template,
+ alg.skcipher.base);
+ struct meson_dev *mc = op->mc;
+ struct meson_desc *desc = &mc->chanlist[rctx->flow].tl[ctx->tloffset];
+ unsigned int blocksize = crypto_skcipher_blocksize(tfm);
+ unsigned int blockmode = algt->blockmode;
+ unsigned int maxlen = rounddown(DESC_MAXLEN, blocksize);
+ unsigned int todo;
+ u32 v;
+
+ ctx->tloffset++;
+
+ todo = min(ctx->cryptlen, maxlen);
+ todo = min(todo, ctx->cryptlen);
+ todo = min(todo, sg_dma_len(ctx->src_sg) - ctx->src_offset);
+ todo = min(todo, sg_dma_len(ctx->dst_sg) - ctx->dst_offset);
+
+ desc->t_src = cpu_to_le32(sg_dma_address(ctx->src_sg) + ctx->src_offset);
+ desc->t_dst = cpu_to_le32(sg_dma_address(ctx->dst_sg) + ctx->dst_offset);
+
+ ctx->cryptlen -= todo;
+ ctx->src_offset += todo;
+ ctx->dst_offset += todo;
+
+ v = DESC_OWN | blockmode | op->keymode | todo;
+ if (rctx->op_dir == MESON_ENCRYPT)
+ v |= DESC_ENCRYPTION;
+
+ if (!ctx->cryptlen || ctx->tloffset == MAXDESC)
+ v |= DESC_LAST;
+
+ desc->t_status = cpu_to_le32(v);
+
+ return v & DESC_LAST;
+}
+
+static int meson_kick_hardware(struct cipher_ctx *ctx)
+{
+ struct meson_cipher_req_ctx *rctx = skcipher_request_ctx(ctx->areq);
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(ctx->areq);
+ struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
+ struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
+ struct meson_alg_template *algt = container_of(alg,
+ struct meson_alg_template,
+ alg.skcipher.base);
+ struct meson_dev *mc = op->mc;
+ unsigned int ivsize = crypto_skcipher_ivsize(tfm);
+ unsigned int blockmode = algt->blockmode;
+ enum dma_data_direction new_iv_dir;
+ dma_addr_t new_iv_phys;
+ void *new_iv;
+ int err;
+
+ if (blockmode == DESC_OPMODE_CBC) {
+ struct scatterlist *sg;
+ unsigned int offset;
+
+ if (rctx->op_dir == MESON_ENCRYPT) {
+ sg = ctx->dst_sg;
+ offset = ctx->dst_offset;
+ new_iv_dir = DMA_FROM_DEVICE;
+ } else {
+ sg = ctx->src_sg;
+ offset = ctx->src_offset;
+ new_iv_dir = DMA_TO_DEVICE;
+ }
+
+ if (ctx->areq->src == ctx->areq->dst)
+ new_iv_dir = DMA_BIDIRECTIONAL;
+
+ offset -= ivsize;
+ new_iv = sg_virt(sg) + offset;
+ new_iv_phys = sg_dma_address(sg) + offset;
+ }
+
+ if (blockmode == DESC_OPMODE_CBC &&
+ rctx->op_dir == MESON_DECRYPT) {
+ dma_sync_single_for_cpu(mc->dev, new_iv_phys,
+ ivsize, new_iv_dir);
+ memcpy(ctx->areq->iv, new_iv, ivsize);
+ }
+
+ reinit_completion(&mc->chanlist[rctx->flow].complete);
+ meson_dma_start(mc, rctx->flow);
+ err = wait_for_completion_interruptible_timeout(&mc->chanlist[rctx->flow].complete,
msecs_to_jiffies(500));
if (err == 0) {
- dev_err(mc->dev, "DMA timeout for flow %d\n", flow);
- err = -EINVAL;
+ dev_err(mc->dev, "DMA timeout for flow %d\n", rctx->flow);
+ return -EINVAL;
} else if (err < 0) {
dev_err(mc->dev, "Waiting for DMA completion is failed (%d)\n", err);
- } else {
- /* No error */
- err = 0;
+ return err;
}
- dma_unmap_single(mc->dev, phykeyiv, keyivlen, DMA_TO_DEVICE);
+ if (blockmode == DESC_OPMODE_CBC &&
+ rctx->op_dir == MESON_ENCRYPT) {
+ dma_sync_single_for_cpu(mc->dev, new_iv_phys,
+ ivsize, new_iv_dir);
+ memcpy(ctx->areq->iv, new_iv, ivsize);
+ }
- if (areq->src == areq->dst) {
- dma_unmap_sg(mc->dev, areq->src, sg_nents(areq->src), DMA_BIDIRECTIONAL);
- } else {
- dma_unmap_sg(mc->dev, areq->src, sg_nents(areq->src), DMA_TO_DEVICE);
- dma_unmap_sg(mc->dev, areq->dst, sg_nents(areq->dst), DMA_FROM_DEVICE);
+ ctx->tloffset = 0;
+
+ return 0;
+}
+
+static int meson_cipher(struct skcipher_request *areq)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
+ struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
+ struct meson_cipher_req_ctx *rctx = skcipher_request_ctx(areq);
+ struct meson_dev *mc = op->mc;
+ struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
+ struct meson_alg_template *algt;
+ struct cipher_ctx ctx = {
+ .areq = areq,
+ .src_offset = 0,
+ .dst_offset = 0,
+ .src_sg = areq->src,
+ .dst_sg = areq->dst,
+ .cryptlen = areq->cryptlen,
+ };
+ int err;
+
+ dev_dbg(mc->dev, "%s %s %u %x IV(%u) key=%u ctx.flow=%d\n", __func__,
+ crypto_tfm_alg_name(areq->base.tfm),
+ areq->cryptlen,
+ rctx->op_dir, crypto_skcipher_ivsize(tfm),
+ op->keylen, rctx->flow);
+
+ algt = container_of(alg, struct meson_alg_template, alg.skcipher.base);
+
+#ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
+ algt->stat_req++;
+ mc->chanlist[rctx->flow].stat_req++;
+#endif
+
+ ctx.bkeyiv = kzalloc(48, GFP_KERNEL | GFP_DMA);
+ if (!ctx.bkeyiv)
+ return -ENOMEM;
+
+ memcpy(ctx.bkeyiv, op->key, op->keylen);
+ ctx.keyiv.len = op->keylen;
+ if (ctx.keyiv.len == AES_KEYSIZE_192)
+ ctx.keyiv.len = AES_MAX_KEY_SIZE;
+
+ ctx.keyiv.addr = dma_map_single(mc->dev, ctx.bkeyiv, ctx.keyiv.len,
+ DMA_TO_DEVICE);
+ err = dma_mapping_error(mc->dev, ctx.keyiv.addr);
+ if (err) {
+ dev_err(mc->dev, "Cannot DMA MAP KEY IV\n");
+ goto theend;
}
- if (areq->iv && ivsize > 0) {
- if (rctx->op_dir == MESON_DECRYPT) {
- memcpy(areq->iv, backup_iv, ivsize);
- } else {
- scatterwalk_map_and_copy(areq->iv, areq->dst,
- areq->cryptlen - ivsize,
- ivsize, 0);
+ err = meson_map_scatterlist(areq, mc);
+ if (err)
+ goto theend;
+
+ ctx.tloffset = 0;
+
+ while (ctx.cryptlen) {
+ meson_setup_keyiv_descs(&ctx);
+
+ if (meson_setup_data_descs(&ctx)) {
+ err = meson_kick_hardware(&ctx);
+ if (err)
+ break;
+ }
+
+ if (ctx.src_offset == sg_dma_len(ctx.src_sg)) {
+ ctx.src_offset = 0;
+ ctx.src_sg = sg_next(ctx.src_sg);
+ }
+
+ if (ctx.dst_offset == sg_dma_len(ctx.dst_sg)) {
+ ctx.dst_offset = 0;
+ ctx.dst_sg = sg_next(ctx.dst_sg);
}
}
+
+ dma_unmap_single(mc->dev, ctx.keyiv.addr, ctx.keyiv.len, DMA_TO_DEVICE);
+ meson_unmap_scatterlist(areq, mc);
+
theend:
- kfree_sensitive(bkeyiv);
- kfree_sensitive(backup_iv);
+ kfree_sensitive(ctx.bkeyiv);
return err;
}
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 95443945bdc1..c7e3b402b14e 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -266,6 +266,7 @@ static void meson_crypto_remove(struct platform_device *pdev)
static const struct meson_pdata meson_gxl_pdata = {
.descs_reg = 0x0,
.status_reg = 0x4,
+ .setup_desc_cnt = 3,
};
static const struct of_device_id meson_crypto_of_match_table[] = {
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index 1ab3462dea42..f3455babb52a 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -82,10 +82,12 @@ struct meson_flow {
* struct meson_pdata - SoC series dependent data.
* @reg_descs: offset to descriptors register
* @reg_status: offset to status register
+ * @setup_desc_cnt: number of setup descriptor to configure.
*/
struct meson_pdata {
u32 descs_reg;
u32 status_reg;
+ u32 setup_desc_cnt;
};
/*
--
2.34.1
It makes no sense to allocate memory via kzalloc, we
can use static buffer, speedup data processing and
don't think about kfree() calls.
Signed-off-by: Alexey Romanov <[email protected]>
Reviewed-by: Neil Armstrong <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 26 ++++++++-------------
drivers/crypto/amlogic/amlogic-gxl.h | 6 ++---
2 files changed, 13 insertions(+), 19 deletions(-)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index a71b133a9e9a..8b8f79f08d6e 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -91,7 +91,6 @@ struct cipher_ctx {
struct skcipher_request *areq;
struct scatterlist *src_sg;
struct scatterlist *dst_sg;
- void *bkeyiv;
unsigned int src_offset;
unsigned int dst_offset;
@@ -156,8 +155,7 @@ static void meson_setup_keyiv_descs(struct cipher_ctx *ctx)
return;
if (blockmode == DESC_OPMODE_CBC) {
- memcpy(ctx->bkeyiv + AES_MAX_KEY_SIZE, ctx->areq->iv, ivsize);
- ctx->keyiv.len = AES_MAX_KEY_SIZE + ivsize;
+ memcpy(op->keyiv + AES_MAX_KEY_SIZE, ctx->areq->iv, ivsize);
dma_sync_single_for_device(mc->dev, ctx->keyiv.addr,
ctx->keyiv.len, DMA_TO_DEVICE);
}
@@ -304,6 +302,7 @@ static int meson_cipher(struct skcipher_request *areq)
.dst_sg = areq->dst,
.cryptlen = areq->cryptlen,
};
+ unsigned int ivsize = crypto_skcipher_ivsize(tfm);
int err;
dev_dbg(mc->dev, "%s %s %u %x IV(%u) key=%u ctx.flow=%d\n", __func__,
@@ -319,16 +318,16 @@ static int meson_cipher(struct skcipher_request *areq)
mc->chanlist[rctx->flow].stat_req++;
#endif
- ctx.bkeyiv = kzalloc(48, GFP_KERNEL | GFP_DMA);
- if (!ctx.bkeyiv)
- return -ENOMEM;
-
- memcpy(ctx.bkeyiv, op->key, op->keylen);
ctx.keyiv.len = op->keylen;
if (ctx.keyiv.len == AES_KEYSIZE_192)
ctx.keyiv.len = AES_MAX_KEY_SIZE;
- ctx.keyiv.addr = dma_map_single(mc->dev, ctx.bkeyiv, ctx.keyiv.len,
+ if (algt->blockmode == DESC_OPMODE_CBC) {
+ memcpy(op->keyiv + AES_MAX_KEY_SIZE, areq->iv, ivsize);
+ ctx.keyiv.len = AES_MAX_KEY_SIZE + ivsize;
+ }
+
+ ctx.keyiv.addr = dma_map_single(mc->dev, op->keyiv, ctx.keyiv.len,
DMA_TO_DEVICE);
err = dma_mapping_error(mc->dev, ctx.keyiv.addr);
if (err) {
@@ -366,8 +365,6 @@ static int meson_cipher(struct skcipher_request *areq)
meson_unmap_scatterlist(areq, mc);
theend:
- kfree_sensitive(ctx.bkeyiv);
-
return err;
}
@@ -450,7 +447,6 @@ static void meson_cipher_exit(struct crypto_tfm *tfm)
{
struct meson_cipher_tfm_ctx *op = crypto_tfm_ctx(tfm);
- kfree_sensitive(op->key);
crypto_free_skcipher(op->fallback_tfm);
}
@@ -474,11 +470,9 @@ static int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
dev_dbg(mc->dev, "ERROR: Invalid keylen %u\n", keylen);
return -EINVAL;
}
- kfree_sensitive(op->key);
+
+ memcpy(op->keyiv, key, keylen);
op->keylen = keylen;
- op->key = kmemdup(key, keylen, GFP_KERNEL | GFP_DMA);
- if (!op->key)
- return -ENOMEM;
return crypto_skcipher_setkey(op->fallback_tfm, key, keylen);
}
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index f3455babb52a..dc5218064f99 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -127,15 +127,15 @@ struct meson_cipher_req_ctx {
/*
* struct meson_cipher_tfm_ctx - context for a skcipher TFM
- * @key: pointer to key data
+ * @keyiv: key data
* @keylen: len of the key
* @keymode: The keymode(type and size of key) associated with this TFM
* @mc: pointer to the private data of driver handling this TFM
* @fallback_tfm: pointer to the fallback TFM
*/
struct meson_cipher_tfm_ctx {
- u32 *key;
- u32 keylen;
+ u8 keyiv[AES_MAX_KEY_SIZE + AES_BLOCK_SIZE] ____cacheline_aligned;
+ u32 keylen ____cacheline_aligned;
u32 keymode;
struct meson_dev *mc;
struct crypto_skcipher *fallback_tfm;
--
2.34.1
Introduce support for SHA1/SHA224/SHA256 hash algos.
Tested via tcrypt and custom tests.
Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/Makefile | 2 +-
drivers/crypto/amlogic/amlogic-gxl-core.c | 25 +-
drivers/crypto/amlogic/amlogic-gxl-hasher.c | 460 ++++++++++++++++++++
drivers/crypto/amlogic/amlogic-gxl.h | 51 +++
4 files changed, 536 insertions(+), 2 deletions(-)
create mode 100644 drivers/crypto/amlogic/amlogic-gxl-hasher.c
diff --git a/drivers/crypto/amlogic/Makefile b/drivers/crypto/amlogic/Makefile
index 39057e62c13e..4b6b388b7880 100644
--- a/drivers/crypto/amlogic/Makefile
+++ b/drivers/crypto/amlogic/Makefile
@@ -1,2 +1,2 @@
obj-$(CONFIG_CRYPTO_DEV_AMLOGIC_GXL) += amlogic-gxl-crypto.o
-amlogic-gxl-crypto-y := amlogic-gxl-core.o amlogic-gxl-cipher.o
+amlogic-gxl-crypto-y := amlogic-gxl-core.o amlogic-gxl-cipher.o amlogic-gxl-hasher.o
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index c7e3b402b14e..c5ee40196801 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -20,6 +20,9 @@
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/platform_device.h>
+#include <crypto/internal/skcipher.h>
+#include <crypto/internal/hash.h>
+#include <linux/dma-mapping.h>
#include "amlogic-gxl.h"
@@ -173,6 +176,15 @@ int meson_register_algs(struct meson_dev *mc, struct meson_alg_template *algs,
return err;
}
break;
+ case CRYPTO_ALG_TYPE_AHASH:
+ err = crypto_engine_register_ahash(&algs[i].alg.ahash);
+ if (err) {
+ dev_err(mc->dev, "Fail to register %s\n",
+ algs[i].alg.ahash.base.halg.base.cra_name);
+ meson_unregister_algs(mc, algs, count);
+ return err;
+ }
+ break;
}
}
@@ -191,6 +203,9 @@ void meson_unregister_algs(struct meson_dev *mc, struct meson_alg_template *algs
case CRYPTO_ALG_TYPE_SKCIPHER:
crypto_engine_unregister_skcipher(&algs[i].alg.skcipher);
break;
+ case CRYPTO_ALG_TYPE_AHASH:
+ crypto_engine_unregister_ahash(&algs[i].alg.ahash);
+ break;
}
}
}
@@ -238,13 +253,20 @@ static int meson_crypto_probe(struct platform_device *pdev)
dbgfs_dir = debugfs_create_dir("gxl-crypto", NULL);
debugfs_create_file("stats", 0444, dbgfs_dir, mc, &meson_debugfs_fops);
-
#ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
mc->dbgfs_dir = dbgfs_dir;
#endif
}
+ err = meson_hasher_register(mc);
+ if (err)
+ goto error_hasher;
+
return 0;
+
+error_hasher:
+ meson_cipher_unregister(mc);
+
error_flow:
meson_free_chanlist(mc, mc->flow_cnt - 1);
return err;
@@ -267,6 +289,7 @@ static const struct meson_pdata meson_gxl_pdata = {
.descs_reg = 0x0,
.status_reg = 0x4,
.setup_desc_cnt = 3,
+ .hasher_supported = false,
};
static const struct of_device_id meson_crypto_of_match_table[] = {
diff --git a/drivers/crypto/amlogic/amlogic-gxl-hasher.c b/drivers/crypto/amlogic/amlogic-gxl-hasher.c
new file mode 100644
index 000000000000..1e7e24729120
--- /dev/null
+++ b/drivers/crypto/amlogic/amlogic-gxl-hasher.c
@@ -0,0 +1,460 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hardware asynchronous hasher for Amlogic SoC's.
+ *
+ * Copyright (c) 2023, SaluteDevices. All Rights Reserved.
+ *
+ * Author: Alexey Romanov <[email protected]>
+ */
+
+#include <linux/crypto.h>
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/err.h>
+#include <crypto/internal/hash.h>
+#include <crypto/sha1.h>
+#include <crypto/sha2.h>
+
+#include "amlogic-gxl.h"
+
+static int meson_sha_init(struct ahash_request *req)
+{
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+
+ memset(rctx, 0, sizeof(struct meson_hasher_req_ctx));
+
+ rctx->flow = meson_get_engine_number(tctx->mc);
+ rctx->begin_req = true;
+
+ return 0;
+}
+
+static int meson_sha_update(struct ahash_request *req)
+{
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+ struct crypto_engine *engine = tctx->mc->chanlist[rctx->flow].engine;
+
+ return crypto_transfer_hash_request_to_engine(engine, req);
+}
+
+static int meson_sha_final(struct ahash_request *req)
+{
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+ struct crypto_engine *engine = tctx->mc->chanlist[rctx->flow].engine;
+
+ rctx->final_req = true;
+
+ return crypto_transfer_hash_request_to_engine(engine, req);
+}
+
+static int meson_hasher_req_map(struct ahash_request *req)
+{
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_dev *mc = tctx->mc;
+ int ret;
+
+ if (!req->nbytes)
+ return 0;
+
+ ret = dma_map_sg(mc->dev, req->src, sg_nents(req->src), DMA_TO_DEVICE);
+ if (!ret) {
+ dev_err(mc->dev, "Cannot DMA MAP request data\n");
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+static void meson_hasher_req_unmap(struct ahash_request *req)
+{
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_dev *mc = tctx->mc;
+
+ if (!req->nbytes)
+ return;
+
+ dma_unmap_sg(mc->dev, req->src, sg_nents(req->src), DMA_TO_DEVICE);
+}
+
+struct hasher_ctx {
+ struct crypto_async_request *areq;
+
+ unsigned int tloffset;
+ unsigned int nbytes;
+ unsigned int todo;
+
+ dma_addr_t state_addr;
+ dma_addr_t src_addr;
+ unsigned int src_offset;
+ struct scatterlist *src_sg;
+};
+
+static bool meson_final(struct hasher_ctx *ctx)
+{
+ struct ahash_request *req = ahash_request_cast(ctx->areq);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+
+ return !ctx->nbytes && rctx->final_req;
+}
+
+static int meson_fill_partial_buffer(struct hasher_ctx *ctx, unsigned int len)
+{
+ struct ahash_request *req = ahash_request_cast(ctx->areq);
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+ struct meson_dev *mc = tctx->mc;
+ unsigned int blocksize = crypto_ahash_blocksize(tfm);
+ unsigned int copy;
+
+ if (len) {
+ copy = min(blocksize - rctx->partial_size, len);
+ memcpy(rctx->partial + rctx->partial_size,
+ sg_virt(ctx->src_sg) + ctx->src_offset, copy);
+
+ rctx->partial_size += copy;
+ ctx->nbytes -= copy;
+ ctx->src_offset += copy;
+ }
+
+ if (rctx->partial_size == blocksize || meson_final(ctx)) {
+ rctx->partial_addr = dma_map_single(mc->dev,
+ rctx->partial,
+ rctx->partial_size,
+ DMA_TO_DEVICE);
+ if (dma_mapping_error(mc->dev, rctx->partial_addr)) {
+ dev_err(mc->dev, "Cannot DMA MAP SHA partial buffer\n");
+ return -ENOMEM;
+ }
+
+ rctx->partial_mapped = true;
+ ctx->todo = rctx->partial_size;
+ ctx->src_addr = rctx->partial_addr;
+ }
+
+ return 0;
+}
+
+static unsigned int meson_setup_data_descs(struct hasher_ctx *ctx)
+{
+ struct ahash_request *req = ahash_request_cast(ctx->areq);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_dev *mc = tctx->mc;
+ struct meson_flow *flow = &mc->chanlist[rctx->flow];
+ struct hash_alg_common *alg = crypto_hash_alg_common(tfm);
+ struct meson_alg_template *algt = container_of(alg,
+ struct meson_alg_template, alg.ahash.base.halg);
+ struct meson_desc *desc = &flow->tl[ctx->tloffset];
+ u32 v;
+
+ ctx->tloffset++;
+
+ v = DESC_OWN | DESC_ENCRYPTION | DESC_OPMODE_SHA |
+ ctx->todo | algt->blockmode;
+ if (rctx->begin_req) {
+ rctx->begin_req = false;
+ v |= DESC_BEGIN;
+ }
+
+ if (!ctx->nbytes && rctx->final_req) {
+ rctx->final_req = false;
+ v |= DESC_END;
+ }
+
+ if (!ctx->nbytes || ctx->tloffset == MAXDESC || rctx->partial_mapped)
+ v |= DESC_LAST;
+
+ desc->t_src = cpu_to_le32(ctx->src_addr);
+ desc->t_dst = cpu_to_le32(ctx->state_addr);
+ desc->t_status = cpu_to_le32(v);
+
+ return v & DESC_LAST;
+}
+
+static int meson_kick_hardware(struct hasher_ctx *ctx)
+{
+ struct ahash_request *req = ahash_request_cast(ctx->areq);
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_dev *mc = tctx->mc;
+ struct meson_flow *flow = &mc->chanlist[rctx->flow];
+ int ret;
+
+ reinit_completion(&flow->complete);
+ meson_dma_start(mc, rctx->flow);
+
+ ret = wait_for_completion_timeout(&flow->complete,
+ msecs_to_jiffies(500));
+ if (ret == 0) {
+ dev_err(mc->dev, "DMA timeout for flow %d\n", rctx->flow);
+ return -EINVAL;
+ } else if (ret < 0) {
+ dev_err(mc->dev, "Waiting for DMA completion is failed (%d)\n", ret);
+ return ret;
+ }
+
+ if (rctx->partial_mapped) {
+ dma_unmap_single(mc->dev, rctx->partial_addr,
+ rctx->partial_size,
+ DMA_TO_DEVICE);
+ rctx->partial_size = 0;
+ rctx->partial_mapped = false;
+ }
+
+ ctx->tloffset = 0;
+
+ return 0;
+}
+
+static void meson_setup_state_descs(struct hasher_ctx *ctx)
+{
+ struct ahash_request *req = ahash_request_cast(ctx->areq);
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_dev *mc = tctx->mc;
+ struct meson_desc *desc;
+ int i;
+
+ if (ctx->tloffset || rctx->begin_req)
+ return;
+
+ for (i = 0; i < mc->pdata->setup_desc_cnt; i++) {
+ int offset = i * 16;
+
+ desc = &mc->chanlist[rctx->flow].tl[ctx->tloffset];
+ desc->t_src = cpu_to_le32(ctx->state_addr + offset);
+ desc->t_dst = cpu_to_le32(offset);
+ desc->t_status = cpu_to_le32(MESON_SHA_BUFFER_SIZE |
+ DESC_MODE_KEY | DESC_OWN);
+
+ ctx->tloffset++;
+ }
+}
+
+static int meson_hasher_do_one_request(struct crypto_engine *engine, void *areq)
+{
+ struct ahash_request *req = ahash_request_cast(areq);
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+ struct meson_dev *mc = tctx->mc;
+ struct hasher_ctx ctx = {
+ .tloffset = 0,
+ .src_offset = 0,
+ .nbytes = req->nbytes,
+ .src_sg = req->src,
+ .areq = areq,
+ };
+ unsigned int blocksize = crypto_ahash_blocksize(tfm);
+ unsigned int digest_size = crypto_ahash_digestsize(tfm);
+ bool final_req = rctx->final_req;
+ int ret;
+
+ ctx.state_addr = dma_map_single(mc->dev, rctx->state,
+ sizeof(rctx->state), DMA_BIDIRECTIONAL);
+ ret = dma_mapping_error(mc->dev, ctx.state_addr);
+ if (ret) {
+ dev_err(mc->dev, "Cannot DMA MAP SHA state buffer");
+ goto fail_map_single;
+ }
+
+ ret = meson_hasher_req_map(req);
+ if (ret)
+ goto fail_map_req;
+
+ for (;;) {
+ unsigned int len = ctx.src_sg ?
+ min(sg_dma_len(ctx.src_sg) - ctx.src_offset, ctx.nbytes) : 0;
+
+ ctx.src_addr = 0;
+ ctx.todo = 0;
+
+ if (!rctx->final_req && !ctx.nbytes)
+ break;
+
+ meson_setup_state_descs(&ctx);
+
+ if (rctx->partial_size && rctx->partial_size < blocksize) {
+ ret = meson_fill_partial_buffer(&ctx, len);
+ if (ret)
+ goto fail;
+ } else if (len && len < blocksize) {
+ memcpy(rctx->partial, sg_virt(ctx.src_sg) + ctx.src_offset, len);
+
+ rctx->partial_size = len;
+ ctx.nbytes -= len;
+ ctx.src_offset += len;
+ } else if (len) {
+ ctx.src_addr = sg_dma_address(ctx.src_sg) + ctx.src_offset;
+ ctx.todo = min(rounddown(DESC_MAXLEN, blocksize),
+ rounddown(len, blocksize));
+ ctx.nbytes -= ctx.todo;
+ ctx.src_offset += ctx.todo;
+ }
+
+ if (ctx.src_sg && ctx.src_offset == sg_dma_len(ctx.src_sg)) {
+ ctx.src_offset = 0;
+ ctx.src_sg = sg_next(ctx.src_sg);
+ }
+
+ if (!ctx.todo && ctx.nbytes)
+ continue;
+
+ if (!ctx.todo && !rctx->final_req && !ctx.tloffset)
+ continue;
+
+ if (meson_setup_data_descs(&ctx)) {
+ ret = meson_kick_hardware(&ctx);
+ if (ret)
+ goto fail;
+ }
+ }
+
+fail:
+ meson_hasher_req_unmap(req);
+
+fail_map_req:
+ dma_unmap_single(mc->dev, ctx.state_addr, sizeof(rctx->state),
+ DMA_BIDIRECTIONAL);
+
+fail_map_single:
+ if (final_req && ret == 0)
+ memcpy(req->result, rctx->state, digest_size);
+
+ local_bh_disable();
+ crypto_finalize_hash_request(engine, req, ret);
+ local_bh_enable();
+
+ return ret;
+}
+
+static int meson_hasher_init(struct crypto_tfm *tfm)
+{
+ struct meson_hasher_tfm_ctx *tctx = crypto_tfm_ctx(tfm);
+ struct crypto_ahash *atfm = __crypto_ahash_cast(tfm);
+ struct hash_alg_common *alg = crypto_hash_alg_common(atfm);
+ struct meson_alg_template *algt = container_of(alg,
+ struct meson_alg_template, alg.ahash.base.halg);
+
+ crypto_ahash_set_reqsize(atfm, crypto_ahash_statesize(atfm));
+
+ memset(tctx, 0, sizeof(struct meson_hasher_tfm_ctx));
+
+ tctx->mc = algt->mc;
+
+ return 0;
+}
+
+static struct meson_alg_template mc_algs[] = {
+{
+ .type = CRYPTO_ALG_TYPE_AHASH,
+ .blockmode = DESC_MODE_SHA1,
+ .alg.ahash.base = {
+ .halg = {
+ .base = {
+ .cra_name = "sha1",
+ .cra_driver_name = "sha1-gxl",
+ .cra_priority = 400,
+ .cra_blocksize = SHA1_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ .cra_ctxsize = sizeof(struct meson_hasher_tfm_ctx),
+ .cra_module = THIS_MODULE,
+ .cra_alignmask = 0,
+ .cra_init = meson_hasher_init,
+ },
+ .digestsize = SHA1_DIGEST_SIZE,
+ .statesize = sizeof(struct meson_hasher_req_ctx),
+ },
+ .init = meson_sha_init,
+ .update = meson_sha_update,
+ .final = meson_sha_final,
+ },
+ .alg.ahash.op = {
+ .do_one_request = meson_hasher_do_one_request,
+ },
+},
+{
+ .type = CRYPTO_ALG_TYPE_AHASH,
+ .blockmode = DESC_MODE_SHA224,
+ .alg.ahash.base = {
+ .halg = {
+ .base = {
+ .cra_name = "sha224",
+ .cra_driver_name = "sha224-gxl",
+ .cra_priority = 400,
+ .cra_blocksize = SHA224_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ .cra_ctxsize = sizeof(struct meson_hasher_tfm_ctx),
+ .cra_module = THIS_MODULE,
+ .cra_alignmask = 0,
+ .cra_init = meson_hasher_init,
+ },
+ .digestsize = SHA224_DIGEST_SIZE,
+ .statesize = sizeof(struct meson_hasher_req_ctx),
+ },
+ .init = meson_sha_init,
+ .update = meson_sha_update,
+ .final = meson_sha_final,
+ },
+ .alg.ahash.op = {
+ .do_one_request = meson_hasher_do_one_request,
+ },
+},
+{
+ .type = CRYPTO_ALG_TYPE_AHASH,
+ .blockmode = DESC_MODE_SHA256,
+ .alg.ahash.base = {
+ .halg = {
+ .base = {
+ .cra_name = "sha256",
+ .cra_driver_name = "sha256-gxl",
+ .cra_priority = 400,
+ .cra_blocksize = SHA256_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ .cra_ctxsize = sizeof(struct meson_hasher_tfm_ctx),
+ .cra_module = THIS_MODULE,
+ .cra_alignmask = 0,
+ .cra_init = meson_hasher_init,
+ },
+ .digestsize = SHA256_DIGEST_SIZE,
+ .statesize = sizeof(struct meson_hasher_req_ctx),
+ },
+ .init = meson_sha_init,
+ .update = meson_sha_update,
+ .final = meson_sha_final,
+ },
+ .alg.ahash.op = {
+ .do_one_request = meson_hasher_do_one_request,
+ },
+},
+};
+
+int meson_hasher_register(struct meson_dev *mc)
+{
+ if (!mc->pdata->hasher_supported) {
+ pr_info("amlogic-gxl-hasher: hasher not supported at current platform");
+ return 0;
+ }
+
+ return meson_register_algs(mc, mc_algs, ARRAY_SIZE(mc_algs));
+}
+
+void meson_hasher_unregister(struct meson_dev *mc)
+{
+ if (!mc->pdata->hasher_supported)
+ return;
+
+ meson_unregister_algs(mc, mc_algs, ARRAY_SIZE(mc_algs));
+}
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index dc5218064f99..cebdfd81f6e2 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -5,6 +5,7 @@
* Copyright (C) 2018-2019 Corentin LABBE <[email protected]>
*/
#include <crypto/aes.h>
+#include <crypto/sha2.h>
#include <crypto/engine.h>
#include <crypto/skcipher.h>
#include <linux/debugfs.h>
@@ -23,13 +24,25 @@
#define DESC_OPMODE_ECB (0 << 26)
#define DESC_OPMODE_CBC (1 << 26)
+#define DESC_OPMODE_SHA (0 << 26)
#define DESC_MAXLEN GENMASK(16, 0)
+#define DESC_MODE_SHA1 (0x5 << 20)
+#define DESC_MODE_SHA224 (0x7 << 20)
+#define DESC_MODE_SHA256 (0x6 << 20)
+
#define DESC_LAST BIT(18)
+#define DESC_BEGIN BIT(24)
+#define DESC_END BIT(25)
#define DESC_ENCRYPTION BIT(28)
#define DESC_OWN BIT(31)
+#define MESON_SHA_BUFFER_SIZE (SHA256_DIGEST_SIZE + 16)
+
+#define MESON_SHA_BEGIN BIT(1)
+#define MESON_SHA_FINAL BIT(2)
+
/*
* struct meson_desc - Descriptor for DMA operations
* Note that without datasheet, some are unknown
@@ -83,11 +96,13 @@ struct meson_flow {
* @reg_descs: offset to descriptors register
* @reg_status: offset to status register
* @setup_desc_cnt: number of setup descriptor to configure.
+ * @hasher_supported: indecates whether hasher is supported.
*/
struct meson_pdata {
u32 descs_reg;
u32 status_reg;
u32 setup_desc_cnt;
+ bool hasher_supported;
};
/*
@@ -141,6 +156,38 @@ struct meson_cipher_tfm_ctx {
struct crypto_skcipher *fallback_tfm;
};
+/*
+ * struct meson_hasher_req_ctx - context for a hasher request
+ * @state: state data
+ * @partial: partial buffer data. Contains sent data which
+ * size < blocksize
+ * @partial_size: size of the partial buffer
+ * @partial_addr: physical address of partial buffer
+ * @partial_mapped: indicates is partial buffer currently mapped or not
+ * @flags: request flags (for example, is this final req or not)
+ * @flow: the flow to use for this request
+ */
+struct meson_hasher_req_ctx {
+ u8 state[SHA256_DIGEST_SIZE + 16] ____cacheline_aligned;
+ u8 partial[SHA256_BLOCK_SIZE] ____cacheline_aligned;
+ unsigned int partial_size ____cacheline_aligned;
+ dma_addr_t partial_addr;
+ bool partial_mapped;
+
+ bool begin_req;
+ bool final_req;
+ int flow;
+};
+
+/*
+ * struct meson_hasher_tfm_ctx - context for a hasher TFM
+ * @enginectx: crypto_engine used by this TFM
+ * @mc: pointer to the private data of driver handling this TFM
+ */
+struct meson_hasher_tfm_ctx {
+ struct meson_dev *mc;
+};
+
/*
* struct meson_alg_template - crypto_alg template
* @type: the CRYPTO_ALG_TYPE for this template
@@ -155,6 +202,7 @@ struct meson_alg_template {
u32 blockmode;
union {
struct skcipher_engine_alg skcipher;
+ struct ahash_engine_alg ahash;
} alg;
struct meson_dev *mc;
#ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
@@ -178,3 +226,6 @@ int meson_cipher_register(struct meson_dev *mc);
void meson_cipher_unregister(struct meson_dev *mc);
void meson_cipher_debugfs_show(struct seq_file *seq, void *v);
int meson_handle_cipher_request(struct crypto_engine *engine, void *areq);
+
+int meson_hasher_register(struct meson_dev *mc);
+void meson_hasher_unregister(struct meson_dev *mc);
--
2.34.1
Unforunately, not all Amlogic SoC's have a 192-bit key
support for AES algo. In this case, use fallback.
Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 7 +++++++
drivers/crypto/amlogic/amlogic-gxl-core.c | 1 +
drivers/crypto/amlogic/amlogic-gxl.h | 2 ++
3 files changed, 10 insertions(+)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index 698868fe2502..3057a5ab3acf 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -41,6 +41,13 @@ static bool meson_cipher_need_fallback_sg(struct skcipher_request *areq,
static bool meson_cipher_need_fallback(struct skcipher_request *areq)
{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
+ struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
+ struct meson_dev *mc = op->mc;
+
+ if (op->keymode == DESC_MODE_AES_192 && !mc->pdata->support_192bit_key)
+ return true;
+
if (areq->cryptlen == 0)
return true;
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index c3cec58cf443..e841ce25de67 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -291,6 +291,7 @@ static const struct meson_pdata meson_gxl_pdata = {
.setup_desc_cnt = 3,
.hasher_supported = false,
.reverse_keyiv = true,
+ .support_192bit_key = true,
};
static const struct of_device_id meson_crypto_of_match_table[] = {
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index de11e174f03d..01913daf1bf4 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -98,6 +98,7 @@ struct meson_flow {
* @reg_status: offset to status register
* @setup_desc_cnt: number of setup descriptor to configure.
* @hasher_supported: indecates whether hasher is supported.
+ * @support_192bit_key: indicates whether platform support AES 192-bit key
*/
struct meson_pdata {
u32 descs_reg;
@@ -105,6 +106,7 @@ struct meson_pdata {
u32 setup_desc_cnt;
bool hasher_supported;
bool reverse_keyiv;
+ bool support_192bit_key;
};
/*
--
2.34.1
Tested via tcrypt module and with custom tests.
Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-core.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index e841ce25de67..a72a76e28c17 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -294,11 +294,24 @@ static const struct meson_pdata meson_gxl_pdata = {
.support_192bit_key = true,
};
+static const struct meson_pdata meson_g12a_pdata = {
+ .descs_reg = 0x0,
+ .status_reg = 0x8,
+ .setup_desc_cnt = 1,
+ .hasher_supported = true,
+ .reverse_keyiv = true,
+ .support_192bit_key = false,
+};
+
static const struct of_device_id meson_crypto_of_match_table[] = {
{
.compatible = "amlogic,gxl-crypto",
.data = &meson_gxl_pdata,
},
+ {
+ .compatible = "amlogic,g12a-crypto",
+ .data = &meson_g12a_pdata,
+ },
{},
};
MODULE_DEVICE_TABLE(of, meson_crypto_of_match_table);
--
2.34.1
Tested via tcrypt module and with custom tests.
Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-core.c | 13 +++++++++++++
1 file changed, 13 insertions(+)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index a72a76e28c17..1ca91cc3fb73 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -303,6 +303,15 @@ static const struct meson_pdata meson_g12a_pdata = {
.support_192bit_key = false,
};
+static const struct meson_pdata meson_axg_pdata = {
+ .descs_reg = 0x0,
+ .status_reg = 0x8,
+ .setup_desc_cnt = 3,
+ .hasher_supported = true,
+ .reverse_keyiv = true,
+ .support_192bit_key = true,
+};
+
static const struct of_device_id meson_crypto_of_match_table[] = {
{
.compatible = "amlogic,gxl-crypto",
@@ -312,6 +321,10 @@ static const struct of_device_id meson_crypto_of_match_table[] = {
.compatible = "amlogic,g12a-crypto",
.data = &meson_g12a_pdata,
},
+ {
+ .compatible = "amlogic,axg-crypto",
+ .data = &meson_axg_pdata,
+ },
{},
};
MODULE_DEVICE_TABLE(of, meson_crypto_of_match_table);
--
2.34.1
This platform data also can be used for S4 as fallback.
Tested via tcrypt module and with custom tests.
Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-core.c | 17 +++++++++++++++++
1 file changed, 17 insertions(+)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 1ca91cc3fb73..20c5924853cc 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -312,6 +312,15 @@ static const struct meson_pdata meson_axg_pdata = {
.support_192bit_key = true,
};
+static const struct meson_pdata meson_a1_pdata = {
+ .descs_reg = 0x0,
+ .status_reg = 0x8,
+ .setup_desc_cnt = 1,
+ .hasher_supported = true,
+ .reverse_keyiv = false,
+ .support_192bit_key = false,
+};
+
static const struct of_device_id meson_crypto_of_match_table[] = {
{
.compatible = "amlogic,gxl-crypto",
@@ -325,6 +334,14 @@ static const struct of_device_id meson_crypto_of_match_table[] = {
.compatible = "amlogic,axg-crypto",
.data = &meson_axg_pdata,
},
+ {
+ .compatible = "amlogic,a1-crypto",
+ .data = &meson_a1_pdata,
+ },
+ {
+ .compatible = "amlogic,s4-crypto",
+ .data = &meson_a1_pdata,
+ },
{},
};
MODULE_DEVICE_TABLE(of, meson_crypto_of_match_table);
--
2.34.1
This patch adds a crypto node declaration. With the
Amlogic crypto driver we can use HW implementation
of SHA1/224/256 and AES algo.
Signed-off-by: Alexey Romanov <[email protected]>
---
arch/arm64/boot/dts/amlogic/meson-a1.dtsi | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/arch/arm64/boot/dts/amlogic/meson-a1.dtsi b/arch/arm64/boot/dts/amlogic/meson-a1.dtsi
index cf150f568335..516b914033ab 100644
--- a/arch/arm64/boot/dts/amlogic/meson-a1.dtsi
+++ b/arch/arm64/boot/dts/amlogic/meson-a1.dtsi
@@ -653,6 +653,13 @@ sd_emmc: sd@10000 {
power-domains = <&pwrc PWRC_SD_EMMC_ID>;
status = "disabled";
};
+
+ crypto: crypto@6000 {
+ compatible = "amlogic,a1-crypto";
+ reg = <0x0 0x6000 0x0 0x48>;
+ interrupts = <GIC_SPI 120 IRQ_TYPE_EDGE_RISING>;
+ power-domains = <&pwrc PWRC_DMA_ID>;
+ };
};
usb: usb@fe004400 {
--
2.34.1
This patch adds a crypto node declaration for Amlogic S4-series.
With the Amlogic crypto driver we can use HW implementation
of SHA1/224/256 and AES algo.
Signed-off-by: Alexey Romanov <[email protected]>
---
arch/arm64/boot/dts/amlogic/meson-s4.dtsi | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/arch/arm64/boot/dts/amlogic/meson-s4.dtsi b/arch/arm64/boot/dts/amlogic/meson-s4.dtsi
index ce90b35686a2..11259d42bfaf 100644
--- a/arch/arm64/boot/dts/amlogic/meson-s4.dtsi
+++ b/arch/arm64/boot/dts/amlogic/meson-s4.dtsi
@@ -339,6 +339,12 @@ mux {
};
+ crypto: crypto@440400 {
+ compatible = "amlogic,s4-crypto", "amlogic,a1-crypto";
+ reg = <0x0 0x440400 0x0 0x48>;
+ interrupts = <GIC_SPI 24 IRQ_TYPE_EDGE_RISING>;
+ };
+
gpio_intc: interrupt-controller@4080 {
compatible = "amlogic,meson-s4-gpio-intc",
"amlogic,meson-gpio-intc";
--
2.34.1
This patch adds a crypto node declaration for Amlogic G12-series.
With the Amlogic crypto driver we can use HW implementation
of SHA1/224/256 and AES algo.
Signed-off-by: Alexey Romanov <[email protected]>
Reviewed-by: Neil Armstrong <[email protected]>
---
arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi b/arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi
index 9d5eab6595d0..50cc75f18ad4 100644
--- a/arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi
+++ b/arch/arm64/boot/dts/amlogic/meson-g12-common.dtsi
@@ -1709,6 +1709,12 @@ internal_ephy: ethernet-phy@8 {
};
};
};
+
+ crypto: crypto@3e000 {
+ compatible = "amlogic,g12a-crypto";
+ reg = <0x0 0x3e000 0x0 0x48>;
+ interrupts = <GIC_SPI 180 IRQ_TYPE_EDGE_RISING>;
+ };
};
aobus: bus@ff800000 {
--
2.34.1
This patch adds a crypto node declaration. With the
Amlogic crypto driver we can use HW implementation
of SHA1/224/256 and AES algo.
Signed-off-by: Alexey Romanov <[email protected]>
Reviewed-by: Neil Armstrong <[email protected]>
---
arch/arm64/boot/dts/amlogic/meson-axg.dtsi | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/arch/arm64/boot/dts/amlogic/meson-axg.dtsi b/arch/arm64/boot/dts/amlogic/meson-axg.dtsi
index 7e5ac9db93f8..39ecb894668e 100644
--- a/arch/arm64/boot/dts/amlogic/meson-axg.dtsi
+++ b/arch/arm64/boot/dts/amlogic/meson-axg.dtsi
@@ -286,6 +286,12 @@ ethmac: ethernet@ff3f0000 {
status = "disabled";
};
+ crypto: crypto@ff63e000 {
+ compatible = "amlogic,axg-crypto";
+ reg = <0x0 0xff63e000 0x0 0x48>;
+ interrupts = <GIC_SPI 180 IRQ_TYPE_EDGE_RISING>;
+ };
+
pcie_phy: phy@ff644000 {
compatible = "amlogic,axg-pcie-phy";
reg = <0x0 0xff644000 0x0 0x1c>;
--
2.34.1
Le Fri, Mar 01, 2024 at 04:29:24PM +0300, Alexey Romanov a ?crit :
> 1. The old alhorithm was not designed to process a large
> amount of memory, and therefore gave incorrect results.
>
> 2. Not all Amlogic SoC's use 3 KEY/IV descriptors.
> Add keyiv descriptors count parameter to platform data.
>
> Signed-off-by: Alexey Romanov <[email protected]>
> ---
> drivers/crypto/amlogic/amlogic-gxl-cipher.c | 441 ++++++++++++--------
> drivers/crypto/amlogic/amlogic-gxl-core.c | 1 +
> drivers/crypto/amlogic/amlogic-gxl.h | 2 +
> 3 files changed, 280 insertions(+), 164 deletions(-)
>
Hello
I have started to test by adding patch one by one and when testing this one I got:
[ 21.674995] gxl-crypto c883e000.crypto: will run requests pump with realtime priority
[ 21.679686] gxl-crypto c883e000.crypto: will run requests pump with realtime priority
[ 21.704366] ------------[ cut here ]------------
[ 21.704480] DMA-API: gxl-crypto c883e000.crypto: device driver tries to sync DMA memory it has not allocated [device address=0x000000000264c000] [size=48 bytes]
[ 21.717684] WARNING: CPU: 1 PID: 263 at kernel/dma/debug.c:1105 check_sync+0x1d0/0x688
[ 21.725512] Modules linked in: meson_rng meson_gxbb_wdt rng_core amlogic_gxl_crypto(+) meson_canvas libphy(+) watchdog ghash_generic gcm xctr xts cts essiv authenc cmac xcbc ccm
[ 21.741211] CPU: 1 PID: 263 Comm: c883e000.crypto Not tainted 6.8.0-rc1-00052-g595d4248b127 #24
[ 21.749834] Hardware name: Libre Computer AML-S905X-CC (DT)
[ 21.755353] pstate: 600000c5 (nZCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 21.762252] pc : check_sync+0x1d0/0x688
[ 21.766047] lr : check_sync+0x1d0/0x688
[ 21.769842] sp : ffff8000820fbb40
[ 21.773119] x29: ffff8000820fbb40 x28: 0000000000000000 x27: 0000000000000000
[ 21.780192] x26: ffff800081d639d0 x25: 0000000000001327 x24: ffff00000288b010
[ 21.787265] x23: 0000000000000050 x22: 0000000000001327 x21: ffff800081c23a20
[ 21.794336] x20: ffff800080fe0bb8 x19: ffff8000820fbbc0 x18: 0000000000000006
[ 21.801410] x17: 645b206465746163 x16: 6f6c6c6120746f6e x15: ffff8000820fb540
[ 21.808482] x14: 000000000000000b x13: ffff800081002748 x12: 0000000000000249
[ 21.815554] x11: 00000000000000c3 x10: ffff80008105a748 x9 : ffff800081002748
[ 21.822626] x8 : 00000000ffffefff x7 : ffff80008105a748 x6 : 80000000fffff000
[ 21.829700] x5 : ffff00007d9abb08 x4 : 0000000000000000 x3 : 0000000000000027
[ 21.836772] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff00000b314ec0
[ 21.843844] Call trace:
[ 21.846260] check_sync+0x1d0/0x688
[ 21.849710] debug_dma_sync_single_for_device+0xb8/0xc0
[ 21.854885] dma_sync_single_for_device+0xc0/0x11c
[ 21.859628] meson_handle_cipher_request+0x49c/0x7c4 [amlogic_gxl_crypto]
[ 21.866356] crypto_pump_work+0x160/0x2ac
[ 21.870324] kthread_worker_fn+0xe4/0x300
[ 21.874291] kthread+0x11c/0x120
[ 21.877483] ret_from_fork+0x10/0x20
[ 21.881020] irq event stamp: 9880
[ 21.884296] hardirqs last enabled at (9879): [<ffff800080988834>] _raw_spin_unlock_irqrestore+0x6c/0x70
[ 21.893698] hardirqs last disabled at (9880): [<ffff800080987ca0>] _raw_spin_lock_irqsave+0x84/0x88
[ 21.902668] softirqs last enabled at (5638): [<ffff800080010674>] __do_softirq+0x494/0x4dc
[ 21.910947] softirqs last disabled at (5627): [<ffff8000800161f0>] ____do_softirq+0x10/0x1c
[ 21.919227] ---[ end trace 0000000000000000 ]---
Regards
On 22/03/2024 09:27, Corentin Labbe wrote:
> Le Fri, Mar 01, 2024 at 04:29:24PM +0300, Alexey Romanov a écrit :
>> 1. The old alhorithm was not designed to process a large
>> amount of memory, and therefore gave incorrect results.
>>
>> 2. Not all Amlogic SoC's use 3 KEY/IV descriptors.
>> Add keyiv descriptors count parameter to platform data.
>>
>> Signed-off-by: Alexey Romanov <[email protected]>
>> ---
>> drivers/crypto/amlogic/amlogic-gxl-cipher.c | 441 ++++++++++++--------
>> drivers/crypto/amlogic/amlogic-gxl-core.c | 1 +
>> drivers/crypto/amlogic/amlogic-gxl.h | 2 +
>> 3 files changed, 280 insertions(+), 164 deletions(-)
>>
>
> Hello
>
> I have started to test by adding patch one by one and when testing this one I got:
> [ 21.674995] gxl-crypto c883e000.crypto: will run requests pump with realtime priority
> [ 21.679686] gxl-crypto c883e000.crypto: will run requests pump with realtime priority
> [ 21.704366] ------------[ cut here ]------------
> [ 21.704480] DMA-API: gxl-crypto c883e000.crypto: device driver tries to sync DMA memory it has not allocated [device address=0x000000000264c000] [size=48 bytes]
> [ 21.717684] WARNING: CPU: 1 PID: 263 at kernel/dma/debug.c:1105 check_sync+0x1d0/0x688
> [ 21.725512] Modules linked in: meson_rng meson_gxbb_wdt rng_core amlogic_gxl_crypto(+) meson_canvas libphy(+) watchdog ghash_generic gcm xctr xts cts essiv authenc cmac xcbc ccm
> [ 21.741211] CPU: 1 PID: 263 Comm: c883e000.crypto Not tainted 6.8.0-rc1-00052-g595d4248b127 #24
> [ 21.749834] Hardware name: Libre Computer AML-S905X-CC (DT)
> [ 21.755353] pstate: 600000c5 (nZCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
> [ 21.762252] pc : check_sync+0x1d0/0x688
> [ 21.766047] lr : check_sync+0x1d0/0x688
> [ 21.769842] sp : ffff8000820fbb40
> [ 21.773119] x29: ffff8000820fbb40 x28: 0000000000000000 x27: 0000000000000000
> [ 21.780192] x26: ffff800081d639d0 x25: 0000000000001327 x24: ffff00000288b010
> [ 21.787265] x23: 0000000000000050 x22: 0000000000001327 x21: ffff800081c23a20
> [ 21.794336] x20: ffff800080fe0bb8 x19: ffff8000820fbbc0 x18: 0000000000000006
> [ 21.801410] x17: 645b206465746163 x16: 6f6c6c6120746f6e x15: ffff8000820fb540
> [ 21.808482] x14: 000000000000000b x13: ffff800081002748 x12: 0000000000000249
> [ 21.815554] x11: 00000000000000c3 x10: ffff80008105a748 x9 : ffff800081002748
> [ 21.822626] x8 : 00000000ffffefff x7 : ffff80008105a748 x6 : 80000000fffff000
> [ 21.829700] x5 : ffff00007d9abb08 x4 : 0000000000000000 x3 : 0000000000000027
> [ 21.836772] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff00000b314ec0
> [ 21.843844] Call trace:
> [ 21.846260] check_sync+0x1d0/0x688
> [ 21.849710] debug_dma_sync_single_for_device+0xb8/0xc0
> [ 21.854885] dma_sync_single_for_device+0xc0/0x11c
> [ 21.859628] meson_handle_cipher_request+0x49c/0x7c4 [amlogic_gxl_crypto]
> [ 21.866356] crypto_pump_work+0x160/0x2ac
> [ 21.870324] kthread_worker_fn+0xe4/0x300
> [ 21.874291] kthread+0x11c/0x120
> [ 21.877483] ret_from_fork+0x10/0x20
> [ 21.881020] irq event stamp: 9880
> [ 21.884296] hardirqs last enabled at (9879): [<ffff800080988834>] _raw_spin_unlock_irqrestore+0x6c/0x70
> [ 21.893698] hardirqs last disabled at (9880): [<ffff800080987ca0>] _raw_spin_lock_irqsave+0x84/0x88
> [ 21.902668] softirqs last enabled at (5638): [<ffff800080010674>] __do_softirq+0x494/0x4dc
> [ 21.910947] softirqs last disabled at (5627): [<ffff8000800161f0>] ____do_softirq+0x10/0x1c
> [ 21.919227] ---[ end trace 0000000000000000 ]---
I confirm I see the same starting with this change with DMA_API_DEBUG=1
[ 4.745240] ------------[ cut here ]------------
[ 4.745295] DMA-API: gxl-crypto c883e000.crypto: device driver tries to sync DMA memory it has not allocated [device address=0x000000000a816030] [size=16 bytes]
[ 4.758496] WARNING: CPU: 3 PID: 75 at kernel/dma/debug.c:1105 check_sync+0x1d8/0x690
[ 4.766242] Modules linked in:
[ 4.769258] CPU: 3 PID: 75 Comm: c883e000.crypto Not tainted 6.8.0-rc6-next-20240229-g99fbe5828431 #167
[ 4.778576] Hardware name: Libre Computer AML-S905X-CC (DT)
[ 4.784094] pstate: 600000c5 (nZCv daIF -PAN -UAO -TCO -DIT -SSBS BTYPE=--)
[ 4.790993] pc : check_sync+0x1d8/0x690
[ 4.794786] lr : check_sync+0x1d8/0x690
[ 4.798581] sp : ffff800082563b90
[ 4.801859] x29: ffff800082563b90 x28: ffff8000816ef298 x27: 0000000000000000
[ 4.808934] x26: ffff0000041ed010 x25: 0000000000000018 x24: 0000000000004000
[ 4.816006] x23: ffff8000816d1190 x22: 0000000000004000 x21: ffff800081539b50
[ 4.823079] x20: ffff800082563c40 x19: ffff8000816ef2b0 x18: 0000000000000030
[ 4.830151] x17: 645b206465746163 x16: 6f6c6c6120746f6e x15: 0720072007200720
[ 4.837223] x14: ffff80008154f2c0 x13: 00000000000002a6 x12: 00000000000000e2
[ 4.844296] x11: 0720072007200720 x10: ffff8000815a72c0 x9 : 00000000fffff000
[ 4.851369] x8 : ffff80008154f2c0 x7 : ffff8000815a72c0 x6 : 0000000000000000
[ 4.858441] x5 : 80000000fffff000 x4 : 000000000000aff5 x3 : 0000000000000000
[ 4.865514] x2 : 0000000000000000 x1 : 0000000000000000 x0 : ffff0000091a9100
[ 4.872587] Call trace:
[ 4.874999] check_sync+0x1d8/0x690
[ 4.878449] debug_dma_sync_single_for_cpu+0x84/0x8c
[ 4.883366] dma_sync_single_for_cpu+0x58/0x10c
[ 4.887851] meson_handle_cipher_request+0x3b0/0x780
[ 4.892767] crypto_pump_work+0x15c/0x2bc
[ 4.896734] kthread_worker_fn+0xcc/0x184
[ 4.900702] kthread+0x118/0x11c
[ 4.903893] ret_from_fork+0x10/0x20
[ 4.907430] ---[ end trace 0000000000000000 ]---
Thanks,
Neil
>
> Regards