2024-02-05 15:55:47

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 00/20] Support more Amlogic SoC families in crypto driver

Hello!

This patchset expand the funcionality of the Amlogic
crypto driver by adding support for more SoC families:
AXG, G12A, G12B, SM1, A1, S4.

Also specify and enable crypto node in device tree
for reference Amlogic devices.

Tested on AXG, G12A/B, SM1, A1 and S4 devices via
custom tests [1] and tcrypt module.

---

Changes V1 -> V2 [2]:

- Rebased over linux-next.
- Adjusted device tree bindings description.
- A1 and S4 dts use their own compatible, which is a G12 fallback.

Changes V2 -> V3 [3]:

- Fix errors in dt-bindings and device tree.
- Add new field in platform data, which determines
whether clock controller should be used for crypto IP.
- Place back MODULE_DEVICE_TABLE.
- Correct commit messages.

Links:
- [1] https://gist.github.com/mRrvz/3fb8943a7487ab7b943ec140706995e7
- [2] https://lore.kernel.org/all/[email protected]/
- [3] https://lore.kernel.org/all/[email protected]/

Alexey Romanov (20):
drivers: crypto: meson: don't hardcode IRQ count
drviers: crypto: meson: add platform data
drivers: crypto: meson: make CLK controller optional
drivers: crypto: meson: add MMIO helpers
drivers: crypto: meson: move get_engine_number()
drivers: crypto: meson: drop status field from meson_flow
drivers: crypto: meson: move algs definition and cipher API to
cipher.c
drivers: crypto: meson: cleanup defines
drivers: crypto: meson: process more than MAXDESCS descriptors
drivers: crypto: meson: avoid kzalloc in engine thread
drivers: crypto: meson: introduce hasher
drivers: crypto: meson: add support for AES-CTR
drivers: crypto: meson: use fallback for 192-bit keys
drivers: crypto: meson: add support for G12-series
drivers: crypto: meson: add support for AXG-series
dt-bindings: crypto: meson: support new SoC's
arch: arm64: dts: meson: a1: add crypto node
arch: arm64: dts: meson: s4: add crypto node
arch: arm64: dts: meson: g12: add crypto node
arch: arm64: dts: meson: axg: add crypto node

.../bindings/crypto/amlogic,gxl-crypto.yaml | 44 +-
arch/arm64/boot/dts/amlogic/meson-a1.dtsi | 7 +
arch/arm64/boot/dts/amlogic/meson-axg.dtsi | 6 +
.../boot/dts/amlogic/meson-g12-common.dtsi | 6 +
arch/arm64/boot/dts/amlogic/meson-s4.dtsi | 6 +
drivers/crypto/amlogic/Makefile | 2 +-
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 602 ++++++++++++------
drivers/crypto/amlogic/amlogic-gxl-core.c | 290 +++++----
drivers/crypto/amlogic/amlogic-gxl-hasher.c | 452 +++++++++++++
drivers/crypto/amlogic/amlogic-gxl.h | 117 +++-
10 files changed, 1183 insertions(+), 349 deletions(-)
create mode 100644 drivers/crypto/amlogic/amlogic-gxl-hasher.c

--
2.34.1



2024-02-05 15:55:54

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 01/20] drivers: crypto: meson: don't hardcode IRQ count

IRQ count is no longer hardcoded, and make it part of
struct meson_flow. We need this for extend driver support for
other Amlogic SoC's.

Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 2 +-
drivers/crypto/amlogic/amlogic-gxl-core.c | 47 ++++++++++++---------
drivers/crypto/amlogic/amlogic-gxl.h | 8 ++--
3 files changed, 31 insertions(+), 26 deletions(-)

diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index 29048da6f50a..b19032f92415 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -19,7 +19,7 @@

static int get_engine_number(struct meson_dev *mc)
{
- return atomic_inc_return(&mc->flow) % MAXFLOW;
+ return atomic_inc_return(&mc->flow) % mc->flow_cnt;
}

static bool meson_cipher_need_fallback(struct skcipher_request *areq)
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index f54ab0d0b1e8..35ec64df5b3a 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -28,8 +28,8 @@ static irqreturn_t meson_irq_handler(int irq, void *data)
int flow;
u32 p;

- for (flow = 0; flow < MAXFLOW; flow++) {
- if (mc->irqs[flow] == irq) {
+ for (flow = 0; flow < mc->flow_cnt; flow++) {
+ if (mc->chanlist[flow].irq == irq) {
p = readl(mc->base + ((0x04 + flow) << 2));
if (p) {
writel_relaxed(0xF, mc->base + ((0x4 + flow) << 2));
@@ -110,7 +110,7 @@ static int meson_debugfs_show(struct seq_file *seq, void *v)
struct meson_dev *mc __maybe_unused = seq->private;
int i;

- for (i = 0; i < MAXFLOW; i++)
+ for (i = 0; i < mc->flow_cnt; i++)
seq_printf(seq, "Channel %d: nreq %lu\n", i,
#ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
mc->chanlist[i].stat_req);
@@ -153,14 +153,32 @@ static void meson_free_chanlist(struct meson_dev *mc, int i)
*/
static int meson_allocate_chanlist(struct meson_dev *mc)
{
+ struct platform_device *pdev = to_platform_device(mc->dev);
int i, err;

- mc->chanlist = devm_kcalloc(mc->dev, MAXFLOW,
+ mc->flow_cnt = platform_irq_count(pdev);
+ if (mc->flow_cnt <= 0) {
+ dev_err(mc->dev, "No IRQs defined\n");
+ return -ENODEV;
+ }
+
+ mc->chanlist = devm_kcalloc(mc->dev, mc->flow_cnt,
sizeof(struct meson_flow), GFP_KERNEL);
if (!mc->chanlist)
return -ENOMEM;

- for (i = 0; i < MAXFLOW; i++) {
+ for (i = 0; i < mc->flow_cnt; i++) {
+ mc->chanlist[i].irq = platform_get_irq(pdev, i);
+ if (mc->chanlist[i].irq < 0)
+ return mc->chanlist[i].irq;
+
+ err = devm_request_irq(mc->dev, mc->chanlist[i].irq,
+ meson_irq_handler, 0, "aml-crypto", mc);
+ if (err < 0) {
+ dev_err(mc->dev, "Cannot request IRQ for flow %d\n", i);
+ return err;
+ }
+
init_completion(&mc->chanlist[i].complete);

mc->chanlist[i].engine = crypto_engine_alloc_init(mc->dev, true);
@@ -230,7 +248,7 @@ static void meson_unregister_algs(struct meson_dev *mc)
static int meson_crypto_probe(struct platform_device *pdev)
{
struct meson_dev *mc;
- int err, i;
+ int err;

mc = devm_kzalloc(&pdev->dev, sizeof(*mc), GFP_KERNEL);
if (!mc)
@@ -252,19 +270,6 @@ static int meson_crypto_probe(struct platform_device *pdev)
return err;
}

- for (i = 0; i < MAXFLOW; i++) {
- mc->irqs[i] = platform_get_irq(pdev, i);
- if (mc->irqs[i] < 0)
- return mc->irqs[i];
-
- err = devm_request_irq(&pdev->dev, mc->irqs[i], meson_irq_handler, 0,
- "gxl-crypto", mc);
- if (err < 0) {
- dev_err(mc->dev, "Cannot request IRQ for flow %d\n", i);
- return err;
- }
- }
-
err = clk_prepare_enable(mc->busclk);
if (err != 0) {
dev_err(&pdev->dev, "Cannot prepare_enable busclk\n");
@@ -294,7 +299,7 @@ static int meson_crypto_probe(struct platform_device *pdev)
error_alg:
meson_unregister_algs(mc);
error_flow:
- meson_free_chanlist(mc, MAXFLOW - 1);
+ meson_free_chanlist(mc, mc->flow_cnt - 1);
clk_disable_unprepare(mc->busclk);
return err;
}
@@ -309,7 +314,7 @@ static void meson_crypto_remove(struct platform_device *pdev)

meson_unregister_algs(mc);

- meson_free_chanlist(mc, MAXFLOW - 1);
+ meson_free_chanlist(mc, mc->flow_cnt - 1);

clk_disable_unprepare(mc->busclk);
}
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index 1013a666c932..79177cfa8b88 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -22,8 +22,6 @@
#define MESON_OPMODE_ECB 0
#define MESON_OPMODE_CBC 1

-#define MAXFLOW 2
-
#define MAXDESC 64

#define DESC_LAST BIT(18)
@@ -62,6 +60,7 @@ struct meson_desc {
* @keylen: keylen for this flow operation
* @complete: completion for the current task on this flow
* @status: set to 1 by interrupt if task is done
+ * @irq: IRQ number for amlogic-crypto
* @t_phy: Physical address of task
* @tl: pointer to the current ce_task for this flow
* @stat_req: number of request done by this flow
@@ -70,6 +69,7 @@ struct meson_flow {
struct crypto_engine *engine;
struct completion complete;
int status;
+ int irq;
unsigned int keylen;
dma_addr_t t_phy;
struct meson_desc *tl;
@@ -85,7 +85,7 @@ struct meson_flow {
* @dev: the platform device
* @chanlist: array of all flow
* @flow: flow to use in next request
- * @irqs: IRQ numbers for amlogic-crypto
+ * @flow_cnt: flow count for amlogic-crypto
* @dbgfs_dir: Debugfs dentry for statistic directory
* @dbgfs_stats: Debugfs dentry for statistic counters
*/
@@ -95,7 +95,7 @@ struct meson_dev {
struct device *dev;
struct meson_flow *chanlist;
atomic_t flow;
- int irqs[MAXFLOW];
+ int flow_cnt;
#ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
struct dentry *dbgfs_dir;
#endif
--
2.34.1


2024-02-05 15:56:00

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 02/20] drviers: crypto: meson: add platform data

To support other Amlogic SoC's we have to
use platform data: descriptors and status registers
offsets are individual for each SoC series.

Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 2 +-
drivers/crypto/amlogic/amlogic-gxl-core.c | 32 +++++++++++++++------
drivers/crypto/amlogic/amlogic-gxl.h | 11 +++++++
3 files changed, 36 insertions(+), 9 deletions(-)

diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index b19032f92415..7eff3ae7356f 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -225,7 +225,7 @@ static int meson_cipher(struct skcipher_request *areq)

reinit_completion(&mc->chanlist[flow].complete);
mc->chanlist[flow].status = 0;
- writel(mc->chanlist[flow].t_phy | 2, mc->base + (flow << 2));
+ writel(mc->chanlist[flow].t_phy | 2, mc->base + ((mc->pdata->descs_reg + flow) << 2));
wait_for_completion_interruptible_timeout(&mc->chanlist[flow].complete,
msecs_to_jiffies(500));
if (mc->chanlist[flow].status == 0) {
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 35ec64df5b3a..4d1b1d5b7a54 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -18,6 +18,7 @@
#include <linux/kernel.h>
#include <linux/module.h>
#include <linux/of.h>
+#include <linux/of_device.h>
#include <linux/platform_device.h>

#include "amlogic-gxl.h"
@@ -30,9 +31,10 @@ static irqreturn_t meson_irq_handler(int irq, void *data)

for (flow = 0; flow < mc->flow_cnt; flow++) {
if (mc->chanlist[flow].irq == irq) {
- p = readl(mc->base + ((0x04 + flow) << 2));
+ p = readl(mc->base + ((mc->pdata->status_reg + flow) << 2));
if (p) {
- writel_relaxed(0xF, mc->base + ((0x4 + flow) << 2));
+ writel_relaxed(0xF, mc->base +
+ ((mc->pdata->status_reg + flow) << 2));
mc->chanlist[flow].status = 1;
complete(&mc->chanlist[flow].complete);
return IRQ_HANDLED;
@@ -245,15 +247,35 @@ static void meson_unregister_algs(struct meson_dev *mc)
}
}

+static const struct meson_pdata meson_gxl_pdata = {
+ .descs_reg = 0x0,
+ .status_reg = 0x4,
+};
+
+static const struct of_device_id meson_crypto_of_match_table[] = {
+ {
+ .compatible = "amlogic,gxl-crypto",
+ .data = &meson_gxl_pdata,
+ },
+ {},
+};
+MODULE_DEVICE_TABLE(of, meson_crypto_of_match_table);
+
static int meson_crypto_probe(struct platform_device *pdev)
{
+ const struct of_device_id *match;
struct meson_dev *mc;
int err;

+ match = of_match_device(meson_crypto_of_match_table, &pdev->dev);
+ if (!match)
+ return -EINVAL;
+
mc = devm_kzalloc(&pdev->dev, sizeof(*mc), GFP_KERNEL);
if (!mc)
return -ENOMEM;

+ mc->pdata = match->data;
mc->dev = &pdev->dev;
platform_set_drvdata(pdev, mc);

@@ -319,12 +341,6 @@ static void meson_crypto_remove(struct platform_device *pdev)
clk_disable_unprepare(mc->busclk);
}

-static const struct of_device_id meson_crypto_of_match_table[] = {
- { .compatible = "amlogic,gxl-crypto", },
- {}
-};
-MODULE_DEVICE_TABLE(of, meson_crypto_of_match_table);
-
static struct platform_driver meson_crypto_driver = {
.probe = meson_crypto_probe,
.remove_new = meson_crypto_remove,
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index 79177cfa8b88..9ad75da214ff 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -78,6 +78,16 @@ struct meson_flow {
#endif
};

+/*
+ * struct meson_pdata - SoC series dependent data.
+ * @reg_descs: offset to descriptors register
+ * @reg_status: offset to status register
+ */
+struct meson_pdata {
+ u32 descs_reg;
+ u32 status_reg;
+};
+
/*
* struct meson_dev - main container for all this driver information
* @base: base address of amlogic-crypto
@@ -93,6 +103,7 @@ struct meson_dev {
void __iomem *base;
struct clk *busclk;
struct device *dev;
+ const struct meson_pdata *pdata;
struct meson_flow *chanlist;
atomic_t flow;
int flow_cnt;
--
2.34.1


2024-02-05 15:56:37

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 03/20] drivers: crypto: meson: make CLK controller optional

Amlogic crypto IP doesn't take a clock input on some
SoCs: AXG / A1 / S4 / G12. So make it optional.

Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-core.c | 21 ++++++++-------------
drivers/crypto/amlogic/amlogic-gxl.h | 2 ++
2 files changed, 10 insertions(+), 13 deletions(-)

diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 4d1b1d5b7a54..54113c524ec5 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -250,6 +250,7 @@ static void meson_unregister_algs(struct meson_dev *mc)
static const struct meson_pdata meson_gxl_pdata = {
.descs_reg = 0x0,
.status_reg = 0x4,
+ .need_clk = true,
};

static const struct of_device_id meson_crypto_of_match_table[] = {
@@ -285,17 +286,14 @@ static int meson_crypto_probe(struct platform_device *pdev)
dev_err(&pdev->dev, "Cannot request MMIO err=%d\n", err);
return err;
}
- mc->busclk = devm_clk_get(&pdev->dev, "blkmv");
- if (IS_ERR(mc->busclk)) {
- err = PTR_ERR(mc->busclk);
- dev_err(&pdev->dev, "Cannot get core clock err=%d\n", err);
- return err;
- }

- err = clk_prepare_enable(mc->busclk);
- if (err != 0) {
- dev_err(&pdev->dev, "Cannot prepare_enable busclk\n");
- return err;
+ if (mc->pdata->need_clk) {
+ mc->busclk = devm_clk_get_enabled(&pdev->dev, "blkmv");
+ if (IS_ERR(mc->busclk)) {
+ err = PTR_ERR(mc->busclk);
+ dev_err(&pdev->dev, "Cannot get and enable core clock err=%d\n", err);
+ return err;
+ }
}

err = meson_allocate_chanlist(mc);
@@ -322,7 +320,6 @@ static int meson_crypto_probe(struct platform_device *pdev)
meson_unregister_algs(mc);
error_flow:
meson_free_chanlist(mc, mc->flow_cnt - 1);
- clk_disable_unprepare(mc->busclk);
return err;
}

@@ -337,8 +334,6 @@ static void meson_crypto_remove(struct platform_device *pdev)
meson_unregister_algs(mc);

meson_free_chanlist(mc, mc->flow_cnt - 1);
-
- clk_disable_unprepare(mc->busclk);
}

static struct platform_driver meson_crypto_driver = {
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index 9ad75da214ff..a36b9bac63a0 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -82,10 +82,12 @@ struct meson_flow {
* struct meson_pdata - SoC series dependent data.
* @reg_descs: offset to descriptors register
* @reg_status: offset to status register
+ * @need_clk: clock input is needed
*/
struct meson_pdata {
u32 descs_reg;
u32 status_reg;
+ bool need_clk;
};

/*
--
2.34.1


2024-02-05 15:56:43

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 07/20] drivers: crypto: meson: move algs definition and cipher API to cipher.c

Because that is proper place for them. In particular,
it takes less of exported symbol between compiling entities.

Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 98 +++++++++++++++--
drivers/crypto/amlogic/amlogic-gxl-core.c | 110 ++++----------------
drivers/crypto/amlogic/amlogic-gxl.h | 14 +--
3 files changed, 119 insertions(+), 103 deletions(-)

diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index dc0b100c5de2..bc3092a8a2c2 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -271,7 +271,7 @@ int meson_handle_cipher_request(struct crypto_engine *engine, void *areq)
return 0;
}

-int meson_skdecrypt(struct skcipher_request *areq)
+static int meson_skdecrypt(struct skcipher_request *areq)
{
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
@@ -289,7 +289,7 @@ int meson_skdecrypt(struct skcipher_request *areq)
return crypto_transfer_skcipher_request_to_engine(engine, areq);
}

-int meson_skencrypt(struct skcipher_request *areq)
+static int meson_skencrypt(struct skcipher_request *areq)
{
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
@@ -307,7 +307,7 @@ int meson_skencrypt(struct skcipher_request *areq)
return crypto_transfer_skcipher_request_to_engine(engine, areq);
}

-int meson_cipher_init(struct crypto_tfm *tfm)
+static int meson_cipher_init(struct crypto_tfm *tfm)
{
struct meson_cipher_tfm_ctx *op = crypto_tfm_ctx(tfm);
struct meson_alg_template *algt;
@@ -333,7 +333,7 @@ int meson_cipher_init(struct crypto_tfm *tfm)
return 0;
}

-void meson_cipher_exit(struct crypto_tfm *tfm)
+static void meson_cipher_exit(struct crypto_tfm *tfm)
{
struct meson_cipher_tfm_ctx *op = crypto_tfm_ctx(tfm);

@@ -341,8 +341,8 @@ void meson_cipher_exit(struct crypto_tfm *tfm)
crypto_free_skcipher(op->fallback_tfm);
}

-int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
- unsigned int keylen)
+static int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
+ unsigned int keylen)
{
struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
struct meson_dev *mc = op->mc;
@@ -369,3 +369,89 @@ int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,

return crypto_skcipher_setkey(op->fallback_tfm, key, keylen);
}
+
+static struct meson_alg_template algs[] = {
+{
+ .type = CRYPTO_ALG_TYPE_SKCIPHER,
+ .blockmode = MESON_OPMODE_CBC,
+ .alg.skcipher.base = {
+ .base = {
+ .cra_name = "cbc(aes)",
+ .cra_driver_name = "cbc-aes-gxl",
+ .cra_priority = 400,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |
+ CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY |
+ CRYPTO_ALG_NEED_FALLBACK,
+ .cra_ctxsize = sizeof(struct meson_cipher_tfm_ctx),
+ .cra_module = THIS_MODULE,
+ .cra_alignmask = 0xf,
+ .cra_init = meson_cipher_init,
+ .cra_exit = meson_cipher_exit,
+ },
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .ivsize = AES_BLOCK_SIZE,
+ .setkey = meson_aes_setkey,
+ .encrypt = meson_skencrypt,
+ .decrypt = meson_skdecrypt,
+ },
+ .alg.skcipher.op = {
+ .do_one_request = meson_handle_cipher_request,
+ },
+},
+{
+ .type = CRYPTO_ALG_TYPE_SKCIPHER,
+ .blockmode = MESON_OPMODE_ECB,
+ .alg.skcipher.base = {
+ .base = {
+ .cra_name = "ecb(aes)",
+ .cra_driver_name = "ecb-aes-gxl",
+ .cra_priority = 400,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |
+ CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY |
+ CRYPTO_ALG_NEED_FALLBACK,
+ .cra_ctxsize = sizeof(struct meson_cipher_tfm_ctx),
+ .cra_module = THIS_MODULE,
+ .cra_alignmask = 0xf,
+ .cra_init = meson_cipher_init,
+ .cra_exit = meson_cipher_exit,
+ },
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .setkey = meson_aes_setkey,
+ .encrypt = meson_skencrypt,
+ .decrypt = meson_skdecrypt,
+ },
+ .alg.skcipher.op = {
+ .do_one_request = meson_handle_cipher_request,
+ },
+},
+};
+
+int meson_cipher_register(struct meson_dev *mc)
+{
+ return meson_register_algs(mc, algs, ARRAY_SIZE(algs));
+}
+
+void meson_cipher_unregister(struct meson_dev *mc)
+{
+ meson_unregister_algs(mc, algs, ARRAY_SIZE(algs));
+}
+
+void meson_cipher_debugfs_show(struct seq_file *seq, void *v)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(algs); i++) {
+ seq_printf(seq, "%s %s %lu %lu\n",
+ algs[i].alg.skcipher.base.base.cra_driver_name,
+ algs[i].alg.skcipher.base.base.cra_name,
+#ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
+ algs[i].stat_req, algs[i].stat_fb);
+#else
+ 0ul, 0ul);
+#endif
+ }
+}
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index f7c60ebffbc3..22ff2768b5e5 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -65,66 +65,6 @@ static irqreturn_t meson_irq_handler(int irq, void *data)
return IRQ_HANDLED;
}

-static struct meson_alg_template mc_algs[] = {
-{
- .type = CRYPTO_ALG_TYPE_SKCIPHER,
- .blockmode = MESON_OPMODE_CBC,
- .alg.skcipher.base = {
- .base = {
- .cra_name = "cbc(aes)",
- .cra_driver_name = "cbc-aes-gxl",
- .cra_priority = 400,
- .cra_blocksize = AES_BLOCK_SIZE,
- .cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |
- CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY |
- CRYPTO_ALG_NEED_FALLBACK,
- .cra_ctxsize = sizeof(struct meson_cipher_tfm_ctx),
- .cra_module = THIS_MODULE,
- .cra_alignmask = 0xf,
- .cra_init = meson_cipher_init,
- .cra_exit = meson_cipher_exit,
- },
- .min_keysize = AES_MIN_KEY_SIZE,
- .max_keysize = AES_MAX_KEY_SIZE,
- .ivsize = AES_BLOCK_SIZE,
- .setkey = meson_aes_setkey,
- .encrypt = meson_skencrypt,
- .decrypt = meson_skdecrypt,
- },
- .alg.skcipher.op = {
- .do_one_request = meson_handle_cipher_request,
- },
-},
-{
- .type = CRYPTO_ALG_TYPE_SKCIPHER,
- .blockmode = MESON_OPMODE_ECB,
- .alg.skcipher.base = {
- .base = {
- .cra_name = "ecb(aes)",
- .cra_driver_name = "ecb-aes-gxl",
- .cra_priority = 400,
- .cra_blocksize = AES_BLOCK_SIZE,
- .cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |
- CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY |
- CRYPTO_ALG_NEED_FALLBACK,
- .cra_ctxsize = sizeof(struct meson_cipher_tfm_ctx),
- .cra_module = THIS_MODULE,
- .cra_alignmask = 0xf,
- .cra_init = meson_cipher_init,
- .cra_exit = meson_cipher_exit,
- },
- .min_keysize = AES_MIN_KEY_SIZE,
- .max_keysize = AES_MAX_KEY_SIZE,
- .setkey = meson_aes_setkey,
- .encrypt = meson_skencrypt,
- .decrypt = meson_skdecrypt,
- },
- .alg.skcipher.op = {
- .do_one_request = meson_handle_cipher_request,
- },
-},
-};
-
static int meson_debugfs_show(struct seq_file *seq, void *v)
{
struct meson_dev *mc __maybe_unused = seq->private;
@@ -138,20 +78,8 @@ static int meson_debugfs_show(struct seq_file *seq, void *v)
0ul);
#endif

- for (i = 0; i < ARRAY_SIZE(mc_algs); i++) {
- switch (mc_algs[i].type) {
- case CRYPTO_ALG_TYPE_SKCIPHER:
- seq_printf(seq, "%s %s %lu %lu\n",
- mc_algs[i].alg.skcipher.base.base.cra_driver_name,
- mc_algs[i].alg.skcipher.base.base.cra_name,
-#ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
- mc_algs[i].stat_req, mc_algs[i].stat_fb);
-#else
- 0ul, 0ul);
-#endif
- break;
- }
- }
+ meson_cipher_debugfs_show(seq, v);
+
return 0;
}
DEFINE_SHOW_ATTRIBUTE(meson_debugfs);
@@ -228,38 +156,40 @@ static int meson_allocate_chanlist(struct meson_dev *mc)
return err;
}

-static int meson_register_algs(struct meson_dev *mc)
+int meson_register_algs(struct meson_dev *mc, struct meson_alg_template *algs,
+ unsigned int count)
{
int err, i;

- for (i = 0; i < ARRAY_SIZE(mc_algs); i++) {
- mc_algs[i].mc = mc;
- switch (mc_algs[i].type) {
+ for (i = 0; i < count; i++) {
+ switch (algs[i].type) {
case CRYPTO_ALG_TYPE_SKCIPHER:
- err = crypto_engine_register_skcipher(&mc_algs[i].alg.skcipher);
+ err = crypto_engine_register_skcipher(&algs[i].alg.skcipher);
if (err) {
dev_err(mc->dev, "Fail to register %s\n",
- mc_algs[i].alg.skcipher.base.base.cra_name);
- mc_algs[i].mc = NULL;
+ algs[i].alg.skcipher.base.base.cra_name);
+ meson_unregister_algs(mc, algs, count);
return err;
}
break;
}
+ algs[i].mc = mc;
}

return 0;
}

-static void meson_unregister_algs(struct meson_dev *mc)
+void meson_unregister_algs(struct meson_dev *mc, struct meson_alg_template *algs,
+ unsigned int count)
{
int i;

- for (i = 0; i < ARRAY_SIZE(mc_algs); i++) {
- if (!mc_algs[i].mc)
+ for (i = 0; i < count; i++) {
+ if (!algs[i].mc)
continue;
- switch (mc_algs[i].type) {
+ switch (algs[i].type) {
case CRYPTO_ALG_TYPE_SKCIPHER:
- crypto_engine_unregister_skcipher(&mc_algs[i].alg.skcipher);
+ crypto_engine_unregister_skcipher(&algs[i].alg.skcipher);
break;
}
}
@@ -318,9 +248,9 @@ static int meson_crypto_probe(struct platform_device *pdev)
if (err)
goto error_flow;

- err = meson_register_algs(mc);
+ err = meson_cipher_register(mc);
if (err)
- goto error_alg;
+ goto error_flow;

if (IS_ENABLED(CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG)) {
struct dentry *dbgfs_dir;
@@ -334,8 +264,6 @@ static int meson_crypto_probe(struct platform_device *pdev)
}

return 0;
-error_alg:
- meson_unregister_algs(mc);
error_flow:
meson_free_chanlist(mc, mc->flow_cnt - 1);
return err;
@@ -349,7 +277,7 @@ static void meson_crypto_remove(struct platform_device *pdev)
debugfs_remove_recursive(mc->dbgfs_dir);
#endif

- meson_unregister_algs(mc);
+ meson_cipher_unregister(mc);

meson_free_chanlist(mc, mc->flow_cnt - 1);
}
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index e27908992ae3..0a03e8144977 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -167,10 +167,12 @@ void meson_dma_start(struct meson_dev *mc, int flow);

int meson_enqueue(struct crypto_async_request *areq, u32 type);

-int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
- unsigned int keylen);
-int meson_cipher_init(struct crypto_tfm *tfm);
-void meson_cipher_exit(struct crypto_tfm *tfm);
-int meson_skdecrypt(struct skcipher_request *areq);
-int meson_skencrypt(struct skcipher_request *areq);
+int meson_register_algs(struct meson_dev *mc, struct meson_alg_template *algs,
+ unsigned int count);
+void meson_unregister_algs(struct meson_dev *mc, struct meson_alg_template *algs,
+ unsigned int count);
+
+int meson_cipher_register(struct meson_dev *mc);
+void meson_cipher_unregister(struct meson_dev *mc);
+void meson_cipher_debugfs_show(struct seq_file *seq, void *v);
int meson_handle_cipher_request(struct crypto_engine *engine, void *areq);
--
2.34.1


2024-02-05 15:56:52

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 04/20] drivers: crypto: meson: add MMIO helpers

Add MMIO access helpers: meson_dma_start() and meson_dma_ready().

Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 2 +-
drivers/crypto/amlogic/amlogic-gxl-core.c | 24 ++++++++++++++++-----
drivers/crypto/amlogic/amlogic-gxl.h | 2 ++
3 files changed, 22 insertions(+), 6 deletions(-)

diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index 7eff3ae7356f..1fe916b0a138 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -225,7 +225,7 @@ static int meson_cipher(struct skcipher_request *areq)

reinit_completion(&mc->chanlist[flow].complete);
mc->chanlist[flow].status = 0;
- writel(mc->chanlist[flow].t_phy | 2, mc->base + ((mc->pdata->descs_reg + flow) << 2));
+ meson_dma_start(mc, flow);
wait_for_completion_interruptible_timeout(&mc->chanlist[flow].complete,
msecs_to_jiffies(500));
if (mc->chanlist[flow].status == 0) {
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 54113c524ec5..372c30f72072 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -23,18 +23,32 @@

#include "amlogic-gxl.h"

+void meson_dma_start(struct meson_dev *mc, int flow)
+{
+ u32 offset = (mc->pdata->descs_reg + flow) << 2;
+
+ writel(mc->chanlist[flow].t_phy | 2, mc->base + offset);
+}
+
+static bool meson_dma_ready(struct meson_dev *mc, int flow)
+{
+ u32 offset = (mc->pdata->status_reg + flow) << 2;
+ u32 data = readl(mc->base + offset);
+
+ if (data)
+ writel_relaxed(0xF, mc->base + offset);
+
+ return data;
+}
+
static irqreturn_t meson_irq_handler(int irq, void *data)
{
struct meson_dev *mc = (struct meson_dev *)data;
int flow;
- u32 p;

for (flow = 0; flow < mc->flow_cnt; flow++) {
if (mc->chanlist[flow].irq == irq) {
- p = readl(mc->base + ((mc->pdata->status_reg + flow) << 2));
- if (p) {
- writel_relaxed(0xF, mc->base +
- ((mc->pdata->status_reg + flow) << 2));
+ if (meson_dma_ready(mc, flow)) {
mc->chanlist[flow].status = 1;
complete(&mc->chanlist[flow].complete);
return IRQ_HANDLED;
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index a36b9bac63a0..59fc6a67e0a9 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -163,6 +163,8 @@ struct meson_alg_template {
#endif
};

+void meson_dma_start(struct meson_dev *mc, int flow);
+
int meson_enqueue(struct crypto_async_request *areq, u32 type);

int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
--
2.34.1


2024-02-05 15:57:01

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 08/20] drivers: crypto: meson: cleanup defines

It is bad to use hardcoded values directly in the code.

Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 24 ++++++++++-----------
drivers/crypto/amlogic/amlogic-gxl.h | 16 ++++++++------
2 files changed, 21 insertions(+), 19 deletions(-)

diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index bc3092a8a2c2..c662c4b86e97 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -141,8 +141,8 @@ static int meson_cipher(struct skcipher_request *areq)
ivsize, 0);
}
}
- if (keyivlen == 24)
- keyivlen = 32;
+ if (keyivlen == AES_KEYSIZE_192)
+ keyivlen = AES_MAX_KEY_SIZE;

phykeyiv = dma_map_single(mc->dev, bkeyiv, keyivlen,
DMA_TO_DEVICE);
@@ -161,7 +161,7 @@ static int meson_cipher(struct skcipher_request *areq)
todo = min(keyivlen - eat, 16u);
desc->t_src = cpu_to_le32(phykeyiv + i * 16);
desc->t_dst = cpu_to_le32(i * 16);
- v = (MODE_KEY << 20) | DESC_OWN | 16;
+ v = DESC_MODE_KEY | DESC_OWN | 16;
desc->t_status = cpu_to_le32(v);

eat += todo;
@@ -205,7 +205,7 @@ static int meson_cipher(struct skcipher_request *areq)
desc->t_src = cpu_to_le32(sg_dma_address(src_sg));
desc->t_dst = cpu_to_le32(sg_dma_address(dst_sg));
todo = min(len, sg_dma_len(src_sg));
- v = (op->keymode << 20) | DESC_OWN | todo | (algt->blockmode << 26);
+ v = op->keymode | DESC_OWN | todo | algt->blockmode;
if (rctx->op_dir)
v |= DESC_ENCRYPTION;
len -= todo;
@@ -348,14 +348,14 @@ static int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
struct meson_dev *mc = op->mc;

switch (keylen) {
- case 128 / 8:
- op->keymode = MODE_AES_128;
+ case AES_KEYSIZE_128:
+ op->keymode = DESC_MODE_AES_128;
break;
- case 192 / 8:
- op->keymode = MODE_AES_192;
+ case AES_KEYSIZE_192:
+ op->keymode = DESC_MODE_AES_192;
break;
- case 256 / 8:
- op->keymode = MODE_AES_256;
+ case AES_KEYSIZE_256:
+ op->keymode = DESC_MODE_AES_256;
break;
default:
dev_dbg(mc->dev, "ERROR: Invalid keylen %u\n", keylen);
@@ -373,7 +373,7 @@ static int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
static struct meson_alg_template algs[] = {
{
.type = CRYPTO_ALG_TYPE_SKCIPHER,
- .blockmode = MESON_OPMODE_CBC,
+ .blockmode = DESC_OPMODE_CBC,
.alg.skcipher.base = {
.base = {
.cra_name = "cbc(aes)",
@@ -402,7 +402,7 @@ static struct meson_alg_template algs[] = {
},
{
.type = CRYPTO_ALG_TYPE_SKCIPHER,
- .blockmode = MESON_OPMODE_ECB,
+ .blockmode = DESC_OPMODE_ECB,
.alg.skcipher.base = {
.base = {
.cra_name = "ecb(aes)",
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index 0a03e8144977..a0d83c82906d 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -11,19 +11,21 @@
#include <linux/crypto.h>
#include <linux/scatterlist.h>

-#define MODE_KEY 1
-#define MODE_AES_128 0x8
-#define MODE_AES_192 0x9
-#define MODE_AES_256 0xa
-
#define MESON_DECRYPT 0
#define MESON_ENCRYPT 1

-#define MESON_OPMODE_ECB 0
-#define MESON_OPMODE_CBC 1
+#define DESC_MODE_KEY (0x1 << 20)
+#define DESC_MODE_AES_128 (0x8 << 20)
+#define DESC_MODE_AES_192 (0x9 << 20)
+#define DESC_MODE_AES_256 (0xa << 20)

#define MAXDESC 64

+#define DESC_OPMODE_ECB (0 << 26)
+#define DESC_OPMODE_CBC (1 << 26)
+
+#define DESC_MAXLEN ((1 << 17) - 1)
+
#define DESC_LAST BIT(18)
#define DESC_ENCRYPTION BIT(28)
#define DESC_OWN BIT(31)
--
2.34.1


2024-02-05 15:57:23

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 05/20] drivers: crypto: meson: move get_engine_number()

Move get_engine_number() function from cipher.c to core.c

Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 9 ++-------
drivers/crypto/amlogic/amlogic-gxl-core.c | 5 +++++
drivers/crypto/amlogic/amlogic-gxl.h | 2 ++
3 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index 1fe916b0a138..18e9e2d39b1f 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -17,11 +17,6 @@
#include <crypto/internal/skcipher.h>
#include "amlogic-gxl.h"

-static int get_engine_number(struct meson_dev *mc)
-{
- return atomic_inc_return(&mc->flow) % mc->flow_cnt;
-}
-
static bool meson_cipher_need_fallback(struct skcipher_request *areq)
{
struct scatterlist *src_sg = areq->src;
@@ -282,7 +277,7 @@ int meson_skdecrypt(struct skcipher_request *areq)
rctx->op_dir = MESON_DECRYPT;
if (meson_cipher_need_fallback(areq))
return meson_cipher_do_fallback(areq);
- e = get_engine_number(op->mc);
+ e = meson_get_engine_number(op->mc);
engine = op->mc->chanlist[e].engine;
rctx->flow = e;

@@ -300,7 +295,7 @@ int meson_skencrypt(struct skcipher_request *areq)
rctx->op_dir = MESON_ENCRYPT;
if (meson_cipher_need_fallback(areq))
return meson_cipher_do_fallback(areq);
- e = get_engine_number(op->mc);
+ e = meson_get_engine_number(op->mc);
engine = op->mc->chanlist[e].engine;
rctx->flow = e;

diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 372c30f72072..51291fdcf8b6 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -23,6 +23,11 @@

#include "amlogic-gxl.h"

+int meson_get_engine_number(struct meson_dev *mc)
+{
+ return atomic_inc_return(&mc->flow) % mc->flow_cnt;
+}
+
void meson_dma_start(struct meson_dev *mc, int flow)
{
u32 offset = (mc->pdata->descs_reg + flow) << 2;
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index 59fc6a67e0a9..5f5e3115fcdf 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -163,6 +163,8 @@ struct meson_alg_template {
#endif
};

+int meson_get_engine_number(struct meson_dev *mc);
+
void meson_dma_start(struct meson_dev *mc, int flow);

int meson_enqueue(struct crypto_async_request *areq, u32 type);
--
2.34.1


2024-02-05 15:57:46

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 09/20] drivers: crypto: meson: process more than MAXDESCS descriptors

1. The old alhorithm was not designed to process a large
amount of memory, and therefore gave incorrect results.

2. Not all Amlogic SoC's use 3 KEY/IV descriptors.
Add keyiv descriptors count parameter to platform data.

Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 443 ++++++++++++--------
drivers/crypto/amlogic/amlogic-gxl-core.c | 1 +
drivers/crypto/amlogic/amlogic-gxl.h | 2 +
3 files changed, 281 insertions(+), 165 deletions(-)

diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index c662c4b86e97..9c96e7b65e1e 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -17,35 +17,41 @@
#include <crypto/internal/skcipher.h>
#include "amlogic-gxl.h"

-static bool meson_cipher_need_fallback(struct skcipher_request *areq)
+static bool meson_cipher_need_fallback_sg(struct skcipher_request *areq,
+ struct scatterlist *sg)
{
- struct scatterlist *src_sg = areq->src;
- struct scatterlist *dst_sg = areq->dst;
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
+ unsigned int blocksize = crypto_skcipher_blocksize(tfm);
+ unsigned int cryptlen = areq->cryptlen;
+
+ while (cryptlen) {
+ unsigned int len = min(cryptlen, sg->length);
+
+ if (!IS_ALIGNED(sg->offset, sizeof(u32)))
+ return true;
+ if (len % blocksize != 0)
+ return true;
+
+ cryptlen -= len;
+ sg = sg_next(sg);
+ }
+
+ return false;
+}

+static bool meson_cipher_need_fallback(struct skcipher_request *areq)
+{
if (areq->cryptlen == 0)
return true;

- if (sg_nents(src_sg) != sg_nents(dst_sg))
+ if (meson_cipher_need_fallback_sg(areq, areq->src))
return true;

- /* KEY/IV descriptors use 3 desc */
- if (sg_nents(src_sg) > MAXDESC - 3 || sg_nents(dst_sg) > MAXDESC - 3)
- return true;
+ if (areq->dst == areq->src)
+ return false;

- while (src_sg && dst_sg) {
- if ((src_sg->length % 16) != 0)
- return true;
- if ((dst_sg->length % 16) != 0)
- return true;
- if (src_sg->length != dst_sg->length)
- return true;
- if (!IS_ALIGNED(src_sg->offset, sizeof(u32)))
- return true;
- if (!IS_ALIGNED(dst_sg->offset, sizeof(u32)))
- return true;
- src_sg = sg_next(src_sg);
- dst_sg = sg_next(dst_sg);
- }
+ if (meson_cipher_need_fallback_sg(areq, areq->dst))
+ return true;

return false;
}
@@ -76,6 +82,211 @@ static int meson_cipher_do_fallback(struct skcipher_request *areq)
return err;
}

+struct cipher_ctx {
+ struct {
+ dma_addr_t addr;
+ unsigned int len;
+ } keyiv;
+
+ struct skcipher_request *areq;
+ struct scatterlist *src_sg;
+ struct scatterlist *dst_sg;
+
+ unsigned int src_offset;
+ unsigned int dst_offset;
+ unsigned int cryptlen;
+ unsigned int tloffset;
+};
+
+static int meson_map_scatterlist(struct skcipher_request *areq, struct meson_dev *mc)
+{
+ int nr_sgs, nr_sgd;
+
+ if (areq->src == areq->dst) {
+ nr_sgs = dma_map_sg(mc->dev, areq->src, sg_nents(areq->src),
+ DMA_BIDIRECTIONAL);
+ if (!nr_sgs) {
+ dev_err(mc->dev, "Invalid SG count %d\n", nr_sgs);
+ return -EINVAL;
+ }
+ } else {
+ nr_sgs = dma_map_sg(mc->dev, areq->src, sg_nents(areq->src),
+ DMA_TO_DEVICE);
+ if (!nr_sgs) {
+ dev_err(mc->dev, "Invalid SG count %d\n", nr_sgs);
+ return -EINVAL;
+ }
+
+ nr_sgd = dma_map_sg(mc->dev, areq->dst, sg_nents(areq->dst),
+ DMA_FROM_DEVICE);
+ if (!nr_sgd) {
+ dev_err(mc->dev, "Invalid SG count %d\n", nr_sgd);
+ return -EINVAL;
+ }
+ }
+
+ return 0;
+}
+
+static void meson_unmap_scatterlist(struct skcipher_request *areq, struct meson_dev *mc)
+{
+ if (areq->src == areq->dst) {
+ dma_unmap_sg(mc->dev, areq->src, sg_nents(areq->src), DMA_BIDIRECTIONAL);
+ } else {
+ dma_unmap_sg(mc->dev, areq->src, sg_nents(areq->src), DMA_TO_DEVICE);
+ dma_unmap_sg(mc->dev, areq->dst, sg_nents(areq->dst), DMA_FROM_DEVICE);
+ }
+}
+
+static void meson_setup_keyiv_descs(struct cipher_ctx *ctx)
+{
+ struct meson_cipher_req_ctx *rctx = skcipher_request_ctx(ctx->areq);
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(ctx->areq);
+ struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
+ struct meson_alg_template *algt = container_of(alg,
+ struct meson_alg_template, alg.skcipher.base);
+ struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
+ struct meson_dev *mc = op->mc;
+ unsigned int ivsize = crypto_skcipher_ivsize(tfm);
+ unsigned int blockmode = algt->blockmode;
+ int i;
+
+ if (ctx->tloffset)
+ return;
+
+ if (blockmode == DESC_OPMODE_CBC) {
+ memcpy(op->key + AES_MAX_KEY_SIZE, ctx->areq->iv, ivsize);
+ ctx->keyiv.len = AES_MAX_KEY_SIZE + ivsize;
+ dma_sync_single_for_device(mc->dev, ctx->keyiv.addr,
+ ctx->keyiv.len, DMA_TO_DEVICE);
+ }
+
+ for (i = 0; i < mc->pdata->setup_desc_cnt; i++) {
+ struct meson_desc *desc =
+ &mc->chanlist[rctx->flow].tl[ctx->tloffset];
+ int offset = i * 16;
+
+ desc->t_src = cpu_to_le32(ctx->keyiv.addr + offset);
+ desc->t_dst = cpu_to_le32(offset);
+ desc->t_status = cpu_to_le32(DESC_OWN | DESC_MODE_KEY | ctx->keyiv.len);
+
+ ctx->tloffset++;
+ }
+}
+
+static bool meson_setup_data_descs(struct cipher_ctx *ctx)
+{
+ struct meson_cipher_req_ctx *rctx = skcipher_request_ctx(ctx->areq);
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(ctx->areq);
+ struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
+ struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
+ struct meson_alg_template *algt = container_of(alg,
+ struct meson_alg_template,
+ alg.skcipher.base);
+ struct meson_dev *mc = op->mc;
+ struct meson_desc *desc = &mc->chanlist[rctx->flow].tl[ctx->tloffset];
+ unsigned int blocksize = crypto_skcipher_blocksize(tfm);
+ unsigned int blockmode = algt->blockmode;
+ unsigned int maxlen = rounddown(DESC_MAXLEN, blocksize);
+ unsigned int todo;
+ u32 v;
+
+ ctx->tloffset++;
+
+ todo = min(ctx->cryptlen, maxlen);
+ todo = min(todo, ctx->cryptlen);
+ todo = min(todo, sg_dma_len(ctx->src_sg) - ctx->src_offset);
+ todo = min(todo, sg_dma_len(ctx->dst_sg) - ctx->dst_offset);
+
+ desc->t_src = cpu_to_le32(sg_dma_address(ctx->src_sg) + ctx->src_offset);
+ desc->t_dst = cpu_to_le32(sg_dma_address(ctx->dst_sg) + ctx->dst_offset);
+
+ ctx->cryptlen -= todo;
+ ctx->src_offset += todo;
+ ctx->dst_offset += todo;
+
+ v = DESC_OWN | blockmode | op->keymode | todo;
+ if (rctx->op_dir == MESON_ENCRYPT)
+ v |= DESC_ENCRYPTION;
+
+ if (!ctx->cryptlen || ctx->tloffset == MAXDESC)
+ v |= DESC_LAST;
+
+ desc->t_status = cpu_to_le32(v);
+
+ return v & DESC_LAST;
+}
+
+static int meson_kick_hardware(struct cipher_ctx *ctx)
+{
+ struct meson_cipher_req_ctx *rctx = skcipher_request_ctx(ctx->areq);
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(ctx->areq);
+ struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
+ struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
+ struct meson_alg_template *algt = container_of(alg,
+ struct meson_alg_template,
+ alg.skcipher.base);
+ struct meson_dev *mc = op->mc;
+ unsigned int ivsize = crypto_skcipher_ivsize(tfm);
+ unsigned int blockmode = algt->blockmode;
+ enum dma_data_direction new_iv_dir;
+ dma_addr_t new_iv_phys;
+ void *new_iv;
+ int err;
+
+ if (blockmode == DESC_OPMODE_CBC) {
+ struct scatterlist *sg;
+ unsigned int offset;
+
+ if (rctx->op_dir == MESON_ENCRYPT) {
+ sg = ctx->dst_sg;
+ offset = ctx->dst_offset;
+ new_iv_dir = DMA_FROM_DEVICE;
+ } else {
+ sg = ctx->src_sg;
+ offset = ctx->src_offset;
+ new_iv_dir = DMA_TO_DEVICE;
+ }
+
+ if (ctx->areq->src == ctx->areq->dst)
+ new_iv_dir = DMA_BIDIRECTIONAL;
+
+ offset -= ivsize;
+ new_iv = sg_virt(sg) + offset;
+ new_iv_phys = sg_dma_address(sg) + offset;
+ }
+
+ if (blockmode == DESC_OPMODE_CBC &&
+ rctx->op_dir == MESON_DECRYPT) {
+ dma_sync_single_for_cpu(mc->dev, new_iv_phys,
+ ivsize, new_iv_dir);
+ memcpy(ctx->areq->iv, new_iv, ivsize);
+ }
+
+ reinit_completion(&mc->chanlist[rctx->flow].complete);
+ meson_dma_start(mc, rctx->flow);
+ err = wait_for_completion_interruptible_timeout(
+ &mc->chanlist[rctx->flow].complete, msecs_to_jiffies(500));
+ if (err == 0) {
+ dev_err(mc->dev, "DMA timeout for flow %d\n", rctx->flow);
+ return -EINVAL;
+ } else if (err < 0) {
+ dev_err(mc->dev, "Waiting for DMA completion is failed (%d)\n", err);
+ return err;
+ }
+
+ if (blockmode == DESC_OPMODE_CBC &&
+ rctx->op_dir == MESON_ENCRYPT) {
+ dma_sync_single_for_cpu(mc->dev, new_iv_phys,
+ ivsize, new_iv_dir);
+ memcpy(ctx->areq->iv, new_iv, ivsize);
+ }
+
+ ctx->tloffset = 0;
+
+ return 0;
+}
+
static int meson_cipher(struct skcipher_request *areq)
{
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
@@ -84,176 +295,78 @@ static int meson_cipher(struct skcipher_request *areq)
struct meson_dev *mc = op->mc;
struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
struct meson_alg_template *algt;
- int flow = rctx->flow;
- unsigned int todo, eat, len;
- struct scatterlist *src_sg = areq->src;
- struct scatterlist *dst_sg = areq->dst;
- struct meson_desc *desc;
- int nr_sgs, nr_sgd;
- int i, err = 0;
- unsigned int keyivlen, ivsize, offset, tloffset;
- dma_addr_t phykeyiv;
- void *backup_iv = NULL, *bkeyiv;
- u32 v;
-
- algt = container_of(alg, struct meson_alg_template, alg.skcipher.base);
+ struct cipher_ctx ctx = {
+ .areq = areq,
+ .src_offset = 0,
+ .dst_offset = 0,
+ .src_sg = areq->src,
+ .dst_sg = areq->dst,
+ .cryptlen = areq->cryptlen,
+ };
+ unsigned int ivsize = crypto_skcipher_ivsize(tfm);
+ int err;

- dev_dbg(mc->dev, "%s %s %u %x IV(%u) key=%u flow=%d\n", __func__,
+ dev_dbg(mc->dev, "%s %s %u %x IV(%u) key=%u ctx.flow=%d\n", __func__,
crypto_tfm_alg_name(areq->base.tfm),
areq->cryptlen,
rctx->op_dir, crypto_skcipher_ivsize(tfm),
- op->keylen, flow);
+ op->keylen, rctx->flow);
+
+ algt = container_of(alg, struct meson_alg_template, alg.skcipher.base);

#ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
algt->stat_req++;
- mc->chanlist[flow].stat_req++;
+ mc->chanlist[rctx->flow].stat_req++;
#endif

- /*
- * The hardware expect a list of meson_desc structures.
- * The 2 first structures store key
- * The third stores IV
- */
- bkeyiv = kzalloc(48, GFP_KERNEL | GFP_DMA);
- if (!bkeyiv)
+ op->key = kzalloc(48, GFP_KERNEL | GFP_DMA);
+ if (!op.key)
return -ENOMEM;

- memcpy(bkeyiv, op->key, op->keylen);
- keyivlen = op->keylen;
+ memcpy(op->key, op->key, op->keylen);
+ ctx.keyiv.len = op->keylen;
+ if (ctx.keyiv.len == AES_KEYSIZE_192)
+ ctx.keyiv.len = AES_MAX_KEY_SIZE;

- ivsize = crypto_skcipher_ivsize(tfm);
- if (areq->iv && ivsize > 0) {
- if (ivsize > areq->cryptlen) {
- dev_err(mc->dev, "invalid ivsize=%d vs len=%d\n", ivsize, areq->cryptlen);
- err = -EINVAL;
- goto theend;
- }
- memcpy(bkeyiv + 32, areq->iv, ivsize);
- keyivlen = 48;
- if (rctx->op_dir == MESON_DECRYPT) {
- backup_iv = kzalloc(ivsize, GFP_KERNEL);
- if (!backup_iv) {
- err = -ENOMEM;
- goto theend;
- }
- offset = areq->cryptlen - ivsize;
- scatterwalk_map_and_copy(backup_iv, areq->src, offset,
- ivsize, 0);
- }
- }
- if (keyivlen == AES_KEYSIZE_192)
- keyivlen = AES_MAX_KEY_SIZE;
-
- phykeyiv = dma_map_single(mc->dev, bkeyiv, keyivlen,
+ ctx.keyiv.addr = dma_map_single(mc->dev, op->key, ctx.keyiv.len,
DMA_TO_DEVICE);
- err = dma_mapping_error(mc->dev, phykeyiv);
+ err = dma_mapping_error(mc->dev, ctx.keyiv.addr);
if (err) {
dev_err(mc->dev, "Cannot DMA MAP KEY IV\n");
goto theend;
}

- tloffset = 0;
- eat = 0;
- i = 0;
- while (keyivlen > eat) {
- desc = &mc->chanlist[flow].tl[tloffset];
- memset(desc, 0, sizeof(struct meson_desc));
- todo = min(keyivlen - eat, 16u);
- desc->t_src = cpu_to_le32(phykeyiv + i * 16);
- desc->t_dst = cpu_to_le32(i * 16);
- v = DESC_MODE_KEY | DESC_OWN | 16;
- desc->t_status = cpu_to_le32(v);
-
- eat += todo;
- i++;
- tloffset++;
- }
-
- if (areq->src == areq->dst) {
- nr_sgs = dma_map_sg(mc->dev, areq->src, sg_nents(areq->src),
- DMA_BIDIRECTIONAL);
- if (!nr_sgs) {
- dev_err(mc->dev, "Invalid SG count %d\n", nr_sgs);
- err = -EINVAL;
- goto theend;
- }
- nr_sgd = nr_sgs;
- } else {
- nr_sgs = dma_map_sg(mc->dev, areq->src, sg_nents(areq->src),
- DMA_TO_DEVICE);
- if (!nr_sgs || nr_sgs > MAXDESC - 3) {
- dev_err(mc->dev, "Invalid SG count %d\n", nr_sgs);
- err = -EINVAL;
- goto theend;
- }
- nr_sgd = dma_map_sg(mc->dev, areq->dst, sg_nents(areq->dst),
- DMA_FROM_DEVICE);
- if (!nr_sgd || nr_sgd > MAXDESC - 3) {
- dev_err(mc->dev, "Invalid SG count %d\n", nr_sgd);
- err = -EINVAL;
- goto theend;
- }
- }
-
- src_sg = areq->src;
- dst_sg = areq->dst;
- len = areq->cryptlen;
- while (src_sg) {
- desc = &mc->chanlist[flow].tl[tloffset];
- memset(desc, 0, sizeof(struct meson_desc));
-
- desc->t_src = cpu_to_le32(sg_dma_address(src_sg));
- desc->t_dst = cpu_to_le32(sg_dma_address(dst_sg));
- todo = min(len, sg_dma_len(src_sg));
- v = op->keymode | DESC_OWN | todo | algt->blockmode;
- if (rctx->op_dir)
- v |= DESC_ENCRYPTION;
- len -= todo;
-
- if (!sg_next(src_sg))
- v |= DESC_LAST;
- desc->t_status = cpu_to_le32(v);
- tloffset++;
- src_sg = sg_next(src_sg);
- dst_sg = sg_next(dst_sg);
- }
+ err = meson_map_scatterlist(areq, mc);
+ if (err)
+ goto theend;

- reinit_completion(&mc->chanlist[flow].complete);
- meson_dma_start(mc, flow);
+ ctx.tloffset = 0;

- err = wait_for_completion_interruptible_timeout(&mc->chanlist[flow].complete,
- msecs_to_jiffies(500));
- if (err == 0) {
- dev_err(mc->dev, "DMA timeout for flow %d\n", flow);
- err = -EINVAL;
- } else if (err < 0) {
- dev_err(mc->dev, "Waiting for DMA completion is failed (%d)\n", err);
- } else {
- /* No error */
- err = 0;
- }
+ while (ctx.cryptlen) {
+ meson_setup_keyiv_descs(&ctx);

- dma_unmap_single(mc->dev, phykeyiv, keyivlen, DMA_TO_DEVICE);
+ if (meson_setup_data_descs(&ctx)) {
+ err = meson_kick_hardware(&ctx);
+ if (err)
+ break;
+ }

- if (areq->src == areq->dst) {
- dma_unmap_sg(mc->dev, areq->src, sg_nents(areq->src), DMA_BIDIRECTIONAL);
- } else {
- dma_unmap_sg(mc->dev, areq->src, sg_nents(areq->src), DMA_TO_DEVICE);
- dma_unmap_sg(mc->dev, areq->dst, sg_nents(areq->dst), DMA_FROM_DEVICE);
- }
+ if (ctx.src_offset == sg_dma_len(ctx.src_sg)) {
+ ctx.src_offset = 0;
+ ctx.src_sg = sg_next(ctx.src_sg);
+ }

- if (areq->iv && ivsize > 0) {
- if (rctx->op_dir == MESON_DECRYPT) {
- memcpy(areq->iv, backup_iv, ivsize);
- } else {
- scatterwalk_map_and_copy(areq->iv, areq->dst,
- areq->cryptlen - ivsize,
- ivsize, 0);
+ if (ctx.dst_offset == sg_dma_len(ctx.dst_sg)) {
+ ctx.dst_offset = 0;
+ ctx.dst_sg = sg_next(ctx.dst_sg);
}
}
+
+ dma_unmap_single(mc->dev, ctx.keyiv.addr, ctx.keyiv.len, DMA_TO_DEVICE);
+ meson_unmap_scatterlist(areq, mc);
+
theend:
- kfree_sensitive(bkeyiv);
- kfree_sensitive(backup_iv);
+ kfree_sensitive(op->key);

return err;
}
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 22ff2768b5e5..f93e14f5717d 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -199,6 +199,7 @@ static const struct meson_pdata meson_gxl_pdata = {
.descs_reg = 0x0,
.status_reg = 0x4,
.need_clk = true,
+ .setup_desc_cnt = 3,
};

static const struct of_device_id meson_crypto_of_match_table[] = {
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index a0d83c82906d..eb2f8cd72b65 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -83,11 +83,13 @@ struct meson_flow {
* @reg_descs: offset to descriptors register
* @reg_status: offset to status register
* @need_clk: clock input is needed
+ * @setup_desc_cnt: number of setup descriptor to configure.
*/
struct meson_pdata {
u32 descs_reg;
u32 status_reg;
bool need_clk;
+ u32 setup_desc_cnt;
};

/*
--
2.34.1


2024-02-05 15:57:48

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 06/20] drivers: crypto: meson: drop status field from meson_flow

This field is used only to check for timeout. But there is more
convenient way to achive the same goal.

Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 13 +++++++++----
drivers/crypto/amlogic/amlogic-gxl-core.c | 1 -
drivers/crypto/amlogic/amlogic-gxl.h | 2 --
3 files changed, 9 insertions(+), 7 deletions(-)

diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index 18e9e2d39b1f..dc0b100c5de2 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -219,13 +219,18 @@ static int meson_cipher(struct skcipher_request *areq)
}

reinit_completion(&mc->chanlist[flow].complete);
- mc->chanlist[flow].status = 0;
meson_dma_start(mc, flow);
- wait_for_completion_interruptible_timeout(&mc->chanlist[flow].complete,
- msecs_to_jiffies(500));
- if (mc->chanlist[flow].status == 0) {
+
+ err = wait_for_completion_interruptible_timeout(&mc->chanlist[flow].complete,
+ msecs_to_jiffies(500));
+ if (err == 0) {
dev_err(mc->dev, "DMA timeout for flow %d\n", flow);
err = -EINVAL;
+ } else if (err < 0) {
+ dev_err(mc->dev, "Waiting for DMA completion is failed (%d)\n", err);
+ } else {
+ /* No error */
+ err = 0;
}

dma_unmap_single(mc->dev, phykeyiv, keyivlen, DMA_TO_DEVICE);
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 51291fdcf8b6..f7c60ebffbc3 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -54,7 +54,6 @@ static irqreturn_t meson_irq_handler(int irq, void *data)
for (flow = 0; flow < mc->flow_cnt; flow++) {
if (mc->chanlist[flow].irq == irq) {
if (meson_dma_ready(mc, flow)) {
- mc->chanlist[flow].status = 1;
complete(&mc->chanlist[flow].complete);
return IRQ_HANDLED;
}
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index 5f5e3115fcdf..e27908992ae3 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -59,7 +59,6 @@ struct meson_desc {
* @engine: ptr to the crypto_engine for this flow
* @keylen: keylen for this flow operation
* @complete: completion for the current task on this flow
- * @status: set to 1 by interrupt if task is done
* @irq: IRQ number for amlogic-crypto
* @t_phy: Physical address of task
* @tl: pointer to the current ce_task for this flow
@@ -68,7 +67,6 @@ struct meson_desc {
struct meson_flow {
struct crypto_engine *engine;
struct completion complete;
- int status;
int irq;
unsigned int keylen;
dma_addr_t t_phy;
--
2.34.1


2024-02-05 15:58:00

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 10/20] drivers: crypto: meson: avoid kzalloc in engine thread

It makes no sense to allocate memory via kzalloc, we
can use static buffer, speedup data processing and
don't think about kfree() calls.

Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 24 ++++++++-------------
drivers/crypto/amlogic/amlogic-gxl.h | 6 +++---
2 files changed, 12 insertions(+), 18 deletions(-)

diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index 9c96e7b65e1e..3f42b2cc568d 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -155,8 +155,7 @@ static void meson_setup_keyiv_descs(struct cipher_ctx *ctx)
return;

if (blockmode == DESC_OPMODE_CBC) {
- memcpy(op->key + AES_MAX_KEY_SIZE, ctx->areq->iv, ivsize);
- ctx->keyiv.len = AES_MAX_KEY_SIZE + ivsize;
+ memcpy(op->keyiv + AES_MAX_KEY_SIZE, ctx->areq->iv, ivsize);
dma_sync_single_for_device(mc->dev, ctx->keyiv.addr,
ctx->keyiv.len, DMA_TO_DEVICE);
}
@@ -319,16 +318,16 @@ static int meson_cipher(struct skcipher_request *areq)
mc->chanlist[rctx->flow].stat_req++;
#endif

- op->key = kzalloc(48, GFP_KERNEL | GFP_DMA);
- if (!op.key)
- return -ENOMEM;
-
- memcpy(op->key, op->key, op->keylen);
ctx.keyiv.len = op->keylen;
if (ctx.keyiv.len == AES_KEYSIZE_192)
ctx.keyiv.len = AES_MAX_KEY_SIZE;

- ctx.keyiv.addr = dma_map_single(mc->dev, op->key, ctx.keyiv.len,
+ if (algt->blockmode == DESC_OPMODE_CBC) {
+ memcpy(op->keyiv + AES_MAX_KEY_SIZE, areq->iv, ivsize);
+ ctx.keyiv.len = AES_MAX_KEY_SIZE + ivsize;
+ }
+
+ ctx.keyiv.addr = dma_map_single(mc->dev, op->keyiv, ctx.keyiv.len,
DMA_TO_DEVICE);
err = dma_mapping_error(mc->dev, ctx.keyiv.addr);
if (err) {
@@ -366,8 +365,6 @@ static int meson_cipher(struct skcipher_request *areq)
meson_unmap_scatterlist(areq, mc);

theend:
- kfree_sensitive(op->key);
-
return err;
}

@@ -450,7 +447,6 @@ static void meson_cipher_exit(struct crypto_tfm *tfm)
{
struct meson_cipher_tfm_ctx *op = crypto_tfm_ctx(tfm);

- kfree_sensitive(op->key);
crypto_free_skcipher(op->fallback_tfm);
}

@@ -474,11 +470,9 @@ static int meson_aes_setkey(struct crypto_skcipher *tfm, const u8 *key,
dev_dbg(mc->dev, "ERROR: Invalid keylen %u\n", keylen);
return -EINVAL;
}
- kfree_sensitive(op->key);
+
+ memcpy(op->keyiv, key, keylen);
op->keylen = keylen;
- op->key = kmemdup(key, keylen, GFP_KERNEL | GFP_DMA);
- if (!op->key)
- return -ENOMEM;

return crypto_skcipher_setkey(op->fallback_tfm, key, keylen);
}
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index eb2f8cd72b65..e1453dd2e9f4 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -129,15 +129,15 @@ struct meson_cipher_req_ctx {

/*
* struct meson_cipher_tfm_ctx - context for a skcipher TFM
- * @key: pointer to key data
+ * @keyiv: key data
* @keylen: len of the key
* @keymode: The keymode(type and size of key) associated with this TFM
* @mc: pointer to the private data of driver handling this TFM
* @fallback_tfm: pointer to the fallback TFM
*/
struct meson_cipher_tfm_ctx {
- u32 *key;
- u32 keylen;
+ u8 keyiv[AES_MAX_KEY_SIZE + AES_BLOCK_SIZE] ____cacheline_aligned;
+ u32 keylen ____cacheline_aligned;
u32 keymode;
struct meson_dev *mc;
struct crypto_skcipher *fallback_tfm;
--
2.34.1


2024-02-05 15:58:24

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 11/20] drivers: crypto: meson: introduce hasher

Introduce support for SHA1/SHA224/SHA256 hash algos.
Tested via tcrypt and custom tests.

Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/Makefile | 2 +-
drivers/crypto/amlogic/amlogic-gxl-core.c | 24 +-
drivers/crypto/amlogic/amlogic-gxl-hasher.c | 452 ++++++++++++++++++++
drivers/crypto/amlogic/amlogic-gxl.h | 49 +++
4 files changed, 525 insertions(+), 2 deletions(-)
create mode 100644 drivers/crypto/amlogic/amlogic-gxl-hasher.c

diff --git a/drivers/crypto/amlogic/Makefile b/drivers/crypto/amlogic/Makefile
index 39057e62c13e..4b6b388b7880 100644
--- a/drivers/crypto/amlogic/Makefile
+++ b/drivers/crypto/amlogic/Makefile
@@ -1,2 +1,2 @@
obj-$(CONFIG_CRYPTO_DEV_AMLOGIC_GXL) += amlogic-gxl-crypto.o
-amlogic-gxl-crypto-y := amlogic-gxl-core.o amlogic-gxl-cipher.o
+amlogic-gxl-crypto-y := amlogic-gxl-core.o amlogic-gxl-cipher.o amlogic-gxl-hasher.o
diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index f93e14f5717d..f3b5e004b3a5 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -20,6 +20,9 @@
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/platform_device.h>
+#include <crypto/internal/skcipher.h>
+#include <crypto/internal/hash.h>
+#include <linux/dma-mapping.h>

#include "amlogic-gxl.h"

@@ -172,6 +175,15 @@ int meson_register_algs(struct meson_dev *mc, struct meson_alg_template *algs,
return err;
}
break;
+ case CRYPTO_ALG_TYPE_AHASH:
+ err = crypto_engine_register_ahash(&algs[i].alg.ahash);
+ if (err) {
+ dev_err(mc->dev, "Fail to register %s\n",
+ algs[i].alg.ahash.base.halg.base.cra_name);
+ meson_unregister_algs(mc, algs, count);
+ return err;
+ }
+ break;
}
algs[i].mc = mc;
}
@@ -191,6 +203,9 @@ void meson_unregister_algs(struct meson_dev *mc, struct meson_alg_template *algs
case CRYPTO_ALG_TYPE_SKCIPHER:
crypto_engine_unregister_skcipher(&algs[i].alg.skcipher);
break;
+ case CRYPTO_ALG_TYPE_AHASH:
+ crypto_engine_unregister_ahash(&algs[i].alg.ahash);
+ break;
}
}
}
@@ -258,13 +273,20 @@ static int meson_crypto_probe(struct platform_device *pdev)

dbgfs_dir = debugfs_create_dir("gxl-crypto", NULL);
debugfs_create_file("stats", 0444, dbgfs_dir, mc, &meson_debugfs_fops);
-
#ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
mc->dbgfs_dir = dbgfs_dir;
#endif
}

+ err = meson_hasher_register(mc);
+ if (err)
+ goto error_hasher;
+
return 0;
+
+error_hasher:
+ meson_cipher_unregister(mc);
+
error_flow:
meson_free_chanlist(mc, mc->flow_cnt - 1);
return err;
diff --git a/drivers/crypto/amlogic/amlogic-gxl-hasher.c b/drivers/crypto/amlogic/amlogic-gxl-hasher.c
new file mode 100644
index 000000000000..04f85cd4d97f
--- /dev/null
+++ b/drivers/crypto/amlogic/amlogic-gxl-hasher.c
@@ -0,0 +1,452 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Hardware asynchronous hasher for Amlogic SoC's.
+ *
+ * Copyright (c) 2023, SaluteDevices. All Rights Reserved.
+ *
+ * Author: Alexey Romanov <[email protected]>
+ */
+
+#include <linux/crypto.h>
+#include <linux/device.h>
+#include <linux/dma-mapping.h>
+#include <linux/err.h>
+#include <crypto/internal/hash.h>
+#include <crypto/sha1.h>
+#include <crypto/sha2.h>
+
+#include "amlogic-gxl.h"
+
+static int meson_sha_init(struct ahash_request *req)
+{
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+
+ memset(rctx, 0, sizeof(struct meson_hasher_req_ctx));
+
+ rctx->flow = meson_get_engine_number(tctx->mc);
+ rctx->begin_req = true;
+
+ return 0;
+}
+
+static int meson_sha_update(struct ahash_request *req)
+{
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+ struct crypto_engine *engine = tctx->mc->chanlist[rctx->flow].engine;
+
+ return crypto_transfer_hash_request_to_engine(engine, req);
+}
+
+static int meson_sha_final(struct ahash_request *req)
+{
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+ struct crypto_engine *engine = tctx->mc->chanlist[rctx->flow].engine;
+
+ rctx->final_req = true;
+
+ return crypto_transfer_hash_request_to_engine(engine, req);
+}
+
+static int meson_hasher_req_map(struct ahash_request *req)
+{
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_dev *mc = tctx->mc;
+ int ret;
+
+ if (!req->nbytes)
+ return 0;
+
+ ret = dma_map_sg(mc->dev, req->src, sg_nents(req->src), DMA_TO_DEVICE);
+ if (!ret) {
+ dev_err(mc->dev, "Cannot DMA MAP request data\n");
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+static void meson_hasher_req_unmap(struct ahash_request *req)
+{
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_dev *mc = tctx->mc;
+
+ if (!req->nbytes)
+ return;
+
+ dma_unmap_sg(mc->dev, req->src, sg_nents(req->src), DMA_TO_DEVICE);
+}
+
+struct hasher_ctx {
+ struct crypto_async_request *areq;
+
+ unsigned int tloffset;
+ unsigned int nbytes;
+ unsigned int todo;
+
+ dma_addr_t state_addr;
+ dma_addr_t src_addr;
+ unsigned int src_offset;
+ struct scatterlist *src_sg;
+};
+
+static bool meson_final(struct hasher_ctx *ctx)
+{
+ struct ahash_request *req = ahash_request_cast(ctx->areq);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+
+ return !ctx->nbytes && rctx->final_req;
+}
+
+static int meson_fill_partial_buffer(struct hasher_ctx *ctx, unsigned int len)
+{
+ struct ahash_request *req = ahash_request_cast(ctx->areq);
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+ struct meson_dev *mc = tctx->mc;
+ unsigned int blocksize = crypto_ahash_blocksize(tfm);
+ unsigned int copy;
+
+ if (len) {
+ copy = min(blocksize - rctx->partial_size, len);
+ memcpy(rctx->partial + rctx->partial_size,
+ sg_virt(ctx->src_sg) + ctx->src_offset, copy);
+
+ rctx->partial_size += copy;
+ ctx->nbytes -= copy;
+ ctx->src_offset += copy;
+ }
+
+ if (rctx->partial_size == blocksize || meson_final(ctx)) {
+ rctx->partial_addr = dma_map_single(mc->dev,
+ rctx->partial,
+ rctx->partial_size,
+ DMA_TO_DEVICE);
+ if (dma_mapping_error(mc->dev, rctx->partial_addr)) {
+ dev_err(mc->dev, "Cannot DMA MAP SHA partial buffer\n");
+ return -ENOMEM;
+ }
+
+ rctx->partial_mapped = true;
+ ctx->todo = rctx->partial_size;
+ ctx->src_addr = rctx->partial_addr;
+ }
+
+ return 0;
+}
+
+static unsigned int meson_setup_data_descs(struct hasher_ctx *ctx)
+{
+ struct ahash_request *req = ahash_request_cast(ctx->areq);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_dev *mc = tctx->mc;
+ struct meson_flow *flow = &mc->chanlist[rctx->flow];
+ struct hash_alg_common *alg = crypto_hash_alg_common(tfm);
+ struct meson_alg_template *algt = container_of(alg,
+ struct meson_alg_template, alg.ahash.base.halg);
+ struct meson_desc *desc = &flow->tl[ctx->tloffset];
+ u32 v;
+
+ ctx->tloffset++;
+
+ v = DESC_OWN | DESC_ENCRYPTION | DESC_OPMODE_SHA |
+ ctx->todo | algt->blockmode;
+ if (rctx->begin_req) {
+ rctx->begin_req = false;
+ v |= DESC_BEGIN;
+ }
+
+ if (!ctx->nbytes && rctx->final_req) {
+ rctx->final_req = false;
+ v |= DESC_END;
+ }
+
+ if (!ctx->nbytes || ctx->tloffset == MAXDESC || rctx->partial_mapped)
+ v |= DESC_LAST;
+
+ desc->t_src = cpu_to_le32(ctx->src_addr);
+ desc->t_dst = cpu_to_le32(ctx->state_addr);
+ desc->t_status = cpu_to_le32(v);
+
+ return v & DESC_LAST;
+}
+
+static int meson_kick_hardware(struct hasher_ctx *ctx)
+{
+ struct ahash_request *req = ahash_request_cast(ctx->areq);
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_dev *mc = tctx->mc;
+ struct meson_flow *flow = &mc->chanlist[rctx->flow];
+ int ret;
+
+ reinit_completion(&flow->complete);
+ meson_dma_start(mc, rctx->flow);
+
+ ret = wait_for_completion_timeout(&flow->complete,
+ msecs_to_jiffies(500));
+ if (ret == 0) {
+ dev_err(mc->dev, "DMA timeout for flow %d\n", rctx->flow);
+ return -EINVAL;
+ } else if (ret < 0) {
+ dev_err(mc->dev, "Waiting for DMA completion is failed (%d)\n", ret);
+ return ret;
+ }
+
+ if (rctx->partial_mapped) {
+ dma_unmap_single(mc->dev, rctx->partial_addr,
+ rctx->partial_size,
+ DMA_TO_DEVICE);
+ rctx->partial_size = 0;
+ rctx->partial_mapped = false;
+ }
+
+ ctx->tloffset = 0;
+
+ return 0;
+}
+
+static void meson_setup_state_descs(struct hasher_ctx *ctx)
+{
+ struct ahash_request *req = ahash_request_cast(ctx->areq);
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_dev *mc = tctx->mc;
+ struct meson_desc *desc;
+ int i;
+
+ if (ctx->tloffset || rctx->begin_req)
+ return;
+
+ for (i = 0; i < mc->pdata->setup_desc_cnt; i++) {
+ int offset = i * 16;
+
+ desc = &mc->chanlist[rctx->flow].tl[ctx->tloffset];
+ desc->t_src = cpu_to_le32(ctx->state_addr + offset);
+ desc->t_dst = cpu_to_le32(offset);
+ desc->t_status = cpu_to_le32(MESON_SHA_BUFFER_SIZE |
+ DESC_MODE_KEY | DESC_OWN);
+
+ ctx->tloffset++;
+ }
+}
+
+static int meson_hasher_do_one_request(struct crypto_engine *engine, void *areq)
+{
+ struct ahash_request *req = ahash_request_cast(areq);
+ struct crypto_ahash *tfm = crypto_ahash_reqtfm(req);
+ struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm);
+ struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req);
+ struct meson_dev *mc = tctx->mc;
+ struct hasher_ctx ctx = {
+ .tloffset = 0,
+ .src_offset = 0,
+ .nbytes = req->nbytes,
+ .src_sg = req->src,
+ .areq = areq,
+ };
+ unsigned int blocksize = crypto_ahash_blocksize(tfm);
+ unsigned int digest_size = crypto_ahash_digestsize(tfm);
+ bool final_req = rctx->final_req;
+ int ret;
+
+ ctx.state_addr = dma_map_single(mc->dev, rctx->state,
+ sizeof(rctx->state), DMA_BIDIRECTIONAL);
+ ret = dma_mapping_error(mc->dev, ctx.state_addr);
+ if (ret) {
+ dev_err(mc->dev, "Cannot DMA MAP SHA state buffer");
+ goto fail_map_single;
+ }
+
+ ret = meson_hasher_req_map(req);
+ if (ret)
+ goto fail_map_req;
+
+ for (;;) {
+ unsigned int len = ctx.src_sg ?
+ min(sg_dma_len(ctx.src_sg) - ctx.src_offset, ctx.nbytes) : 0;
+
+ ctx.src_addr = 0;
+ ctx.todo = 0;
+
+ if (!rctx->final_req && !ctx.nbytes)
+ break;
+
+ meson_setup_state_descs(&ctx);
+
+ if (rctx->partial_size && rctx->partial_size < blocksize) {
+ ret = meson_fill_partial_buffer(&ctx, len);
+ if (ret)
+ goto fail;
+ } else if (len && len < blocksize) {
+ memcpy(rctx->partial, sg_virt(ctx.src_sg) + ctx.src_offset, len);
+
+ rctx->partial_size = len;
+ ctx.nbytes -= len;
+ ctx.src_offset += len;
+ } else if (len) {
+ ctx.src_addr = sg_dma_address(ctx.src_sg) + ctx.src_offset;
+ ctx.todo = min(rounddown(DESC_MAXLEN, blocksize),
+ rounddown(len, blocksize));
+ ctx.nbytes -= ctx.todo;
+ ctx.src_offset += ctx.todo;
+ }
+
+ if (ctx.src_sg && ctx.src_offset == sg_dma_len(ctx.src_sg)) {
+ ctx.src_offset = 0;
+ ctx.src_sg = sg_next(ctx.src_sg);
+ }
+
+ if (!ctx.todo && ctx.nbytes)
+ continue;
+
+ if (!ctx.todo && !rctx->final_req && !ctx.tloffset)
+ continue;
+
+ if (meson_setup_data_descs(&ctx)) {
+ ret = meson_kick_hardware(&ctx);
+ if (ret)
+ goto fail;
+ }
+ }
+
+fail:
+ meson_hasher_req_unmap(req);
+
+fail_map_req:
+ dma_unmap_single(mc->dev, ctx.state_addr, sizeof(rctx->state),
+ DMA_BIDIRECTIONAL);
+
+fail_map_single:
+ if (final_req && ret == 0)
+ memcpy(req->result, rctx->state, digest_size);
+
+ local_bh_disable();
+ crypto_finalize_hash_request(engine, req, ret);
+ local_bh_enable();
+
+ return ret;
+}
+
+static int meson_hasher_init(struct crypto_tfm *tfm)
+{
+ struct meson_hasher_tfm_ctx *tctx = crypto_tfm_ctx(tfm);
+ struct crypto_ahash *atfm = __crypto_ahash_cast(tfm);
+ struct hash_alg_common *alg = crypto_hash_alg_common(atfm);
+ struct meson_alg_template *algt = container_of(alg,
+ struct meson_alg_template, alg.ahash.base.halg);
+
+ crypto_ahash_set_reqsize(atfm, crypto_ahash_statesize(atfm));
+
+ memset(tctx, 0, sizeof(struct meson_hasher_tfm_ctx));
+
+ tctx->mc = algt->mc;
+
+ return 0;
+}
+
+static struct meson_alg_template mc_algs[] = {
+{
+ .type = CRYPTO_ALG_TYPE_AHASH,
+ .blockmode = DESC_MODE_SHA1,
+ .alg.ahash.base = {
+ .halg = {
+ .base = {
+ .cra_name = "sha1",
+ .cra_driver_name = "sha1-gxl",
+ .cra_priority = 400,
+ .cra_blocksize = SHA1_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ .cra_ctxsize = sizeof(struct meson_hasher_tfm_ctx),
+ .cra_module = THIS_MODULE,
+ .cra_alignmask = 0,
+ .cra_init = meson_hasher_init,
+ },
+ .digestsize = SHA1_DIGEST_SIZE,
+ .statesize = sizeof(struct meson_hasher_req_ctx),
+ },
+ .init = meson_sha_init,
+ .update = meson_sha_update,
+ .final = meson_sha_final,
+ },
+ .alg.ahash.op = {
+ .do_one_request = meson_hasher_do_one_request,
+ },
+},
+{
+ .type = CRYPTO_ALG_TYPE_AHASH,
+ .blockmode = DESC_MODE_SHA224,
+ .alg.ahash.base = {
+ .halg = {
+ .base = {
+ .cra_name = "sha224",
+ .cra_driver_name = "sha224-gxl",
+ .cra_priority = 400,
+ .cra_blocksize = SHA224_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ .cra_ctxsize = sizeof(struct meson_hasher_tfm_ctx),
+ .cra_module = THIS_MODULE,
+ .cra_alignmask = 0,
+ .cra_init = meson_hasher_init,
+ },
+ .digestsize = SHA224_DIGEST_SIZE,
+ .statesize = sizeof(struct meson_hasher_req_ctx),
+ },
+ .init = meson_sha_init,
+ .update = meson_sha_update,
+ .final = meson_sha_final,
+ },
+ .alg.ahash.op = {
+ .do_one_request = meson_hasher_do_one_request,
+ },
+},
+{
+ .type = CRYPTO_ALG_TYPE_AHASH,
+ .blockmode = DESC_MODE_SHA256,
+ .alg.ahash.base = {
+ .halg = {
+ .base = {
+ .cra_name = "sha256",
+ .cra_driver_name = "sha256-gxl",
+ .cra_priority = 400,
+ .cra_blocksize = SHA256_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_ASYNC,
+ .cra_ctxsize = sizeof(struct meson_hasher_tfm_ctx),
+ .cra_module = THIS_MODULE,
+ .cra_alignmask = 0,
+ .cra_init = meson_hasher_init,
+ },
+ .digestsize = SHA256_DIGEST_SIZE,
+ .statesize = sizeof(struct meson_hasher_req_ctx),
+ },
+ .init = meson_sha_init,
+ .update = meson_sha_update,
+ .final = meson_sha_final,
+ },
+ .alg.ahash.op = {
+ .do_one_request = meson_hasher_do_one_request,
+ },
+},
+};
+
+int meson_hasher_register(struct meson_dev *mc)
+{
+ return meson_register_algs(mc, mc_algs, ARRAY_SIZE(mc_algs));
+}
+
+void meson_hasher_unregister(struct meson_dev *mc)
+{
+ meson_unregister_algs(mc, mc_algs, ARRAY_SIZE(mc_algs));
+}
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index e1453dd2e9f4..a07b4f6b3bcc 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -5,6 +5,7 @@
* Copyright (C) 2018-2019 Corentin LABBE <[email protected]>
*/
#include <crypto/aes.h>
+#include <crypto/sha2.h>
#include <crypto/engine.h>
#include <crypto/skcipher.h>
#include <linux/debugfs.h>
@@ -23,13 +24,25 @@

#define DESC_OPMODE_ECB (0 << 26)
#define DESC_OPMODE_CBC (1 << 26)
+#define DESC_OPMODE_SHA (0 << 26)

#define DESC_MAXLEN ((1 << 17) - 1)

+#define DESC_MODE_SHA1 (0x5 << 20)
+#define DESC_MODE_SHA224 (0x7 << 20)
+#define DESC_MODE_SHA256 (0x6 << 20)
+
#define DESC_LAST BIT(18)
+#define DESC_BEGIN BIT(24)
+#define DESC_END BIT(25)
#define DESC_ENCRYPTION BIT(28)
#define DESC_OWN BIT(31)

+#define MESON_SHA_BUFFER_SIZE (SHA256_DIGEST_SIZE + 16)
+
+#define MESON_SHA_BEGIN BIT(1)
+#define MESON_SHA_FINAL BIT(2)
+
/*
* struct meson_desc - Descriptor for DMA operations
* Note that without datasheet, some are unknown
@@ -143,6 +156,38 @@ struct meson_cipher_tfm_ctx {
struct crypto_skcipher *fallback_tfm;
};

+/*
+ * struct meson_hasher_req_ctx - context for a hasher request
+ * @state: state data
+ * @partial: partial buffer data. Contains sent data which
+ * size < blocksize
+ * @partial_size: size of the partial buffer
+ * @partial_addr: physical address of partial buffer
+ * @partial_mapped: indicates is partial buffer currently mapped or not
+ * @flags: request flags (for example, is this final req or not)
+ * @flow: the flow to use for this request
+ */
+struct meson_hasher_req_ctx {
+ u8 state[SHA256_DIGEST_SIZE + 16] ____cacheline_aligned;
+ u8 partial[SHA256_BLOCK_SIZE] ____cacheline_aligned;
+ unsigned int partial_size ____cacheline_aligned;
+ dma_addr_t partial_addr;
+ bool partial_mapped;
+
+ bool begin_req;
+ bool final_req;
+ int flow;
+};
+
+/*
+ * struct meson_hasher_tfm_ctx - context for a hasher TFM
+ * @enginectx: crypto_engine used by this TFM
+ * @mc: pointer to the private data of driver handling this TFM
+ */
+struct meson_hasher_tfm_ctx {
+ struct meson_dev *mc;
+};
+
/*
* struct meson_alg_template - crypto_alg template
* @type: the CRYPTO_ALG_TYPE for this template
@@ -157,6 +202,7 @@ struct meson_alg_template {
u32 blockmode;
union {
struct skcipher_engine_alg skcipher;
+ struct ahash_engine_alg ahash;
} alg;
struct meson_dev *mc;
#ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
@@ -180,3 +226,6 @@ int meson_cipher_register(struct meson_dev *mc);
void meson_cipher_unregister(struct meson_dev *mc);
void meson_cipher_debugfs_show(struct seq_file *seq, void *v);
int meson_handle_cipher_request(struct crypto_engine *engine, void *areq);
+
+int meson_hasher_register(struct meson_dev *mc);
+void meson_hasher_unregister(struct meson_dev *mc);
--
2.34.1


2024-02-05 15:58:39

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 12/20] drivers: crypto: meson: add support for AES-CTR

This patch adds support for AES-CTR algorithm.
Tested via tcrypt and custom tests.

Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-cipher.c | 44 +++++++++++++++++++--
drivers/crypto/amlogic/amlogic-gxl.h | 1 +
2 files changed, 42 insertions(+), 3 deletions(-)

diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
index 3f42b2cc568d..828109f4a1c3 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
@@ -154,7 +154,7 @@ static void meson_setup_keyiv_descs(struct cipher_ctx *ctx)
if (ctx->tloffset)
return;

- if (blockmode == DESC_OPMODE_CBC) {
+ if (blockmode == DESC_OPMODE_CBC || blockmode == DESC_OPMODE_CTR) {
memcpy(op->keyiv + AES_MAX_KEY_SIZE, ctx->areq->iv, ivsize);
dma_sync_single_for_device(mc->dev, ctx->keyiv.addr,
ctx->keyiv.len, DMA_TO_DEVICE);
@@ -186,6 +186,7 @@ static bool meson_setup_data_descs(struct cipher_ctx *ctx)
struct meson_desc *desc = &mc->chanlist[rctx->flow].tl[ctx->tloffset];
unsigned int blocksize = crypto_skcipher_blocksize(tfm);
unsigned int blockmode = algt->blockmode;
+ unsigned int ivsize = crypto_skcipher_ivsize(tfm);
unsigned int maxlen = rounddown(DESC_MAXLEN, blocksize);
unsigned int todo;
u32 v;
@@ -204,8 +205,15 @@ static bool meson_setup_data_descs(struct cipher_ctx *ctx)
ctx->src_offset += todo;
ctx->dst_offset += todo;

+ if (blockmode == DESC_OPMODE_CTR) {
+ unsigned int nblocks = todo / blocksize;
+
+ while (nblocks--)
+ crypto_inc(ctx->areq->iv, ivsize);
+ }
+
v = DESC_OWN | blockmode | op->keymode | todo;
- if (rctx->op_dir == MESON_ENCRYPT)
+ if (rctx->op_dir == MESON_ENCRYPT || blockmode == DESC_OPMODE_CTR)
v |= DESC_ENCRYPTION;

if (!ctx->cryptlen || ctx->tloffset == MAXDESC)
@@ -322,7 +330,8 @@ static int meson_cipher(struct skcipher_request *areq)
if (ctx.keyiv.len == AES_KEYSIZE_192)
ctx.keyiv.len = AES_MAX_KEY_SIZE;

- if (algt->blockmode == DESC_OPMODE_CBC) {
+ if (algt->blockmode == DESC_OPMODE_CBC ||
+ algt->blockmode == DESC_OPMODE_CTR) {
memcpy(op->keyiv + AES_MAX_KEY_SIZE, areq->iv, ivsize);
ctx.keyiv.len = AES_MAX_KEY_SIZE + ivsize;
}
@@ -535,6 +544,35 @@ static struct meson_alg_template algs[] = {
.do_one_request = meson_handle_cipher_request,
},
},
+{
+ .type = CRYPTO_ALG_TYPE_SKCIPHER,
+ .blockmode = DESC_OPMODE_CTR,
+ .alg.skcipher.base = {
+ .base = {
+ .cra_name = "ctr(aes)",
+ .cra_driver_name = "ctr-aes-gxl",
+ .cra_priority = 400,
+ .cra_blocksize = AES_BLOCK_SIZE,
+ .cra_flags = CRYPTO_ALG_TYPE_SKCIPHER |
+ CRYPTO_ALG_ASYNC | CRYPTO_ALG_ALLOCATES_MEMORY |
+ CRYPTO_ALG_NEED_FALLBACK,
+ .cra_ctxsize = sizeof(struct meson_cipher_tfm_ctx),
+ .cra_module = THIS_MODULE,
+ .cra_alignmask = 0xf,
+ .cra_init = meson_cipher_init,
+ .cra_exit = meson_cipher_exit,
+ },
+ .min_keysize = AES_MIN_KEY_SIZE,
+ .max_keysize = AES_MAX_KEY_SIZE,
+ .ivsize = AES_BLOCK_SIZE,
+ .setkey = meson_aes_setkey,
+ .encrypt = meson_skencrypt,
+ .decrypt = meson_skdecrypt,
+ },
+ .alg.skcipher.op = {
+ .do_one_request = meson_handle_cipher_request,
+ },
+},
};

int meson_cipher_register(struct meson_dev *mc)
diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
index a07b4f6b3bcc..7f6d91e7b365 100644
--- a/drivers/crypto/amlogic/amlogic-gxl.h
+++ b/drivers/crypto/amlogic/amlogic-gxl.h
@@ -24,6 +24,7 @@

#define DESC_OPMODE_ECB (0 << 26)
#define DESC_OPMODE_CBC (1 << 26)
+#define DESC_OPMODE_CTR (2 << 26)
#define DESC_OPMODE_SHA (0 << 26)

#define DESC_MAXLEN ((1 << 17) - 1)
--
2.34.1


2024-02-05 15:58:54

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 14/20] drivers: crypto: meson: add support for G12-series

This platform data also can be used for A1 and S4 as fallback.
Tested via tcrypt module and with custom tests.

Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-core.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)

diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 429c3474028b..2ffe5994b353 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -218,11 +218,31 @@ static const struct meson_pdata meson_gxl_pdata = {
.support_192bit_key = true,
};

+static const struct meson_pdata meson_g12a_pdata = {
+ .descs_reg = 0x0,
+ .status_reg = 0x8,
+ .need_clk = false,
+ .setup_desc_cnt = 1,
+ .support_192bit_key = false,
+};
+
static const struct of_device_id meson_crypto_of_match_table[] = {
{
.compatible = "amlogic,gxl-crypto",
.data = &meson_gxl_pdata,
},
+ {
+ .compatible = "amlogic,g12a-crypto",
+ .data = &meson_g12a_pdata,
+ },
+ {
+ .compatible = "amlogic,a1-crypto",
+ .data = &meson_g12a_pdata,
+ },
+ {
+ .compatible = "amlogic,s4-crypto",
+ .data = &meson_g12a_pdata,
+ },
{},
};
MODULE_DEVICE_TABLE(of, meson_crypto_of_match_table);
--
2.34.1


2024-02-05 15:59:29

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 15/20] drivers: crypto: meson: add support for AXG-series

Tested via tcrypt module and with custom tests.

Signed-off-by: Alexey Romanov <[email protected]>
---
drivers/crypto/amlogic/amlogic-gxl-core.c | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
index 2ffe5994b353..536ad438f713 100644
--- a/drivers/crypto/amlogic/amlogic-gxl-core.c
+++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
@@ -226,6 +226,14 @@ static const struct meson_pdata meson_g12a_pdata = {
.support_192bit_key = false,
};

+static const struct meson_pdata meson_axg_pdata = {
+ .descs_reg = 0x0,
+ .status_reg = 0x8,
+ .need_clk = false,
+ .setup_desc_cnt = 3,
+ .support_192bit_key = true,
+};
+
static const struct of_device_id meson_crypto_of_match_table[] = {
{
.compatible = "amlogic,gxl-crypto",
@@ -243,6 +251,10 @@ static const struct of_device_id meson_crypto_of_match_table[] = {
.compatible = "amlogic,s4-crypto",
.data = &meson_g12a_pdata,
},
+ {
+ .compatible = "amlogic,axg-crypto",
+ .data = &meson_axg_pdata,
+ },
{},
};
MODULE_DEVICE_TABLE(of, meson_crypto_of_match_table);
--
2.34.1


2024-02-05 15:59:37

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 18/20] arch: arm64: dts: meson: s4: add crypto node

This patch adds a crypto node declaration for Amlogic S4-series.
With the Amlogic crypto driver we can use HW implementation
of SHA1/224/256 and AES algo.

Signed-off-by: Alexey Romanov <[email protected]>
---
arch/arm64/boot/dts/amlogic/meson-s4.dtsi | 6 ++++++
1 file changed, 6 insertions(+)

diff --git a/arch/arm64/boot/dts/amlogic/meson-s4.dtsi b/arch/arm64/boot/dts/amlogic/meson-s4.dtsi
index ce90b35686a2..dc05e2c5da3b 100644
--- a/arch/arm64/boot/dts/amlogic/meson-s4.dtsi
+++ b/arch/arm64/boot/dts/amlogic/meson-s4.dtsi
@@ -339,6 +339,12 @@ mux {

};

+ crypto: crypto@440400 {
+ compatible = "amlogic,s4-crypto", "amlogic,g12a-crypto";
+ reg = <0x0 0x440400 0x0 0x48>;
+ interrupts = <GIC_SPI 24 IRQ_TYPE_EDGE_RISING>;
+ };
+
gpio_intc: interrupt-controller@4080 {
compatible = "amlogic,meson-s4-gpio-intc",
"amlogic,meson-gpio-intc";
--
2.34.1


2024-02-05 16:02:28

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 20/20] arch: arm64: dts: meson: axg: add crypto node

This patch adds a crypto node declaration. With the
Amlogic crypto driver we can use HW implementation
of SHA1/224/256 and AES algo.

Signed-off-by: Alexey Romanov <[email protected]>
---
arch/arm64/boot/dts/amlogic/meson-axg.dtsi | 6 ++++++
1 file changed, 6 insertions(+)

diff --git a/arch/arm64/boot/dts/amlogic/meson-axg.dtsi b/arch/arm64/boot/dts/amlogic/meson-axg.dtsi
index 7e5ac9db93f8..39ecb894668e 100644
--- a/arch/arm64/boot/dts/amlogic/meson-axg.dtsi
+++ b/arch/arm64/boot/dts/amlogic/meson-axg.dtsi
@@ -286,6 +286,12 @@ ethmac: ethernet@ff3f0000 {
status = "disabled";
};

+ crypto: crypto@ff63e000 {
+ compatible = "amlogic,axg-crypto";
+ reg = <0x0 0xff63e000 0x0 0x48>;
+ interrupts = <GIC_SPI 180 IRQ_TYPE_EDGE_RISING>;
+ };
+
pcie_phy: phy@ff644000 {
compatible = "amlogic,axg-pcie-phy";
reg = <0x0 0xff644000 0x0 0x1c>;
--
2.34.1


2024-02-05 16:05:00

by Alexey Romanov

[permalink] [raw]
Subject: [PATCH v3 16/20] dt-bindings: crypto: meson: support new SoC's

Now crypto module available at G12A/G12B/S4/A1/SM1/AXG.

1. Add new compatibles:
- amlogic,g12a-crypto
- amlogic,s4-crypto (uses g12a-crypto as fallback)
- amlogic,a1-crypto (uses g12a-crypto as fallback)
- amlogic,axg-crypto

2. All SoC's, exclude GXL, doesn't take a clock input for
Crypto IP. Make it required only for amlogic,gxl-crypto.

3. All SoC's, exclude GXL, uses only one interrupt flow
for Crypto IP.

4. Add power-domains in schema.

Signed-off-by: Alexey Romanov <[email protected]>
---
.../bindings/crypto/amlogic,gxl-crypto.yaml | 44 +++++++++++++++----
1 file changed, 36 insertions(+), 8 deletions(-)

diff --git a/Documentation/devicetree/bindings/crypto/amlogic,gxl-crypto.yaml b/Documentation/devicetree/bindings/crypto/amlogic,gxl-crypto.yaml
index 948e11ebe4ee..62f772036b06 100644
--- a/Documentation/devicetree/bindings/crypto/amlogic,gxl-crypto.yaml
+++ b/Documentation/devicetree/bindings/crypto/amlogic,gxl-crypto.yaml
@@ -11,20 +11,30 @@ maintainers:

properties:
compatible:
- items:
- - const: amlogic,gxl-crypto
+ oneOf:
+ - items:
+ - enum:
+ - amlogic,a1-crypto
+ - amlogic,s4-crypto
+ - const: amlogic,g12a-crypto
+ - items:
+ - const: amlogic,gxl-crypto
+ - items:
+ - const: amlogic,axg-crypto
+ - items:
+ - const: amlogic,g12a-crypto

reg:
maxItems: 1

- interrupts:
- items:
- - description: Interrupt for flow 0
- - description: Interrupt for flow 1
+ interrupts: true

clocks:
maxItems: 1

+ power-domains:
+ maxItems: 1
+
clock-names:
const: blkmv

@@ -32,8 +42,26 @@ required:
- compatible
- reg
- interrupts
- - clocks
- - clock-names
+
+allOf:
+ - if:
+ properties:
+ compatible:
+ contains:
+ const: amlogic,gxl-crypto
+ then:
+ required:
+ - clocks
+ - clock-names
+ properties:
+ interrupts:
+ maxItems: 2
+ minItems: 2
+ else:
+ properties:
+ interrupts:
+ maxItems: 1
+ minItems: 1

additionalProperties: false

--
2.34.1


2024-02-06 13:14:18

by Neil Armstrong

[permalink] [raw]
Subject: Re: [PATCH v3 09/20] drivers: crypto: meson: process more than MAXDESCS descriptors

Hi,

On 05/02/2024 16:55, Alexey Romanov wrote:
> 1. The old alhorithm was not designed to process a large
> amount of memory, and therefore gave incorrect results.
>
> 2. Not all Amlogic SoC's use 3 KEY/IV descriptors.
> Add keyiv descriptors count parameter to platform data.
>
> Signed-off-by: Alexey Romanov <[email protected]>
> ---
> drivers/crypto/amlogic/amlogic-gxl-cipher.c | 443 ++++++++++++--------
> drivers/crypto/amlogic/amlogic-gxl-core.c | 1 +
> drivers/crypto/amlogic/amlogic-gxl.h | 2 +
> 3 files changed, 281 insertions(+), 165 deletions(-)
>
> diff --git a/drivers/crypto/amlogic/amlogic-gxl-cipher.c b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
> index c662c4b86e97..9c96e7b65e1e 100644
> --- a/drivers/crypto/amlogic/amlogic-gxl-cipher.c
> +++ b/drivers/crypto/amlogic/amlogic-gxl-cipher.c
> @@ -17,35 +17,41 @@
> #include <crypto/internal/skcipher.h>
> #include "amlogic-gxl.h"
>
> -static bool meson_cipher_need_fallback(struct skcipher_request *areq)
> +static bool meson_cipher_need_fallback_sg(struct skcipher_request *areq,
> + struct scatterlist *sg)
> {
> - struct scatterlist *src_sg = areq->src;
> - struct scatterlist *dst_sg = areq->dst;
> + struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
> + unsigned int blocksize = crypto_skcipher_blocksize(tfm);
> + unsigned int cryptlen = areq->cryptlen;
> +
> + while (cryptlen) {
> + unsigned int len = min(cryptlen, sg->length);
> +
> + if (!IS_ALIGNED(sg->offset, sizeof(u32)))
> + return true;
> + if (len % blocksize != 0)
> + return true;
> +
> + cryptlen -= len;
> + sg = sg_next(sg);
> + }
> +
> + return false;
> +}
>
> +static bool meson_cipher_need_fallback(struct skcipher_request *areq)
> +{
> if (areq->cryptlen == 0)
> return true;
>
> - if (sg_nents(src_sg) != sg_nents(dst_sg))
> + if (meson_cipher_need_fallback_sg(areq, areq->src))
> return true;
>
> - /* KEY/IV descriptors use 3 desc */
> - if (sg_nents(src_sg) > MAXDESC - 3 || sg_nents(dst_sg) > MAXDESC - 3)
> - return true;
> + if (areq->dst == areq->src)
> + return false;
>
> - while (src_sg && dst_sg) {
> - if ((src_sg->length % 16) != 0)
> - return true;
> - if ((dst_sg->length % 16) != 0)
> - return true;
> - if (src_sg->length != dst_sg->length)
> - return true;
> - if (!IS_ALIGNED(src_sg->offset, sizeof(u32)))
> - return true;
> - if (!IS_ALIGNED(dst_sg->offset, sizeof(u32)))
> - return true;
> - src_sg = sg_next(src_sg);
> - dst_sg = sg_next(dst_sg);
> - }
> + if (meson_cipher_need_fallback_sg(areq, areq->dst))
> + return true;
>
> return false;
> }
> @@ -76,6 +82,211 @@ static int meson_cipher_do_fallback(struct skcipher_request *areq)
> return err;
> }
>
> +struct cipher_ctx {
> + struct {
> + dma_addr_t addr;
> + unsigned int len;
> + } keyiv;
> +
> + struct skcipher_request *areq;
> + struct scatterlist *src_sg;
> + struct scatterlist *dst_sg;
> +
> + unsigned int src_offset;
> + unsigned int dst_offset;
> + unsigned int cryptlen;
> + unsigned int tloffset;
> +};
> +
> +static int meson_map_scatterlist(struct skcipher_request *areq, struct meson_dev *mc)
> +{
> + int nr_sgs, nr_sgd;
> +
> + if (areq->src == areq->dst) {
> + nr_sgs = dma_map_sg(mc->dev, areq->src, sg_nents(areq->src),
> + DMA_BIDIRECTIONAL);
> + if (!nr_sgs) {
> + dev_err(mc->dev, "Invalid SG count %d\n", nr_sgs);
> + return -EINVAL;
> + }
> + } else {
> + nr_sgs = dma_map_sg(mc->dev, areq->src, sg_nents(areq->src),
> + DMA_TO_DEVICE);
> + if (!nr_sgs) {
> + dev_err(mc->dev, "Invalid SG count %d\n", nr_sgs);
> + return -EINVAL;
> + }
> +
> + nr_sgd = dma_map_sg(mc->dev, areq->dst, sg_nents(areq->dst),
> + DMA_FROM_DEVICE);
> + if (!nr_sgd) {
> + dev_err(mc->dev, "Invalid SG count %d\n", nr_sgd);
> + return -EINVAL;
> + }
> + }
> +
> + return 0;
> +}
> +
> +static void meson_unmap_scatterlist(struct skcipher_request *areq, struct meson_dev *mc)
> +{
> + if (areq->src == areq->dst) {
> + dma_unmap_sg(mc->dev, areq->src, sg_nents(areq->src), DMA_BIDIRECTIONAL);
> + } else {
> + dma_unmap_sg(mc->dev, areq->src, sg_nents(areq->src), DMA_TO_DEVICE);
> + dma_unmap_sg(mc->dev, areq->dst, sg_nents(areq->dst), DMA_FROM_DEVICE);
> + }
> +}
> +
> +static void meson_setup_keyiv_descs(struct cipher_ctx *ctx)
> +{
> + struct meson_cipher_req_ctx *rctx = skcipher_request_ctx(ctx->areq);
> + struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(ctx->areq);
> + struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
> + struct meson_alg_template *algt = container_of(alg,
> + struct meson_alg_template, alg.skcipher.base);
> + struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
> + struct meson_dev *mc = op->mc;
> + unsigned int ivsize = crypto_skcipher_ivsize(tfm);
> + unsigned int blockmode = algt->blockmode;
> + int i;
> +
> + if (ctx->tloffset)
> + return;
> +
> + if (blockmode == DESC_OPMODE_CBC) {
> + memcpy(op->key + AES_MAX_KEY_SIZE, ctx->areq->iv, ivsize);
> + ctx->keyiv.len = AES_MAX_KEY_SIZE + ivsize;
> + dma_sync_single_for_device(mc->dev, ctx->keyiv.addr,
> + ctx->keyiv.len, DMA_TO_DEVICE);
> + }
> +
> + for (i = 0; i < mc->pdata->setup_desc_cnt; i++) {
> + struct meson_desc *desc =
> + &mc->chanlist[rctx->flow].tl[ctx->tloffset];
> + int offset = i * 16;
> +
> + desc->t_src = cpu_to_le32(ctx->keyiv.addr + offset);
> + desc->t_dst = cpu_to_le32(offset);
> + desc->t_status = cpu_to_le32(DESC_OWN | DESC_MODE_KEY | ctx->keyiv.len);
> +
> + ctx->tloffset++;
> + }
> +}
> +
> +static bool meson_setup_data_descs(struct cipher_ctx *ctx)
> +{
> + struct meson_cipher_req_ctx *rctx = skcipher_request_ctx(ctx->areq);
> + struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(ctx->areq);
> + struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
> + struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
> + struct meson_alg_template *algt = container_of(alg,
> + struct meson_alg_template,
> + alg.skcipher.base);
> + struct meson_dev *mc = op->mc;
> + struct meson_desc *desc = &mc->chanlist[rctx->flow].tl[ctx->tloffset];
> + unsigned int blocksize = crypto_skcipher_blocksize(tfm);
> + unsigned int blockmode = algt->blockmode;
> + unsigned int maxlen = rounddown(DESC_MAXLEN, blocksize);
> + unsigned int todo;
> + u32 v;
> +
> + ctx->tloffset++;
> +
> + todo = min(ctx->cryptlen, maxlen);
> + todo = min(todo, ctx->cryptlen);
> + todo = min(todo, sg_dma_len(ctx->src_sg) - ctx->src_offset);
> + todo = min(todo, sg_dma_len(ctx->dst_sg) - ctx->dst_offset);
> +
> + desc->t_src = cpu_to_le32(sg_dma_address(ctx->src_sg) + ctx->src_offset);
> + desc->t_dst = cpu_to_le32(sg_dma_address(ctx->dst_sg) + ctx->dst_offset);
> +
> + ctx->cryptlen -= todo;
> + ctx->src_offset += todo;
> + ctx->dst_offset += todo;
> +
> + v = DESC_OWN | blockmode | op->keymode | todo;
> + if (rctx->op_dir == MESON_ENCRYPT)
> + v |= DESC_ENCRYPTION;
> +
> + if (!ctx->cryptlen || ctx->tloffset == MAXDESC)
> + v |= DESC_LAST;
> +
> + desc->t_status = cpu_to_le32(v);
> +
> + return v & DESC_LAST;
> +}
> +
> +static int meson_kick_hardware(struct cipher_ctx *ctx)
> +{
> + struct meson_cipher_req_ctx *rctx = skcipher_request_ctx(ctx->areq);
> + struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(ctx->areq);
> + struct meson_cipher_tfm_ctx *op = crypto_skcipher_ctx(tfm);
> + struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
> + struct meson_alg_template *algt = container_of(alg,
> + struct meson_alg_template,
> + alg.skcipher.base);
> + struct meson_dev *mc = op->mc;
> + unsigned int ivsize = crypto_skcipher_ivsize(tfm);
> + unsigned int blockmode = algt->blockmode;
> + enum dma_data_direction new_iv_dir;
> + dma_addr_t new_iv_phys;
> + void *new_iv;
> + int err;
> +
> + if (blockmode == DESC_OPMODE_CBC) {
> + struct scatterlist *sg;
> + unsigned int offset;
> +
> + if (rctx->op_dir == MESON_ENCRYPT) {
> + sg = ctx->dst_sg;
> + offset = ctx->dst_offset;
> + new_iv_dir = DMA_FROM_DEVICE;
> + } else {
> + sg = ctx->src_sg;
> + offset = ctx->src_offset;
> + new_iv_dir = DMA_TO_DEVICE;
> + }
> +
> + if (ctx->areq->src == ctx->areq->dst)
> + new_iv_dir = DMA_BIDIRECTIONAL;
> +
> + offset -= ivsize;
> + new_iv = sg_virt(sg) + offset;
> + new_iv_phys = sg_dma_address(sg) + offset;
> + }
> +
> + if (blockmode == DESC_OPMODE_CBC &&
> + rctx->op_dir == MESON_DECRYPT) {
> + dma_sync_single_for_cpu(mc->dev, new_iv_phys,
> + ivsize, new_iv_dir);
> + memcpy(ctx->areq->iv, new_iv, ivsize);
> + }
> +
> + reinit_completion(&mc->chanlist[rctx->flow].complete);
> + meson_dma_start(mc, rctx->flow);
> + err = wait_for_completion_interruptible_timeout(
> + &mc->chanlist[rctx->flow].complete, msecs_to_jiffies(500));
> + if (err == 0) {
> + dev_err(mc->dev, "DMA timeout for flow %d\n", rctx->flow);
> + return -EINVAL;
> + } else if (err < 0) {
> + dev_err(mc->dev, "Waiting for DMA completion is failed (%d)\n", err);
> + return err;
> + }
> +
> + if (blockmode == DESC_OPMODE_CBC &&
> + rctx->op_dir == MESON_ENCRYPT) {
> + dma_sync_single_for_cpu(mc->dev, new_iv_phys,
> + ivsize, new_iv_dir);
> + memcpy(ctx->areq->iv, new_iv, ivsize);
> + }
> +
> + ctx->tloffset = 0;
> +
> + return 0;
> +}
> +
> static int meson_cipher(struct skcipher_request *areq)
> {
> struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
> @@ -84,176 +295,78 @@ static int meson_cipher(struct skcipher_request *areq)
> struct meson_dev *mc = op->mc;
> struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
> struct meson_alg_template *algt;
> - int flow = rctx->flow;
> - unsigned int todo, eat, len;
> - struct scatterlist *src_sg = areq->src;
> - struct scatterlist *dst_sg = areq->dst;
> - struct meson_desc *desc;
> - int nr_sgs, nr_sgd;
> - int i, err = 0;
> - unsigned int keyivlen, ivsize, offset, tloffset;
> - dma_addr_t phykeyiv;
> - void *backup_iv = NULL, *bkeyiv;
> - u32 v;
> -
> - algt = container_of(alg, struct meson_alg_template, alg.skcipher.base);
> + struct cipher_ctx ctx = {
> + .areq = areq,
> + .src_offset = 0,
> + .dst_offset = 0,
> + .src_sg = areq->src,
> + .dst_sg = areq->dst,
> + .cryptlen = areq->cryptlen,
> + };
> + unsigned int ivsize = crypto_skcipher_ivsize(tfm);

I'm getting build errors with W=1 build:

drivers/crypto/amlogic/amlogic-gxl-cipher.c:306:22: error: unused variable ‘ivsize’ [-Werror=unused-variable]
306 | unsigned int ivsize = crypto_skcipher_ivsize(tfm);
| ^~~~~~

> + int err;
>
> - dev_dbg(mc->dev, "%s %s %u %x IV(%u) key=%u flow=%d\n", __func__,
> + dev_dbg(mc->dev, "%s %s %u %x IV(%u) key=%u ctx.flow=%d\n", __func__,
> crypto_tfm_alg_name(areq->base.tfm),
> areq->cryptlen,
> rctx->op_dir, crypto_skcipher_ivsize(tfm),
> - op->keylen, flow);
> + op->keylen, rctx->flow);
> +
> + algt = container_of(alg, struct meson_alg_template, alg.skcipher.base);
>
> #ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG
> algt->stat_req++;
> - mc->chanlist[flow].stat_req++;
> + mc->chanlist[rctx->flow].stat_req++;
> #endif
>
> - /*
> - * The hardware expect a list of meson_desc structures.
> - * The 2 first structures store key
> - * The third stores IV
> - */
> - bkeyiv = kzalloc(48, GFP_KERNEL | GFP_DMA);
> - if (!bkeyiv)
> + op->key = kzalloc(48, GFP_KERNEL | GFP_DMA);
> + if (!op.key)
> return -ENOMEM;

drivers/crypto/amlogic/amlogic-gxl-cipher.c: In function ‘meson_cipher’:
drivers/crypto/amlogic/amlogic-gxl-cipher.c:323:16: error: ‘op’ is a pointer; did you mean to use ‘->’?
323 | if (!op.key)
| ^

>
> - memcpy(bkeyiv, op->key, op->keylen);
> - keyivlen = op->keylen;
> + memcpy(op->key, op->key, op->keylen);
> + ctx.keyiv.len = op->keylen;
> + if (ctx.keyiv.len == AES_KEYSIZE_192)
> + ctx.keyiv.len = AES_MAX_KEY_SIZE;
>
> - ivsize = crypto_skcipher_ivsize(tfm);
> - if (areq->iv && ivsize > 0) {
> - if (ivsize > areq->cryptlen) {
> - dev_err(mc->dev, "invalid ivsize=%d vs len=%d\n", ivsize, areq->cryptlen);
> - err = -EINVAL;
> - goto theend;
> - }
> - memcpy(bkeyiv + 32, areq->iv, ivsize);
> - keyivlen = 48;
> - if (rctx->op_dir == MESON_DECRYPT) {
> - backup_iv = kzalloc(ivsize, GFP_KERNEL);
> - if (!backup_iv) {
> - err = -ENOMEM;
> - goto theend;
> - }
> - offset = areq->cryptlen - ivsize;
> - scatterwalk_map_and_copy(backup_iv, areq->src, offset,
> - ivsize, 0);
> - }
> - }
> - if (keyivlen == AES_KEYSIZE_192)
> - keyivlen = AES_MAX_KEY_SIZE;
> -
> - phykeyiv = dma_map_single(mc->dev, bkeyiv, keyivlen,
> + ctx.keyiv.addr = dma_map_single(mc->dev, op->key, ctx.keyiv.len,
> DMA_TO_DEVICE);
> - err = dma_mapping_error(mc->dev, phykeyiv);
> + err = dma_mapping_error(mc->dev, ctx.keyiv.addr);
> if (err) {
> dev_err(mc->dev, "Cannot DMA MAP KEY IV\n");
> goto theend;
> }
>
> - tloffset = 0;
> - eat = 0;
> - i = 0;
> - while (keyivlen > eat) {
> - desc = &mc->chanlist[flow].tl[tloffset];
> - memset(desc, 0, sizeof(struct meson_desc));
> - todo = min(keyivlen - eat, 16u);
> - desc->t_src = cpu_to_le32(phykeyiv + i * 16);
> - desc->t_dst = cpu_to_le32(i * 16);
> - v = DESC_MODE_KEY | DESC_OWN | 16;
> - desc->t_status = cpu_to_le32(v);
> -
> - eat += todo;
> - i++;
> - tloffset++;
> - }
> -
> - if (areq->src == areq->dst) {
> - nr_sgs = dma_map_sg(mc->dev, areq->src, sg_nents(areq->src),
> - DMA_BIDIRECTIONAL);
> - if (!nr_sgs) {
> - dev_err(mc->dev, "Invalid SG count %d\n", nr_sgs);
> - err = -EINVAL;
> - goto theend;
> - }
> - nr_sgd = nr_sgs;
> - } else {
> - nr_sgs = dma_map_sg(mc->dev, areq->src, sg_nents(areq->src),
> - DMA_TO_DEVICE);
> - if (!nr_sgs || nr_sgs > MAXDESC - 3) {
> - dev_err(mc->dev, "Invalid SG count %d\n", nr_sgs);
> - err = -EINVAL;
> - goto theend;
> - }
> - nr_sgd = dma_map_sg(mc->dev, areq->dst, sg_nents(areq->dst),
> - DMA_FROM_DEVICE);
> - if (!nr_sgd || nr_sgd > MAXDESC - 3) {
> - dev_err(mc->dev, "Invalid SG count %d\n", nr_sgd);
> - err = -EINVAL;
> - goto theend;
> - }
> - }
> -
> - src_sg = areq->src;
> - dst_sg = areq->dst;
> - len = areq->cryptlen;
> - while (src_sg) {
> - desc = &mc->chanlist[flow].tl[tloffset];
> - memset(desc, 0, sizeof(struct meson_desc));
> -
> - desc->t_src = cpu_to_le32(sg_dma_address(src_sg));
> - desc->t_dst = cpu_to_le32(sg_dma_address(dst_sg));
> - todo = min(len, sg_dma_len(src_sg));
> - v = op->keymode | DESC_OWN | todo | algt->blockmode;
> - if (rctx->op_dir)
> - v |= DESC_ENCRYPTION;
> - len -= todo;
> -
> - if (!sg_next(src_sg))
> - v |= DESC_LAST;
> - desc->t_status = cpu_to_le32(v);
> - tloffset++;
> - src_sg = sg_next(src_sg);
> - dst_sg = sg_next(dst_sg);
> - }
> + err = meson_map_scatterlist(areq, mc);
> + if (err)
> + goto theend;
>
> - reinit_completion(&mc->chanlist[flow].complete);
> - meson_dma_start(mc, flow);
> + ctx.tloffset = 0;
>
> - err = wait_for_completion_interruptible_timeout(&mc->chanlist[flow].complete,
> - msecs_to_jiffies(500));
> - if (err == 0) {
> - dev_err(mc->dev, "DMA timeout for flow %d\n", flow);
> - err = -EINVAL;
> - } else if (err < 0) {
> - dev_err(mc->dev, "Waiting for DMA completion is failed (%d)\n", err);
> - } else {
> - /* No error */
> - err = 0;
> - }
> + while (ctx.cryptlen) {
> + meson_setup_keyiv_descs(&ctx);
>
> - dma_unmap_single(mc->dev, phykeyiv, keyivlen, DMA_TO_DEVICE);
> + if (meson_setup_data_descs(&ctx)) {
> + err = meson_kick_hardware(&ctx);
> + if (err)
> + break;
> + }
>
> - if (areq->src == areq->dst) {
> - dma_unmap_sg(mc->dev, areq->src, sg_nents(areq->src), DMA_BIDIRECTIONAL);
> - } else {
> - dma_unmap_sg(mc->dev, areq->src, sg_nents(areq->src), DMA_TO_DEVICE);
> - dma_unmap_sg(mc->dev, areq->dst, sg_nents(areq->dst), DMA_FROM_DEVICE);
> - }
> + if (ctx.src_offset == sg_dma_len(ctx.src_sg)) {
> + ctx.src_offset = 0;
> + ctx.src_sg = sg_next(ctx.src_sg);
> + }
>
> - if (areq->iv && ivsize > 0) {
> - if (rctx->op_dir == MESON_DECRYPT) {
> - memcpy(areq->iv, backup_iv, ivsize);
> - } else {
> - scatterwalk_map_and_copy(areq->iv, areq->dst,
> - areq->cryptlen - ivsize,
> - ivsize, 0);
> + if (ctx.dst_offset == sg_dma_len(ctx.dst_sg)) {
> + ctx.dst_offset = 0;
> + ctx.dst_sg = sg_next(ctx.dst_sg);
> }
> }
> +
> + dma_unmap_single(mc->dev, ctx.keyiv.addr, ctx.keyiv.len, DMA_TO_DEVICE);
> + meson_unmap_scatterlist(areq, mc);
> +
> theend:
> - kfree_sensitive(bkeyiv);
> - kfree_sensitive(backup_iv);
> + kfree_sensitive(op->key);
>
> return err;
> }
> diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c
> index 22ff2768b5e5..f93e14f5717d 100644
> --- a/drivers/crypto/amlogic/amlogic-gxl-core.c
> +++ b/drivers/crypto/amlogic/amlogic-gxl-core.c
> @@ -199,6 +199,7 @@ static const struct meson_pdata meson_gxl_pdata = {
> .descs_reg = 0x0,
> .status_reg = 0x4,
> .need_clk = true,
> + .setup_desc_cnt = 3,
> };
>
> static const struct of_device_id meson_crypto_of_match_table[] = {
> diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h
> index a0d83c82906d..eb2f8cd72b65 100644
> --- a/drivers/crypto/amlogic/amlogic-gxl.h
> +++ b/drivers/crypto/amlogic/amlogic-gxl.h
> @@ -83,11 +83,13 @@ struct meson_flow {
> * @reg_descs: offset to descriptors register
> * @reg_status: offset to status register
> * @need_clk: clock input is needed
> + * @setup_desc_cnt: number of setup descriptor to configure.
> */
> struct meson_pdata {
> u32 descs_reg;
> u32 status_reg;
> bool need_clk;
> + u32 setup_desc_cnt;
> };
>
> /*

Thanks,
Neil


2024-02-08 08:38:50

by Krzysztof Kozlowski

[permalink] [raw]
Subject: Re: [PATCH v3 16/20] dt-bindings: crypto: meson: support new SoC's

On 05/02/2024 16:55, Alexey Romanov wrote:
> Now crypto module available at G12A/G12B/S4/A1/SM1/AXG.
>
> 1. Add new compatibles:
> - amlogic,g12a-crypto
> - amlogic,s4-crypto (uses g12a-crypto as fallback)
> - amlogic,a1-crypto (uses g12a-crypto as fallback)
> - amlogic,axg-crypto
>
> 2. All SoC's, exclude GXL, doesn't take a clock input for
> Crypto IP. Make it required only for amlogic,gxl-crypto.
>
> 3. All SoC's, exclude GXL, uses only one interrupt flow
> for Crypto IP.
>
> 4. Add power-domains in schema.
>
> Signed-off-by: Alexey Romanov <[email protected]>
> ---
> .../bindings/crypto/amlogic,gxl-crypto.yaml | 44 +++++++++++++++----
> 1 file changed, 36 insertions(+), 8 deletions(-)
>
> diff --git a/Documentation/devicetree/bindings/crypto/amlogic,gxl-crypto.yaml b/Documentation/devicetree/bindings/crypto/amlogic,gxl-crypto.yaml
> index 948e11ebe4ee..62f772036b06 100644
> --- a/Documentation/devicetree/bindings/crypto/amlogic,gxl-crypto.yaml
> +++ b/Documentation/devicetree/bindings/crypto/amlogic,gxl-crypto.yaml
> @@ -11,20 +11,30 @@ maintainers:
>
> properties:
> compatible:
> - items:
> - - const: amlogic,gxl-crypto
> + oneOf:
> + - items:
> + - enum:
> + - amlogic,a1-crypto
> + - amlogic,s4-crypto
> + - const: amlogic,g12a-crypto
> + - items:
> + - const: amlogic,gxl-crypto
> + - items:
> + - const: amlogic,axg-crypto
> + - items:
> + - const: amlogic,g12a-crypto

You just ignored my comment, so repeat: that's just enum with three entries.

>
> reg:
> maxItems: 1
>
> - interrupts:
> - items:
> - - description: Interrupt for flow 0
> - - description: Interrupt for flow 1
> + interrupts: true

No, you need widest constraints here.

>
> clocks:
> maxItems: 1
>
> + power-domains:
> + maxItems: 1
> +
> clock-names:
> const: blkmv
>
> @@ -32,8 +42,26 @@ required:
> - compatible
> - reg
> - interrupts
> - - clocks
> - - clock-names
> +
> +allOf:
> + - if:
> + properties:
> + compatible:
> + contains:
> + const: amlogic,gxl-crypto
> + then:
> + required:
> + - clocks
> + - clock-names
> + properties:
> + interrupts:
> + maxItems: 2
> + minItems: 2

Instead describe items like it was in original code.

> + else:
> + properties:
> + interrupts:
> + maxItems: 1
> + minItems: 1

and what interrupt is expected here?

>
> additionalProperties: false
>

Best regards,
Krzysztof