Hello
The rockchip crypto driver is broken and do not pass self-tests.
This serie's goal is to permit to become usable and pass self-tests.
This whole serie is tested on a rk3328-rock64, rk3288-miqi and
rk3399-khadas-edge-v with selftests (with CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y)
Regards
Changes since v1:
- select CRYPTO_ENGINE
- forgot to free fallbacks TFMs
- fixed kernel test robots warning
- add the PM patch
Changes since v2:
- Added DMA clock back to 3288 since it dont work without it
- fallback needed to select CBC and ECB configs
- Added support for rk3399
- Added more patch (style, read_poll_timeout)
Changes since v3:
- full rewrite of support for RK3399
- splited dt-binding patch in two
Changes since v4:
- Another full rewrite of support for RK3399
- Fixed dt-binding from Krzysztof Kozlowski's comments
- Use readl_poll_timeout() instead of read_poll_timeout()
- Rewrite the fallback SG tests
Changes since v5:
- fixed errors in DT binding patch
Change since v6:
- remove quotes around const values in dt-bindings
Corentin Labbe (33):
crypto: rockchip: use dev_err for error message about interrupt
crypto: rockchip: do not use uninitialized variable
crypto: rockchip: do not do custom power management
crypto: rockchip: fix privete/private typo
crypto: rockchip: do not store mode globally
crypto: rockchip: add fallback for cipher
crypto: rockchip: add fallback for ahash
crypto: rockchip: better handle cipher key
crypto: rockchip: remove non-aligned handling
crypto: rockchip: rework by using crypto_engine
crypto: rockchip: rewrite type
crypto: rockchip: add debugfs
crypto: rockchip: introduce PM
crypto: rockchip: handle reset also in PM
crypto: rockchip: use clk_bulk to simplify clock management
crypto: rockchip: add myself as maintainer
crypto: rockchip: use read_poll_timeout
crypto: rockchip: fix style issue
crypto: rockchip: add support for rk3328
crypto: rockchip: rename ablk functions to cipher
crypto: rockchip: rework rk_handle_req function
crypto: rockchip: use a rk_crypto_info variable instead of lot of
indirection
crypto: rockchip: use the rk_crypto_info given as parameter
dt-bindings: crypto: convert rockchip-crypto to YAML
dt-bindings: crypto: rockchip: convert to new driver bindings
clk: rk3399: use proper crypto0 name
arm64: dts: rockchip: add rk3328 crypto node
arm64: dts: rockchip: rk3399: add crypto node
crypto: rockchip: store crypto_info in request context
crypto: rockchip: Check for clocks numbers and their frequencies
crypto: rockchip: rk_ahash_reg_init use crypto_info from parameter
crypto: rockchip: permit to have more than one reset
crypto: rockchip: Add support for RK3399
.../crypto/rockchip,rk3288-crypto.yaml | 133 +++++
.../bindings/crypto/rockchip-crypto.txt | 28 -
MAINTAINERS | 7 +
arch/arm64/boot/dts/rockchip/rk3328.dtsi | 11 +
arch/arm64/boot/dts/rockchip/rk3399.dtsi | 20 +
drivers/crypto/Kconfig | 15 +
drivers/crypto/rockchip/rk3288_crypto.c | 511 ++++++++--------
drivers/crypto/rockchip/rk3288_crypto.h | 107 ++--
drivers/crypto/rockchip/rk3288_crypto_ahash.c | 267 +++++----
.../crypto/rockchip/rk3288_crypto_skcipher.c | 543 ++++++++++--------
include/dt-bindings/clock/rk3399-cru.h | 6 +-
11 files changed, 953 insertions(+), 695 deletions(-)
create mode 100644 Documentation/devicetree/bindings/crypto/rockchip,rk3288-crypto.yaml
delete mode 100644 Documentation/devicetree/bindings/crypto/rockchip-crypto.txt
--
2.35.1
Now driver have fallback for un-aligned cases, remove all code handling
those cases.
Fixes: ce0183cb6464b ("crypto: rockchip - switch to skcipher API")
Signed-off-by: Corentin Labbe <[email protected]>
---
drivers/crypto/rockchip/rk3288_crypto.c | 69 +++++--------------
drivers/crypto/rockchip/rk3288_crypto.h | 4 --
drivers/crypto/rockchip/rk3288_crypto_ahash.c | 22 ++----
.../crypto/rockchip/rk3288_crypto_skcipher.c | 39 +++--------
4 files changed, 31 insertions(+), 103 deletions(-)
diff --git a/drivers/crypto/rockchip/rk3288_crypto.c b/drivers/crypto/rockchip/rk3288_crypto.c
index 4cff49b82983..b3db096e2ec2 100644
--- a/drivers/crypto/rockchip/rk3288_crypto.c
+++ b/drivers/crypto/rockchip/rk3288_crypto.c
@@ -88,63 +88,26 @@ static int rk_load_data(struct rk_crypto_info *dev,
{
unsigned int count;
- dev->aligned = dev->aligned ?
- check_alignment(sg_src, sg_dst, dev->align_size) :
- dev->aligned;
- if (dev->aligned) {
- count = min(dev->left_bytes, sg_src->length);
- dev->left_bytes -= count;
-
- if (!dma_map_sg(dev->dev, sg_src, 1, DMA_TO_DEVICE)) {
- dev_err(dev->dev, "[%s:%d] dma_map_sg(src) error\n",
+ count = min(dev->left_bytes, sg_src->length);
+ dev->left_bytes -= count;
+
+ if (!dma_map_sg(dev->dev, sg_src, 1, DMA_TO_DEVICE)) {
+ dev_err(dev->dev, "[%s:%d] dma_map_sg(src) error\n",
__func__, __LINE__);
- return -EINVAL;
- }
- dev->addr_in = sg_dma_address(sg_src);
+ return -EINVAL;
+ }
+ dev->addr_in = sg_dma_address(sg_src);
- if (sg_dst) {
- if (!dma_map_sg(dev->dev, sg_dst, 1, DMA_FROM_DEVICE)) {
- dev_err(dev->dev,
+ if (sg_dst) {
+ if (!dma_map_sg(dev->dev, sg_dst, 1, DMA_FROM_DEVICE)) {
+ dev_err(dev->dev,
"[%s:%d] dma_map_sg(dst) error\n",
__func__, __LINE__);
- dma_unmap_sg(dev->dev, sg_src, 1,
- DMA_TO_DEVICE);
- return -EINVAL;
- }
- dev->addr_out = sg_dma_address(sg_dst);
- }
- } else {
- count = (dev->left_bytes > PAGE_SIZE) ?
- PAGE_SIZE : dev->left_bytes;
-
- if (!sg_pcopy_to_buffer(dev->first, dev->src_nents,
- dev->addr_vir, count,
- dev->total - dev->left_bytes)) {
- dev_err(dev->dev, "[%s:%d] pcopy err\n",
- __func__, __LINE__);
+ dma_unmap_sg(dev->dev, sg_src, 1,
+ DMA_TO_DEVICE);
return -EINVAL;
}
- dev->left_bytes -= count;
- sg_init_one(&dev->sg_tmp, dev->addr_vir, count);
- if (!dma_map_sg(dev->dev, &dev->sg_tmp, 1, DMA_TO_DEVICE)) {
- dev_err(dev->dev, "[%s:%d] dma_map_sg(sg_tmp) error\n",
- __func__, __LINE__);
- return -ENOMEM;
- }
- dev->addr_in = sg_dma_address(&dev->sg_tmp);
-
- if (sg_dst) {
- if (!dma_map_sg(dev->dev, &dev->sg_tmp, 1,
- DMA_FROM_DEVICE)) {
- dev_err(dev->dev,
- "[%s:%d] dma_map_sg(sg_tmp) error\n",
- __func__, __LINE__);
- dma_unmap_sg(dev->dev, &dev->sg_tmp, 1,
- DMA_TO_DEVICE);
- return -ENOMEM;
- }
- dev->addr_out = sg_dma_address(&dev->sg_tmp);
- }
+ dev->addr_out = sg_dma_address(sg_dst);
}
dev->count = count;
return 0;
@@ -154,11 +117,11 @@ static void rk_unload_data(struct rk_crypto_info *dev)
{
struct scatterlist *sg_in, *sg_out;
- sg_in = dev->aligned ? dev->sg_src : &dev->sg_tmp;
+ sg_in = dev->sg_src;
dma_unmap_sg(dev->dev, sg_in, 1, DMA_TO_DEVICE);
if (dev->sg_dst) {
- sg_out = dev->aligned ? dev->sg_dst : &dev->sg_tmp;
+ sg_out = dev->sg_dst;
dma_unmap_sg(dev->dev, sg_out, 1, DMA_FROM_DEVICE);
}
}
diff --git a/drivers/crypto/rockchip/rk3288_crypto.h b/drivers/crypto/rockchip/rk3288_crypto.h
index 540b81a14b9b..a7de5738f6dc 100644
--- a/drivers/crypto/rockchip/rk3288_crypto.h
+++ b/drivers/crypto/rockchip/rk3288_crypto.h
@@ -204,12 +204,8 @@ struct rk_crypto_info {
/* the public variable */
struct scatterlist *sg_src;
struct scatterlist *sg_dst;
- struct scatterlist sg_tmp;
struct scatterlist *first;
unsigned int left_bytes;
- void *addr_vir;
- int aligned;
- int align_size;
size_t src_nents;
size_t dst_nents;
unsigned int total;
diff --git a/drivers/crypto/rockchip/rk3288_crypto_ahash.c b/drivers/crypto/rockchip/rk3288_crypto_ahash.c
index 16009bb0bf16..c762e462eb57 100644
--- a/drivers/crypto/rockchip/rk3288_crypto_ahash.c
+++ b/drivers/crypto/rockchip/rk3288_crypto_ahash.c
@@ -236,8 +236,6 @@ static int rk_ahash_start(struct rk_crypto_info *dev)
dev->total = req->nbytes;
dev->left_bytes = req->nbytes;
- dev->aligned = 0;
- dev->align_size = 4;
dev->sg_dst = NULL;
dev->sg_src = req->src;
dev->first = req->src;
@@ -272,15 +270,13 @@ static int rk_ahash_crypto_rx(struct rk_crypto_info *dev)
dev->unload_data(dev);
if (dev->left_bytes) {
- if (dev->aligned) {
- if (sg_is_last(dev->sg_src)) {
- dev_warn(dev->dev, "[%s:%d], Lack of data\n",
- __func__, __LINE__);
- err = -ENOMEM;
- goto out_rx;
- }
- dev->sg_src = sg_next(dev->sg_src);
+ if (sg_is_last(dev->sg_src)) {
+ dev_warn(dev->dev, "[%s:%d], Lack of data\n",
+ __func__, __LINE__);
+ err = -ENOMEM;
+ goto out_rx;
}
+ dev->sg_src = sg_next(dev->sg_src);
err = rk_ahash_set_data_start(dev);
} else {
/*
@@ -318,11 +314,6 @@ static int rk_cra_hash_init(struct crypto_tfm *tfm)
algt = container_of(alg, struct rk_crypto_tmp, alg.hash);
tctx->dev = algt->dev;
- tctx->dev->addr_vir = (void *)__get_free_page(GFP_KERNEL);
- if (!tctx->dev->addr_vir) {
- dev_err(tctx->dev->dev, "failed to kmalloc for addr_vir\n");
- return -ENOMEM;
- }
tctx->dev->start = rk_ahash_start;
tctx->dev->update = rk_ahash_crypto_rx;
tctx->dev->complete = rk_ahash_crypto_complete;
@@ -345,7 +336,6 @@ static void rk_cra_hash_exit(struct crypto_tfm *tfm)
{
struct rk_ahash_ctx *tctx = crypto_tfm_ctx(tfm);
- free_page((unsigned long)tctx->dev->addr_vir);
crypto_free_ahash(tctx->fallback_tfm);
}
diff --git a/drivers/crypto/rockchip/rk3288_crypto_skcipher.c b/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
index 1ef94f8db2c5..d067b7f09165 100644
--- a/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
+++ b/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
@@ -356,7 +356,6 @@ static int rk_ablk_start(struct rk_crypto_info *dev)
dev->src_nents = sg_nents(req->src);
dev->sg_dst = req->dst;
dev->dst_nents = sg_nents(req->dst);
- dev->aligned = 1;
spin_lock_irqsave(&dev->lock, flags);
rk_ablk_hw_init(dev);
@@ -376,13 +375,9 @@ static void rk_iv_copyback(struct rk_crypto_info *dev)
/* Update the IV buffer to contain the next IV for encryption mode. */
if (!(rctx->mode & RK_CRYPTO_DEC)) {
- if (dev->aligned) {
- memcpy(req->iv, sg_virt(dev->sg_dst) +
- dev->sg_dst->length - ivsize, ivsize);
- } else {
- memcpy(req->iv, dev->addr_vir +
- dev->count - ivsize, ivsize);
- }
+ memcpy(req->iv,
+ sg_virt(dev->sg_dst) + dev->sg_dst->length - ivsize,
+ ivsize);
}
}
@@ -420,27 +415,16 @@ static int rk_ablk_rx(struct rk_crypto_info *dev)
skcipher_request_cast(dev->async_req);
dev->unload_data(dev);
- if (!dev->aligned) {
- if (!sg_pcopy_from_buffer(req->dst, dev->dst_nents,
- dev->addr_vir, dev->count,
- dev->total - dev->left_bytes -
- dev->count)) {
- err = -EINVAL;
- goto out_rx;
- }
- }
if (dev->left_bytes) {
rk_update_iv(dev);
- if (dev->aligned) {
- if (sg_is_last(dev->sg_src)) {
- dev_err(dev->dev, "[%s:%d] Lack of data\n",
+ if (sg_is_last(dev->sg_src)) {
+ dev_err(dev->dev, "[%s:%d] Lack of data\n",
__func__, __LINE__);
- err = -ENOMEM;
- goto out_rx;
- }
- dev->sg_src = sg_next(dev->sg_src);
- dev->sg_dst = sg_next(dev->sg_dst);
+ err = -ENOMEM;
+ goto out_rx;
}
+ dev->sg_src = sg_next(dev->sg_src);
+ dev->sg_dst = sg_next(dev->sg_dst);
err = rk_set_data_start(dev);
} else {
rk_iv_copyback(dev);
@@ -462,13 +446,9 @@ static int rk_ablk_init_tfm(struct crypto_skcipher *tfm)
algt = container_of(alg, struct rk_crypto_tmp, alg.skcipher);
ctx->dev = algt->dev;
- ctx->dev->align_size = crypto_tfm_alg_alignmask(crypto_skcipher_tfm(tfm)) + 1;
ctx->dev->start = rk_ablk_start;
ctx->dev->update = rk_ablk_rx;
ctx->dev->complete = rk_crypto_complete;
- ctx->dev->addr_vir = (char *)__get_free_page(GFP_KERNEL);
- if (!ctx->dev->addr_vir)
- return -ENOMEM;
ctx->fallback_tfm = crypto_alloc_skcipher(name, 0, CRYPTO_ALG_NEED_FALLBACK);
if (IS_ERR(ctx->fallback_tfm)) {
@@ -488,7 +468,6 @@ static void rk_ablk_exit_tfm(struct crypto_skcipher *tfm)
struct rk_cipher_ctx *ctx = crypto_skcipher_ctx(tfm);
memzero_explicit(ctx->key, ctx->keylen);
- free_page((unsigned long)ctx->dev->addr_vir);
crypto_free_skcipher(ctx->fallback_tfm);
}
--
2.35.1
Use read_poll_timeout instead of open coding it.
In the same time, fix indentation of related comment.
Signed-off-by: Corentin Labbe <[email protected]>
---
drivers/crypto/rockchip/rk3288_crypto_ahash.c | 24 +++++++++----------
1 file changed, 12 insertions(+), 12 deletions(-)
diff --git a/drivers/crypto/rockchip/rk3288_crypto_ahash.c b/drivers/crypto/rockchip/rk3288_crypto_ahash.c
index 137013bd4410..1fbab86c9238 100644
--- a/drivers/crypto/rockchip/rk3288_crypto_ahash.c
+++ b/drivers/crypto/rockchip/rk3288_crypto_ahash.c
@@ -10,6 +10,7 @@
*/
#include <linux/device.h>
#include <asm/unaligned.h>
+#include <linux/iopoll.h>
#include "rk3288_crypto.h"
/*
@@ -295,18 +296,17 @@ static int rk_hash_run(struct crypto_engine *engine, void *breq)
sg = sg_next(sg);
}
- /*
- * it will take some time to process date after last dma
- * transmission.
- *
- * waiting time is relative with the last date len,
- * so cannot set a fixed time here.
- * 10us makes system not call here frequently wasting
- * efficiency, and make it response quickly when dma
- * complete.
- */
- while (!CRYPTO_READ(tctx->dev, RK_CRYPTO_HASH_STS))
- udelay(10);
+ /*
+ * it will take some time to process date after last dma
+ * transmission.
+ *
+ * waiting time is relative with the last date len,
+ * so cannot set a fixed time here.
+ * 10us makes system not call here frequently wasting
+ * efficiency, and make it response quickly when dma
+ * complete.
+ */
+ readl_poll_timeout(tctx->dev->reg + RK_CRYPTO_HASH_STS, v, v == 0, 10, 1000);
for (i = 0; i < crypto_ahash_digestsize(tfm) / 4; i++) {
v = readl(tctx->dev->reg + RK_CRYPTO_HASH_DOUT_0 + i * 4);
--
2.35.1
Add the number of clocks needed for each compatible.
Rockchip's datasheet give maximum frequencies for some clocks, so add
checks for verifying they are within limits. Let's start with rk3288 for
clock frequency check, other will came later.
Signed-off-by: Corentin Labbe <[email protected]>
---
drivers/crypto/rockchip/rk3288_crypto.c | 75 +++++++++++++++++++++----
drivers/crypto/rockchip/rk3288_crypto.h | 16 +++++-
2 files changed, 79 insertions(+), 12 deletions(-)
diff --git a/drivers/crypto/rockchip/rk3288_crypto.c b/drivers/crypto/rockchip/rk3288_crypto.c
index 6147ce44f757..d6d78b8af57c 100644
--- a/drivers/crypto/rockchip/rk3288_crypto.c
+++ b/drivers/crypto/rockchip/rk3288_crypto.c
@@ -14,10 +14,58 @@
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/of.h>
+#include <linux/of_device.h>
#include <linux/clk.h>
#include <linux/crypto.h>
#include <linux/reset.h>
+static const struct rk_variant rk3288_variant = {
+ .num_clks = 4,
+ .rkclks = {
+ { "sclk", 150000000},
+ }
+};
+
+static const struct rk_variant rk3328_variant = {
+ .num_clks = 3,
+};
+
+static int rk_crypto_get_clks(struct rk_crypto_info *dev)
+{
+ int i, j, err;
+ unsigned long cr;
+
+ dev->num_clks = devm_clk_bulk_get_all(dev->dev, &dev->clks);
+ if (dev->num_clks < dev->variant->num_clks) {
+ dev_err(dev->dev, "Missing clocks, got %d instead of %d\n",
+ dev->num_clks, dev->variant->num_clks);
+ return -EINVAL;
+ }
+
+ for (i = 0; i < dev->num_clks; i++) {
+ cr = clk_get_rate(dev->clks[i].clk);
+ for (j = 0; j < ARRAY_SIZE(dev->variant->rkclks); j++) {
+ if (dev->variant->rkclks[j].max == 0)
+ continue;
+ if (strcmp(dev->variant->rkclks[j].name, dev->clks[i].id))
+ continue;
+ if (cr > dev->variant->rkclks[j].max) {
+ err = clk_set_rate(dev->clks[i].clk,
+ dev->variant->rkclks[j].max);
+ if (err)
+ dev_err(dev->dev, "Fail downclocking %s from %lu to %lu\n",
+ dev->variant->rkclks[j].name, cr,
+ dev->variant->rkclks[j].max);
+ else
+ dev_info(dev->dev, "Downclocking %s from %lu to %lu\n",
+ dev->variant->rkclks[j].name, cr,
+ dev->variant->rkclks[j].max);
+ }
+ }
+ }
+ return 0;
+}
+
static int rk_crypto_enable_clk(struct rk_crypto_info *dev)
{
int err;
@@ -196,8 +244,12 @@ static void rk_crypto_unregister(void)
}
static const struct of_device_id crypto_of_id_table[] = {
- { .compatible = "rockchip,rk3288-crypto" },
- { .compatible = "rockchip,rk3328-crypto" },
+ { .compatible = "rockchip,rk3288-crypto",
+ .data = &rk3288_variant,
+ },
+ { .compatible = "rockchip,rk3328-crypto",
+ .data = &rk3328_variant,
+ },
{}
};
MODULE_DEVICE_TABLE(of, crypto_of_id_table);
@@ -215,6 +267,15 @@ static int rk_crypto_probe(struct platform_device *pdev)
goto err_crypto;
}
+ crypto_info->dev = &pdev->dev;
+ platform_set_drvdata(pdev, crypto_info);
+
+ crypto_info->variant = of_device_get_match_data(&pdev->dev);
+ if (!crypto_info->variant) {
+ dev_err(&pdev->dev, "Missing variant\n");
+ return -EINVAL;
+ }
+
crypto_info->rst = devm_reset_control_get(dev, "crypto-rst");
if (IS_ERR(crypto_info->rst)) {
err = PTR_ERR(crypto_info->rst);
@@ -227,12 +288,9 @@ static int rk_crypto_probe(struct platform_device *pdev)
goto err_crypto;
}
- crypto_info->num_clks = devm_clk_bulk_get_all(&pdev->dev,
- &crypto_info->clks);
- if (crypto_info->num_clks < 3) {
- err = -EINVAL;
+ err = rk_crypto_get_clks(crypto_info);
+ if (err)
goto err_crypto;
- }
crypto_info->irq = platform_get_irq(pdev, 0);
if (crypto_info->irq < 0) {
@@ -250,9 +308,6 @@ static int rk_crypto_probe(struct platform_device *pdev)
goto err_crypto;
}
- crypto_info->dev = &pdev->dev;
- platform_set_drvdata(pdev, crypto_info);
-
crypto_info->engine = crypto_engine_alloc_init(&pdev->dev, true);
crypto_engine_start(crypto_info->engine);
init_completion(&crypto_info->complete);
diff --git a/drivers/crypto/rockchip/rk3288_crypto.h b/drivers/crypto/rockchip/rk3288_crypto.h
index ff9fc25972eb..ac979d67ced9 100644
--- a/drivers/crypto/rockchip/rk3288_crypto.h
+++ b/drivers/crypto/rockchip/rk3288_crypto.h
@@ -188,14 +188,26 @@
#define CRYPTO_WRITE(dev, offset, val) \
writel_relaxed((val), ((dev)->reg + (offset)))
+#define RK_MAX_CLKS 4
+
+struct rk_clks {
+ const char *name;
+ unsigned long max;
+};
+
+struct rk_variant {
+ int num_clks;
+ struct rk_clks rkclks[RK_MAX_CLKS];
+};
+
struct rk_crypto_info {
struct device *dev;
struct clk_bulk_data *clks;
- int num_clks;
+ int num_clks;
struct reset_control *rst;
void __iomem *reg;
int irq;
-
+ const struct rk_variant *variant;
struct crypto_engine *engine;
struct completion complete;
int status;
--
2.35.1
The RK3399 has 2 rk3288 compatible crypto device named crypto0 and
crypto1. The only difference is lack of RSA in crypto1.
We need to add driver support for 2 parallel instance as only one need
to register crypto algorithms.
Then the driver will round robin each request on each device.
For avoiding complexity (device bringup after a TFM is created), PM is
modified to be handled per request.
Signed-off-by: Corentin Labbe <[email protected]>
---
drivers/crypto/rockchip/rk3288_crypto.c | 92 +++++++++++++++----
drivers/crypto/rockchip/rk3288_crypto.h | 25 +++--
drivers/crypto/rockchip/rk3288_crypto_ahash.c | 37 ++++----
.../crypto/rockchip/rk3288_crypto_skcipher.c | 37 ++++----
4 files changed, 123 insertions(+), 68 deletions(-)
diff --git a/drivers/crypto/rockchip/rk3288_crypto.c b/drivers/crypto/rockchip/rk3288_crypto.c
index 9ba9b9e433c4..baf015f6707b 100644
--- a/drivers/crypto/rockchip/rk3288_crypto.c
+++ b/drivers/crypto/rockchip/rk3288_crypto.c
@@ -19,6 +19,23 @@
#include <linux/crypto.h>
#include <linux/reset.h>
+static struct rockchip_ip rocklist = {
+ .dev_list = LIST_HEAD_INIT(rocklist.dev_list),
+ .lock = __SPIN_LOCK_UNLOCKED(rocklist.lock),
+};
+
+struct rk_crypto_info *get_rk_crypto(void)
+{
+ struct rk_crypto_info *first;
+
+ spin_lock(&rocklist.lock);
+ first = list_first_entry_or_null(&rocklist.dev_list,
+ struct rk_crypto_info, list);
+ list_rotate_left(&rocklist.dev_list);
+ spin_unlock(&rocklist.lock);
+ return first;
+}
+
static const struct rk_variant rk3288_variant = {
.num_clks = 4,
.rkclks = {
@@ -30,6 +47,10 @@ static const struct rk_variant rk3328_variant = {
.num_clks = 3,
};
+static const struct rk_variant rk3399_variant = {
+ .num_clks = 3,
+};
+
static int rk_crypto_get_clks(struct rk_crypto_info *dev)
{
int i, j, err;
@@ -83,8 +104,8 @@ static void rk_crypto_disable_clk(struct rk_crypto_info *dev)
}
/*
- * Power management strategy: The device is suspended unless a TFM exists for
- * one of the algorithms proposed by this driver.
+ * Power management strategy: The device is suspended until a request
+ * is handled. For avoiding suspend/resume yoyo, the autosuspend is set to 2s.
*/
static int rk_crypto_pm_suspend(struct device *dev)
{
@@ -161,8 +182,17 @@ static struct rk_crypto_tmp *rk_cipher_algs[] = {
#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP_DEBUG
static int rk_crypto_debugfs_show(struct seq_file *seq, void *v)
{
+ struct rk_crypto_info *dd;
unsigned int i;
+ spin_lock(&rocklist.lock);
+ list_for_each_entry(dd, &rocklist.dev_list, list) {
+ seq_printf(seq, "%s %s requests: %lu\n",
+ dev_driver_string(dd->dev), dev_name(dd->dev),
+ dd->nreq);
+ }
+ spin_unlock(&rocklist.lock);
+
for (i = 0; i < ARRAY_SIZE(rk_cipher_algs); i++) {
if (!rk_cipher_algs[i]->dev)
continue;
@@ -193,6 +223,18 @@ static int rk_crypto_debugfs_show(struct seq_file *seq, void *v)
DEFINE_SHOW_ATTRIBUTE(rk_crypto_debugfs);
#endif
+static void register_debugfs(struct rk_crypto_info *crypto_info)
+{
+#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP_DEBUG
+ /* Ignore error of debugfs */
+ rocklist.dbgfs_dir = debugfs_create_dir("rk3288_crypto", NULL);
+ rocklist.dbgfs_stats = debugfs_create_file("stats", 0444,
+ rocklist.dbgfs_dir,
+ &rocklist,
+ &rk_crypto_debugfs_fops);
+#endif
+}
+
static int rk_crypto_register(struct rk_crypto_info *crypto_info)
{
unsigned int i, k;
@@ -250,6 +292,9 @@ static const struct of_device_id crypto_of_id_table[] = {
{ .compatible = "rockchip,rk3328-crypto",
.data = &rk3328_variant,
},
+ { .compatible = "rockchip,rk3399-crypto",
+ .data = &rk3399_variant,
+ },
{}
};
MODULE_DEVICE_TABLE(of, crypto_of_id_table);
@@ -257,7 +302,7 @@ MODULE_DEVICE_TABLE(of, crypto_of_id_table);
static int rk_crypto_probe(struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
- struct rk_crypto_info *crypto_info;
+ struct rk_crypto_info *crypto_info, *first;
int err = 0;
crypto_info = devm_kzalloc(&pdev->dev,
@@ -316,22 +361,22 @@ static int rk_crypto_probe(struct platform_device *pdev)
if (err)
goto err_pm;
- err = rk_crypto_register(crypto_info);
- if (err) {
- dev_err(dev, "err in register alg");
- goto err_register_alg;
- }
+ spin_lock(&rocklist.lock);
+ first = list_first_entry_or_null(&rocklist.dev_list,
+ struct rk_crypto_info, list);
+ list_add_tail(&crypto_info->list, &rocklist.dev_list);
+ spin_unlock(&rocklist.lock);
+
+ if (!first) {
+ err = rk_crypto_register(crypto_info);
+ if (err) {
+ dev_err(dev, "Fail to register crypto algorithms");
+ goto err_register_alg;
+ }
-#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP_DEBUG
- /* Ignore error of debugfs */
- crypto_info->dbgfs_dir = debugfs_create_dir("rk3288_crypto", NULL);
- crypto_info->dbgfs_stats = debugfs_create_file("stats", 0444,
- crypto_info->dbgfs_dir,
- crypto_info,
- &rk_crypto_debugfs_fops);
-#endif
+ register_debugfs(crypto_info);
+ }
- dev_info(dev, "Crypto Accelerator successfully registered\n");
return 0;
err_register_alg:
@@ -346,11 +391,20 @@ static int rk_crypto_probe(struct platform_device *pdev)
static int rk_crypto_remove(struct platform_device *pdev)
{
struct rk_crypto_info *crypto_tmp = platform_get_drvdata(pdev);
+ struct rk_crypto_info *first;
+
+ spin_lock_bh(&rocklist.lock);
+ list_del(&crypto_tmp->list);
+ first = list_first_entry_or_null(&rocklist.dev_list,
+ struct rk_crypto_info, list);
+ spin_unlock_bh(&rocklist.lock);
+ if (!first) {
#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP_DEBUG
- debugfs_remove_recursive(crypto_tmp->dbgfs_dir);
+ debugfs_remove_recursive(rocklist.dbgfs_dir);
#endif
- rk_crypto_unregister();
+ rk_crypto_unregister();
+ }
rk_crypto_pm_exit(crypto_tmp);
crypto_engine_exit(crypto_tmp->engine);
return 0;
diff --git a/drivers/crypto/rockchip/rk3288_crypto.h b/drivers/crypto/rockchip/rk3288_crypto.h
index ac979d67ced9..b2695258cade 100644
--- a/drivers/crypto/rockchip/rk3288_crypto.h
+++ b/drivers/crypto/rockchip/rk3288_crypto.h
@@ -190,6 +190,20 @@
#define RK_MAX_CLKS 4
+/*
+ * struct rockchip_ip - struct for managing a list of RK crypto instance
+ * @dev_list: Used for doing a list of rk_crypto_info
+ * @lock: Control access to dev_list
+ * @dbgfs_dir: Debugfs dentry for statistic directory
+ * @dbgfs_stats: Debugfs dentry for statistic counters
+ */
+struct rockchip_ip {
+ struct list_head dev_list;
+ spinlock_t lock; /* Control access to dev_list */
+ struct dentry *dbgfs_dir;
+ struct dentry *dbgfs_stats;
+};
+
struct rk_clks {
const char *name;
unsigned long max;
@@ -201,6 +215,7 @@ struct rk_variant {
};
struct rk_crypto_info {
+ struct list_head list;
struct device *dev;
struct clk_bulk_data *clks;
int num_clks;
@@ -208,19 +223,15 @@ struct rk_crypto_info {
void __iomem *reg;
int irq;
const struct rk_variant *variant;
+ unsigned long nreq;
struct crypto_engine *engine;
struct completion complete;
int status;
-#ifdef CONFIG_CRYPTO_DEV_ROCKCHIP_DEBUG
- struct dentry *dbgfs_dir;
- struct dentry *dbgfs_stats;
-#endif
};
/* the private variable of hash */
struct rk_ahash_ctx {
struct crypto_engine_ctx enginectx;
- struct rk_crypto_info *dev;
/* for fallback */
struct crypto_ahash *fallback_tfm;
};
@@ -236,7 +247,6 @@ struct rk_ahash_rctx {
/* the private variable of cipher */
struct rk_cipher_ctx {
struct crypto_engine_ctx enginectx;
- struct rk_crypto_info *dev;
unsigned int keylen;
u8 key[AES_MAX_KEY_SIZE];
u8 iv[AES_BLOCK_SIZE];
@@ -252,7 +262,7 @@ struct rk_cipher_rctx {
struct rk_crypto_tmp {
u32 type;
- struct rk_crypto_info *dev;
+ struct rk_crypto_info *dev;
union {
struct skcipher_alg skcipher;
struct ahash_alg hash;
@@ -276,4 +286,5 @@ extern struct rk_crypto_tmp rk_ahash_sha1;
extern struct rk_crypto_tmp rk_ahash_sha256;
extern struct rk_crypto_tmp rk_ahash_md5;
+struct rk_crypto_info *get_rk_crypto(void);
#endif
diff --git a/drivers/crypto/rockchip/rk3288_crypto_ahash.c b/drivers/crypto/rockchip/rk3288_crypto_ahash.c
index 30f78256c955..a78ff3dcd0b1 100644
--- a/drivers/crypto/rockchip/rk3288_crypto_ahash.c
+++ b/drivers/crypto/rockchip/rk3288_crypto_ahash.c
@@ -199,8 +199,8 @@ static int rk_ahash_export(struct ahash_request *req, void *out)
static int rk_ahash_digest(struct ahash_request *req)
{
struct rk_ahash_rctx *rctx = ahash_request_ctx(req);
- struct rk_ahash_ctx *tctx = crypto_tfm_ctx(req->base.tfm);
- struct rk_crypto_info *dev = tctx->dev;
+ struct rk_crypto_info *dev;
+ struct crypto_engine *engine;
if (rk_ahash_need_fallback(req))
return rk_ahash_digest_fb(req);
@@ -208,9 +208,12 @@ static int rk_ahash_digest(struct ahash_request *req)
if (!req->nbytes)
return zero_message_process(req);
+ dev = get_rk_crypto();
+
rctx->dev = dev;
+ engine = dev->engine;
- return crypto_transfer_hash_request_to_engine(dev->engine, req);
+ return crypto_transfer_hash_request_to_engine(engine, req);
}
static void crypto_ahash_dma_start(struct rk_crypto_info *dev, struct scatterlist *sg)
@@ -260,9 +263,14 @@ static int rk_hash_run(struct crypto_engine *engine, void *breq)
int i;
u32 v;
+ err = pm_runtime_resume_and_get(rkc->dev);
+ if (err)
+ return err;
+
rctx->mode = 0;
algt->stat_req++;
+ rkc->nreq++;
switch (crypto_ahash_digestsize(tfm)) {
case SHA1_DIGEST_SIZE:
@@ -313,6 +321,8 @@ static int rk_hash_run(struct crypto_engine *engine, void *breq)
}
theend:
+ pm_runtime_put_autosuspend(rkc->dev);
+
local_bh_disable();
crypto_finalize_hash_request(engine, breq, err);
local_bh_enable();
@@ -323,21 +333,15 @@ static int rk_hash_run(struct crypto_engine *engine, void *breq)
static int rk_cra_hash_init(struct crypto_tfm *tfm)
{
struct rk_ahash_ctx *tctx = crypto_tfm_ctx(tfm);
- struct rk_crypto_tmp *algt;
- struct ahash_alg *alg = __crypto_ahash_alg(tfm->__crt_alg);
-
const char *alg_name = crypto_tfm_alg_name(tfm);
- int err;
-
- algt = container_of(alg, struct rk_crypto_tmp, alg.hash);
-
- tctx->dev = algt->dev;
+ struct ahash_alg *alg = __crypto_ahash_alg(tfm->__crt_alg);
+ struct rk_crypto_tmp *algt = container_of(alg, struct rk_crypto_tmp, alg.hash);
/* for fallback */
tctx->fallback_tfm = crypto_alloc_ahash(alg_name, 0,
CRYPTO_ALG_NEED_FALLBACK);
if (IS_ERR(tctx->fallback_tfm)) {
- dev_err(tctx->dev->dev, "Could not load fallback driver.\n");
+ dev_err(algt->dev->dev, "Could not load fallback driver.\n");
return PTR_ERR(tctx->fallback_tfm);
}
@@ -349,15 +353,7 @@ static int rk_cra_hash_init(struct crypto_tfm *tfm)
tctx->enginectx.op.prepare_request = rk_hash_prepare;
tctx->enginectx.op.unprepare_request = rk_hash_unprepare;
- err = pm_runtime_resume_and_get(tctx->dev->dev);
- if (err < 0)
- goto error_pm;
-
return 0;
-error_pm:
- crypto_free_ahash(tctx->fallback_tfm);
-
- return err;
}
static void rk_cra_hash_exit(struct crypto_tfm *tfm)
@@ -365,7 +361,6 @@ static void rk_cra_hash_exit(struct crypto_tfm *tfm)
struct rk_ahash_ctx *tctx = crypto_tfm_ctx(tfm);
crypto_free_ahash(tctx->fallback_tfm);
- pm_runtime_put_autosuspend(tctx->dev->dev);
}
struct rk_crypto_tmp rk_ahash_sha1 = {
diff --git a/drivers/crypto/rockchip/rk3288_crypto_skcipher.c b/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
index 0b1c90ababb7..59069457582b 100644
--- a/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
+++ b/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
@@ -17,11 +17,11 @@
static int rk_cipher_need_fallback(struct skcipher_request *req)
{
struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
- unsigned int bs = crypto_skcipher_blocksize(tfm);
struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
struct rk_crypto_tmp *algt = container_of(alg, struct rk_crypto_tmp, alg.skcipher);
struct scatterlist *sgs, *sgd;
unsigned int stodo, dtodo, len;
+ unsigned int bs = crypto_skcipher_blocksize(tfm);
if (!req->cryptlen)
return true;
@@ -84,15 +84,16 @@ static int rk_cipher_fallback(struct skcipher_request *areq)
static int rk_cipher_handle_req(struct skcipher_request *req)
{
- struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
- struct rk_cipher_ctx *tctx = crypto_skcipher_ctx(tfm);
struct rk_cipher_rctx *rctx = skcipher_request_ctx(req);
- struct rk_crypto_info *rkc = tctx->dev;
- struct crypto_engine *engine = rkc->engine;
+ struct rk_crypto_info *rkc;
+ struct crypto_engine *engine;
if (rk_cipher_need_fallback(req))
return rk_cipher_fallback(req);
+ rkc = get_rk_crypto();
+
+ engine = rkc->engine;
rctx->dev = rkc;
return crypto_transfer_skcipher_request_to_engine(engine, req);
@@ -307,7 +308,12 @@ static int rk_cipher_run(struct crypto_engine *engine, void *async_req)
struct rk_crypto_tmp *algt = container_of(alg, struct rk_crypto_tmp, alg.skcipher);
struct rk_crypto_info *rkc = rctx->dev;
+ err = pm_runtime_resume_and_get(rkc->dev);
+ if (err)
+ return err;
+
algt->stat_req++;
+ rkc->nreq++;
ivsize = crypto_skcipher_ivsize(tfm);
if (areq->iv && crypto_skcipher_ivsize(tfm) > 0) {
@@ -401,6 +407,8 @@ static int rk_cipher_run(struct crypto_engine *engine, void *async_req)
}
theend:
+ pm_runtime_put_autosuspend(rkc->dev);
+
local_bh_disable();
crypto_finalize_skcipher_request(engine, areq, err);
local_bh_enable();
@@ -420,18 +428,13 @@ static int rk_cipher_run(struct crypto_engine *engine, void *async_req)
static int rk_cipher_tfm_init(struct crypto_skcipher *tfm)
{
struct rk_cipher_ctx *ctx = crypto_skcipher_ctx(tfm);
- struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
const char *name = crypto_tfm_alg_name(&tfm->base);
- struct rk_crypto_tmp *algt;
- int err;
-
- algt = container_of(alg, struct rk_crypto_tmp, alg.skcipher);
-
- ctx->dev = algt->dev;
+ struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
+ struct rk_crypto_tmp *algt = container_of(alg, struct rk_crypto_tmp, alg.skcipher);
ctx->fallback_tfm = crypto_alloc_skcipher(name, 0, CRYPTO_ALG_NEED_FALLBACK);
if (IS_ERR(ctx->fallback_tfm)) {
- dev_err(ctx->dev->dev, "ERROR: Cannot allocate fallback for %s %ld\n",
+ dev_err(algt->dev->dev, "ERROR: Cannot allocate fallback for %s %ld\n",
name, PTR_ERR(ctx->fallback_tfm));
return PTR_ERR(ctx->fallback_tfm);
}
@@ -441,14 +444,7 @@ static int rk_cipher_tfm_init(struct crypto_skcipher *tfm)
ctx->enginectx.op.do_one_request = rk_cipher_run;
- err = pm_runtime_resume_and_get(ctx->dev->dev);
- if (err < 0)
- goto error_pm;
-
return 0;
-error_pm:
- crypto_free_skcipher(ctx->fallback_tfm);
- return err;
}
static void rk_cipher_tfm_exit(struct crypto_skcipher *tfm)
@@ -457,7 +453,6 @@ static void rk_cipher_tfm_exit(struct crypto_skcipher *tfm)
memzero_explicit(ctx->key, ctx->keylen);
crypto_free_skcipher(ctx->fallback_tfm);
- pm_runtime_put_autosuspend(ctx->dev->dev);
}
struct rk_crypto_tmp rk_ecb_aes_alg = {
--
2.35.1
The hardware does not handle 0 size length request, let's add a
fallback.
Furthermore fallback will be used for all unaligned case the hardware
cannot handle.
Fixes: ce0183cb6464b ("crypto: rockchip - switch to skcipher API")
Signed-off-by: Corentin Labbe <[email protected]>
---
drivers/crypto/Kconfig | 4 +
drivers/crypto/rockchip/rk3288_crypto.h | 2 +
.../crypto/rockchip/rk3288_crypto_skcipher.c | 97 ++++++++++++++++---
3 files changed, 90 insertions(+), 13 deletions(-)
diff --git a/drivers/crypto/Kconfig b/drivers/crypto/Kconfig
index 7b2d138bc83e..84ab14afcbd9 100644
--- a/drivers/crypto/Kconfig
+++ b/drivers/crypto/Kconfig
@@ -784,6 +784,10 @@ config CRYPTO_DEV_IMGTEC_HASH
config CRYPTO_DEV_ROCKCHIP
tristate "Rockchip's Cryptographic Engine driver"
depends on OF && ARCH_ROCKCHIP
+ depends on PM
+ select CRYPTO_ECB
+ select CRYPTO_CBC
+ select CRYPTO_DES
select CRYPTO_AES
select CRYPTO_LIB_DES
select CRYPTO_MD5
diff --git a/drivers/crypto/rockchip/rk3288_crypto.h b/drivers/crypto/rockchip/rk3288_crypto.h
index c919d9a43a08..8b1e15d8ddc6 100644
--- a/drivers/crypto/rockchip/rk3288_crypto.h
+++ b/drivers/crypto/rockchip/rk3288_crypto.h
@@ -246,10 +246,12 @@ struct rk_cipher_ctx {
struct rk_crypto_info *dev;
unsigned int keylen;
u8 iv[AES_BLOCK_SIZE];
+ struct crypto_skcipher *fallback_tfm;
};
struct rk_cipher_rctx {
u32 mode;
+ struct skcipher_request fallback_req; // keep at the end
};
enum alg_type {
diff --git a/drivers/crypto/rockchip/rk3288_crypto_skcipher.c b/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
index bbd0bf52bf07..eac5bba66e25 100644
--- a/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
+++ b/drivers/crypto/rockchip/rk3288_crypto_skcipher.c
@@ -13,6 +13,63 @@
#define RK_CRYPTO_DEC BIT(0)
+static int rk_cipher_need_fallback(struct skcipher_request *req)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(req);
+ unsigned int bs = crypto_skcipher_blocksize(tfm);
+ struct scatterlist *sgs, *sgd;
+ unsigned int stodo, dtodo, len;
+
+ if (!req->cryptlen)
+ return true;
+
+ len = req->cryptlen;
+ sgs = req->src;
+ sgd = req->dst;
+ while (sgs && sgd) {
+ if (!IS_ALIGNED(sgs->offset, sizeof(u32))) {
+ return true;
+ }
+ if (!IS_ALIGNED(sgd->offset, sizeof(u32))) {
+ return true;
+ }
+ stodo = min(len, sgs->length);
+ if (stodo % bs) {
+ return true;
+ }
+ dtodo = min(len, sgd->length);
+ if (dtodo % bs) {
+ return true;
+ }
+ if (stodo != dtodo) {
+ return true;
+ }
+ len -= stodo;
+ sgs = sg_next(sgs);
+ sgd = sg_next(sgd);
+ }
+ return false;
+}
+
+static int rk_cipher_fallback(struct skcipher_request *areq)
+{
+ struct crypto_skcipher *tfm = crypto_skcipher_reqtfm(areq);
+ struct rk_cipher_ctx *op = crypto_skcipher_ctx(tfm);
+ struct rk_cipher_rctx *rctx = skcipher_request_ctx(areq);
+ int err;
+
+ skcipher_request_set_tfm(&rctx->fallback_req, op->fallback_tfm);
+ skcipher_request_set_callback(&rctx->fallback_req, areq->base.flags,
+ areq->base.complete, areq->base.data);
+ skcipher_request_set_crypt(&rctx->fallback_req, areq->src, areq->dst,
+ areq->cryptlen, areq->iv);
+ if (rctx->mode & RK_CRYPTO_DEC)
+ err = crypto_skcipher_decrypt(&rctx->fallback_req);
+ else
+ err = crypto_skcipher_encrypt(&rctx->fallback_req);
+ return err;
+}
+
static void rk_crypto_complete(struct crypto_async_request *base, int err)
{
if (base->complete)
@@ -22,10 +79,10 @@ static void rk_crypto_complete(struct crypto_async_request *base, int err)
static int rk_handle_req(struct rk_crypto_info *dev,
struct skcipher_request *req)
{
- if (!IS_ALIGNED(req->cryptlen, dev->align_size))
- return -EINVAL;
- else
- return dev->enqueue(dev, &req->base);
+ if (rk_cipher_need_fallback(req))
+ return rk_cipher_fallback(req);
+
+ return dev->enqueue(dev, &req->base);
}
static int rk_aes_setkey(struct crypto_skcipher *cipher,
@@ -39,7 +96,8 @@ static int rk_aes_setkey(struct crypto_skcipher *cipher,
return -EINVAL;
ctx->keylen = keylen;
memcpy_toio(ctx->dev->reg + RK_CRYPTO_AES_KEY_0, key, keylen);
- return 0;
+
+ return crypto_skcipher_setkey(ctx->fallback_tfm, key, keylen);
}
static int rk_des_setkey(struct crypto_skcipher *cipher,
@@ -54,7 +112,8 @@ static int rk_des_setkey(struct crypto_skcipher *cipher,
ctx->keylen = keylen;
memcpy_toio(ctx->dev->reg + RK_CRYPTO_TDES_KEY1_0, key, keylen);
- return 0;
+
+ return crypto_skcipher_setkey(ctx->fallback_tfm, key, keylen);
}
static int rk_tdes_setkey(struct crypto_skcipher *cipher,
@@ -69,7 +128,7 @@ static int rk_tdes_setkey(struct crypto_skcipher *cipher,
ctx->keylen = keylen;
memcpy_toio(ctx->dev->reg + RK_CRYPTO_TDES_KEY1_0, key, keylen);
- return 0;
+ return crypto_skcipher_setkey(ctx->fallback_tfm, key, keylen);
}
static int rk_aes_ecb_encrypt(struct skcipher_request *req)
@@ -394,6 +453,7 @@ static int rk_ablk_init_tfm(struct crypto_skcipher *tfm)
{
struct rk_cipher_ctx *ctx = crypto_skcipher_ctx(tfm);
struct skcipher_alg *alg = crypto_skcipher_alg(tfm);
+ const char *name = crypto_tfm_alg_name(&tfm->base);
struct rk_crypto_tmp *algt;
algt = container_of(alg, struct rk_crypto_tmp, alg.skcipher);
@@ -407,6 +467,16 @@ static int rk_ablk_init_tfm(struct crypto_skcipher *tfm)
if (!ctx->dev->addr_vir)
return -ENOMEM;
+ ctx->fallback_tfm = crypto_alloc_skcipher(name, 0, CRYPTO_ALG_NEED_FALLBACK);
+ if (IS_ERR(ctx->fallback_tfm)) {
+ dev_err(ctx->dev->dev, "ERROR: Cannot allocate fallback for %s %ld\n",
+ name, PTR_ERR(ctx->fallback_tfm));
+ return PTR_ERR(ctx->fallback_tfm);
+ }
+
+ tfm->reqsize = sizeof(struct rk_cipher_rctx) +
+ crypto_skcipher_reqsize(ctx->fallback_tfm);
+
return 0;
}
@@ -415,6 +485,7 @@ static void rk_ablk_exit_tfm(struct crypto_skcipher *tfm)
struct rk_cipher_ctx *ctx = crypto_skcipher_ctx(tfm);
free_page((unsigned long)ctx->dev->addr_vir);
+ crypto_free_skcipher(ctx->fallback_tfm);
}
struct rk_crypto_tmp rk_ecb_aes_alg = {
@@ -423,7 +494,7 @@ struct rk_crypto_tmp rk_ecb_aes_alg = {
.base.cra_name = "ecb(aes)",
.base.cra_driver_name = "ecb-aes-rk",
.base.cra_priority = 300,
- .base.cra_flags = CRYPTO_ALG_ASYNC,
+ .base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK,
.base.cra_blocksize = AES_BLOCK_SIZE,
.base.cra_ctxsize = sizeof(struct rk_cipher_ctx),
.base.cra_alignmask = 0x0f,
@@ -445,7 +516,7 @@ struct rk_crypto_tmp rk_cbc_aes_alg = {
.base.cra_name = "cbc(aes)",
.base.cra_driver_name = "cbc-aes-rk",
.base.cra_priority = 300,
- .base.cra_flags = CRYPTO_ALG_ASYNC,
+ .base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK,
.base.cra_blocksize = AES_BLOCK_SIZE,
.base.cra_ctxsize = sizeof(struct rk_cipher_ctx),
.base.cra_alignmask = 0x0f,
@@ -468,7 +539,7 @@ struct rk_crypto_tmp rk_ecb_des_alg = {
.base.cra_name = "ecb(des)",
.base.cra_driver_name = "ecb-des-rk",
.base.cra_priority = 300,
- .base.cra_flags = CRYPTO_ALG_ASYNC,
+ .base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK,
.base.cra_blocksize = DES_BLOCK_SIZE,
.base.cra_ctxsize = sizeof(struct rk_cipher_ctx),
.base.cra_alignmask = 0x07,
@@ -490,7 +561,7 @@ struct rk_crypto_tmp rk_cbc_des_alg = {
.base.cra_name = "cbc(des)",
.base.cra_driver_name = "cbc-des-rk",
.base.cra_priority = 300,
- .base.cra_flags = CRYPTO_ALG_ASYNC,
+ .base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK,
.base.cra_blocksize = DES_BLOCK_SIZE,
.base.cra_ctxsize = sizeof(struct rk_cipher_ctx),
.base.cra_alignmask = 0x07,
@@ -513,7 +584,7 @@ struct rk_crypto_tmp rk_ecb_des3_ede_alg = {
.base.cra_name = "ecb(des3_ede)",
.base.cra_driver_name = "ecb-des3-ede-rk",
.base.cra_priority = 300,
- .base.cra_flags = CRYPTO_ALG_ASYNC,
+ .base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK,
.base.cra_blocksize = DES_BLOCK_SIZE,
.base.cra_ctxsize = sizeof(struct rk_cipher_ctx),
.base.cra_alignmask = 0x07,
@@ -535,7 +606,7 @@ struct rk_crypto_tmp rk_cbc_des3_ede_alg = {
.base.cra_name = "cbc(des3_ede)",
.base.cra_driver_name = "cbc-des3-ede-rk",
.base.cra_priority = 300,
- .base.cra_flags = CRYPTO_ALG_ASYNC,
+ .base.cra_flags = CRYPTO_ALG_ASYNC | CRYPTO_ALG_NEED_FALLBACK,
.base.cra_blocksize = DES_BLOCK_SIZE,
.base.cra_ctxsize = sizeof(struct rk_cipher_ctx),
.base.cra_alignmask = 0x07,
--
2.35.1
Le Sun, May 08, 2022 at 06:59:24PM +0000, Corentin Labbe a ?crit :
> Hello
>
> The rockchip crypto driver is broken and do not pass self-tests.
> This serie's goal is to permit to become usable and pass self-tests.
>
> This whole serie is tested on a rk3328-rock64, rk3288-miqi and
> rk3399-khadas-edge-v with selftests (with CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y)
>
> Regards
>
This is a gentle ping since this serie has now, no comment to address.
Regards
On Mon, Jun 13, 2022 at 01:23:04PM +0200, LABBE Corentin wrote:
> Le Sun, May 08, 2022 at 06:59:24PM +0000, Corentin Labbe a ?crit :
> > The rockchip crypto driver is broken and do not pass self-tests.
> > This serie's goal is to permit to become usable and pass self-tests.
> >
> > This whole serie is tested on a rk3328-rock64, rk3288-miqi and
> > rk3399-khadas-edge-v with selftests (with CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y)
> >
> > Regards
> >
>
> This is a gentle ping since this serie has now, no comment to address.
I've just replied to patch 14 with a comment, but other than that I
agree that patches 1-24 look good. It would be good to get these merged
soon as crypto acceleration on Rockchip hardware has been totally broken
for several releases now.
Patches 1-13 and 15-24 are:
Reviewed-by: John Keeping <[email protected]>
I'm less sure about patches 25-33 as I don't really know much about the
crypto API, but it seems strange that dispatching requests to two
(nearly) identical accelerators should need so much special handling in
the driver and that there isn't some higher level management for this.
On Sunday, 8 May 2022 20:59:24 CEST Corentin Labbe wrote:
> The rockchip crypto driver is broken and do not pass self-tests.
> This serie's goal is to permit to become usable and pass self-tests.
>
> This whole serie is tested on a rk3328-rock64, rk3288-miqi and
> rk3399-khadas-edge-v with selftests (with
> CONFIG_CRYPTO_MANAGER_EXTRA_TESTS=y)
https://salsa.debian.org/kernel-team/linux/-/merge_requests/493 is where I
integrated this patch set with the Debian kernel and tested that successfully
on my Pine64 Rock64.
IOW: Tested-by Diederik de Haas <[email protected]>
AFAIK, this patch set has also been tested on rk3399-rockpro64 by sigmaris.