Received: by 2002:ab2:3141:0:b0:1ed:23cc:44d1 with SMTP id i1csp290637lqg; Fri, 1 Mar 2024 05:33:21 -0800 (PST) X-Forwarded-Encrypted: i=3; AJvYcCUiLD1hku8nTr7KiiYJxST28vAeBUUjq0Um7Voa/g/H75btU3xlUoglGBb80FCOkmw+XAm+aNGeUZVJFjb/7b+8DYZFsLZXTE3yzmqwDA== X-Google-Smtp-Source: AGHT+IE9hMuHG/hX7GIdhaNy1+YenAsRYMHyQZnZZoo1/cm9lzilWsoYE659EgC5UulTcT0Vf2oF X-Received: by 2002:a17:903:1c2:b0:1db:fd4e:329f with SMTP id e2-20020a17090301c200b001dbfd4e329fmr1658157plh.15.1709300001046; Fri, 01 Mar 2024 05:33:21 -0800 (PST) ARC-Seal: i=2; a=rsa-sha256; t=1709300001; cv=pass; d=google.com; s=arc-20160816; b=xLAEA0rOmhSOeAFuSyP5rekw41ZTXMtz++zC4wqyoH6FG3TJ1OmiGxiBlVr2slM0ed /kHNxXOLPa+XnBu3v+XkNu2AuwxFMn6w/1kk5WkviGe+nShH1w3CQC+4G24x+QZ7m6Vc ccuEk2m+kiR0DCWjQjKVydV9ku9KoOLmOZ3Y1jzktUJtsI0DHnXILKx6QpYNLtKv5fTd XYZTnPBMiE7Hq2qsqheHj7L7v+/Z9RuOLNtuOOlw1uYWGUlk1z86hM+X4ZOIXGGCIMX4 cF5SCuBM5oBbR+HTbRaIfkXQWiK5wAs5JM9HUEBhvBXDszeMdny5A4IKrwrW9/PEXh9X /HIQ== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:list-unsubscribe :list-subscribe:list-id:precedence:references:in-reply-to:message-id :date:subject:cc:to:from:dkim-signature:dkim-filter; bh=/hD4RKgA8kdmVJ0Yd81c9subYETcwlyUA6WD27vbUp4=; fh=7+QqR+U+m0B5ryyywou1kgMZVOfT/GgVHz0ZH2Lvlxg=; b=s74TG/1vpNIj1Vr65V2VYFF0RKspnY+H3b5dBFihbF2xSAGeNtoQRoKIR349nXJEwP Z1b5RRzNAWqRF2W4KBR1D95h5K3dXlrK0ynEMyTF+zZm7CXpSkzD3FQ1TIRG6EeFMqcw XU/h1+CwebS4bcWbSQhxCxXPSPOMrW0jrsojNb6xQghHDygBLYJYbnSrD81cW+5pacKr frvFVm4jTz0LNO8YNuZQ6V+rMtCNnIkQTf1idZRTwJSddUp1dvEGfPVzaW7FBSI6yl6b 56jOK4+aOFuLYt7GRPulcwBHwy1ryP9pgMlvEsVcSb2/k6VvR9IyqQ+Hof3zBnomIkA3 MSCg==; dara=google.com ARC-Authentication-Results: i=2; mx.google.com; dkim=pass header.i=@salutedevices.com header.s=mail header.b=Li9suypQ; arc=pass (i=1 spf=pass spfdomain=salutedevices.com dkim=pass dkdomain=salutedevices.com dmarc=pass fromdomain=salutedevices.com); spf=pass (google.com: domain of linux-crypto+bounces-2437-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-crypto+bounces-2437-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=salutedevices.com Return-Path: Received: from sv.mirrors.kernel.org (sv.mirrors.kernel.org. [2604:1380:45e3:2400::1]) by mx.google.com with ESMTPS id h10-20020a170902f70a00b001dca850c99esi3717058plo.78.2024.03.01.05.33.20 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 01 Mar 2024 05:33:21 -0800 (PST) Received-SPF: pass (google.com: domain of linux-crypto+bounces-2437-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1; Authentication-Results: mx.google.com; dkim=pass header.i=@salutedevices.com header.s=mail header.b=Li9suypQ; arc=pass (i=1 spf=pass spfdomain=salutedevices.com dkim=pass dkdomain=salutedevices.com dmarc=pass fromdomain=salutedevices.com); spf=pass (google.com: domain of linux-crypto+bounces-2437-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) smtp.mailfrom="linux-crypto+bounces-2437-linux.lists.archive=gmail.com@vger.kernel.org"; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=salutedevices.com Received: from smtp.subspace.kernel.org (wormhole.subspace.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by sv.mirrors.kernel.org (Postfix) with ESMTPS id A4DC1289BCB for ; Fri, 1 Mar 2024 13:33:20 +0000 (UTC) Received: from localhost.localdomain (localhost.localdomain [127.0.0.1]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 485D374267; Fri, 1 Mar 2024 13:30:20 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=salutedevices.com header.i=@salutedevices.com header.b="Li9suypQ" X-Original-To: linux-crypto@vger.kernel.org Received: from mx1.sberdevices.ru (mx2.sberdevices.ru [45.89.224.132]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 80D9671B3C; Fri, 1 Mar 2024 13:30:17 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=45.89.224.132 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709299820; cv=none; b=X9d7CSZZRj7tahQ6mJjkymbglQKP1RZSdrcJxmalqc1xeacurbjwX2AC2eKj0d4pOICp+BrLGgltyWffQMfSejzcumq1najeuEWZEUWh5w/8MwYU7B8bASW+XwU+vnoUg/OXxtQ9KNSTbNPs1bX3XJFyQbogxN43nWq820cNr6Q= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1709299820; c=relaxed/simple; bh=lsfHQurx8jIurwhrd9KnrxPUsprkToAxPjIRyIWDZtI=; h=From:To:CC:Subject:Date:Message-ID:In-Reply-To:References: MIME-Version:Content-Type; b=TT04aBgG9TMtKWk70aelTrBRhZzvOn02r55lIcXxJ0PKHVTb6hmOzKZrsHYbe15ILlgz1IaE+fQPiorblk5Sw0M1RVc2xFC5nIt9xEUYy9ZD5Vbflj3j/okjUGY2lZec2XJU6AhJihWkmTNLS90IllZvtKKfkXaR5QZ/Hl/yYGk= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=salutedevices.com; spf=pass smtp.mailfrom=salutedevices.com; dkim=pass (2048-bit key) header.d=salutedevices.com header.i=@salutedevices.com header.b=Li9suypQ; arc=none smtp.client-ip=45.89.224.132 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=salutedevices.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=salutedevices.com Received: from p-infra-ksmg-sc-msk02 (localhost [127.0.0.1]) by mx1.sberdevices.ru (Postfix) with ESMTP id A17BD1200ED; Fri, 1 Mar 2024 16:30:15 +0300 (MSK) DKIM-Filter: OpenDKIM Filter v2.11.0 mx1.sberdevices.ru A17BD1200ED DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=salutedevices.com; s=mail; t=1709299815; bh=/hD4RKgA8kdmVJ0Yd81c9subYETcwlyUA6WD27vbUp4=; h=From:To:Subject:Date:Message-ID:MIME-Version:Content-Type:From; b=Li9suypQfgAf4j3i7gtt+Zvf+GVXHC94+7M1qlV9IQTAXVSUdsBbwC87TydiCsjB3 BNDWc+H8XBh9+lEaSxmxEMJYO46FhXmjOCDmGNxOD/K7NwnXos4KFr8+6kSEp4iNXZ PNNxfGNKcrBOR03Nut1bzIFwqlMRFb7I28zRocKsd4Nt3wrYTfc0jUw6ZMAky/+1Kk 4YQEqcbSW7K3qLjwAR3cZ+up3v87/6exLRN2QtKH9ZtuYI4hoQc5SUUGginzCQnHGy ZlJVp60RCkZ5EkXOSRSmQvRaUeXPF7LGpbkRXyLRlO6V880EasYkUMUXXzgT11MmQz wVNT8wJvgaiEg== Received: from smtp.sberdevices.ru (p-i-exch-sc-m01.sberdevices.ru [172.16.192.107]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mx1.sberdevices.ru (Postfix) with ESMTPS; Fri, 1 Mar 2024 16:30:15 +0300 (MSK) Received: from user-A520M-DS3H.sigma.sbrf.ru (100.64.160.123) by p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) with Microsoft SMTP Server (version=TLS1_2, cipher=TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384) id 15.2.1118.40; Fri, 1 Mar 2024 16:30:14 +0300 From: Alexey Romanov To: , , , , , , , , , , CC: , , , , , , Alexey Romanov Subject: [PATCH v5 11/21] drivers: crypto: meson: introduce hasher Date: Fri, 1 Mar 2024 16:29:26 +0300 Message-ID: <20240301132936.621238-12-avromanov@salutedevices.com> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20240301132936.621238-1-avromanov@salutedevices.com> References: <20240301132936.621238-1-avromanov@salutedevices.com> Precedence: bulk X-Mailing-List: linux-crypto@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Content-Type: text/plain X-ClientProxiedBy: p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) To p-i-exch-sc-m01.sberdevices.ru (172.16.192.107) X-KSMG-Rule-ID: 10 X-KSMG-Message-Action: clean X-KSMG-AntiSpam-Lua-Profiles: 183875 [Feb 29 2024] X-KSMG-AntiSpam-Version: 6.1.0.3 X-KSMG-AntiSpam-Envelope-From: avromanov@salutedevices.com X-KSMG-AntiSpam-Rate: 0 X-KSMG-AntiSpam-Status: not_detected X-KSMG-AntiSpam-Method: none X-KSMG-AntiSpam-Auth: dkim=none X-KSMG-AntiSpam-Info: LuaCore: 7 0.3.7 6d6bf5bd8eea7373134f756a2fd73e9456bb7d1a, {Tracking_from_domain_doesnt_match_to}, FromAlignment: s, ApMailHostAddress: 100.64.160.123 X-MS-Exchange-Organization-SCL: -1 X-KSMG-AntiSpam-Interceptor-Info: scan successful X-KSMG-AntiPhishing: Clean X-KSMG-LinksScanning: Clean X-KSMG-AntiVirus: Kaspersky Secure Mail Gateway, version 2.0.1.6960, bases: 2024/02/29 19:21:00 #23899999 X-KSMG-AntiVirus-Status: Clean, skipped Introduce support for SHA1/SHA224/SHA256 hash algos. Tested via tcrypt and custom tests. Signed-off-by: Alexey Romanov --- drivers/crypto/amlogic/Makefile | 2 +- drivers/crypto/amlogic/amlogic-gxl-core.c | 25 +- drivers/crypto/amlogic/amlogic-gxl-hasher.c | 460 ++++++++++++++++++++ drivers/crypto/amlogic/amlogic-gxl.h | 51 +++ 4 files changed, 536 insertions(+), 2 deletions(-) create mode 100644 drivers/crypto/amlogic/amlogic-gxl-hasher.c diff --git a/drivers/crypto/amlogic/Makefile b/drivers/crypto/amlogic/Makefile index 39057e62c13e..4b6b388b7880 100644 --- a/drivers/crypto/amlogic/Makefile +++ b/drivers/crypto/amlogic/Makefile @@ -1,2 +1,2 @@ obj-$(CONFIG_CRYPTO_DEV_AMLOGIC_GXL) += amlogic-gxl-crypto.o -amlogic-gxl-crypto-y := amlogic-gxl-core.o amlogic-gxl-cipher.o +amlogic-gxl-crypto-y := amlogic-gxl-core.o amlogic-gxl-cipher.o amlogic-gxl-hasher.o diff --git a/drivers/crypto/amlogic/amlogic-gxl-core.c b/drivers/crypto/amlogic/amlogic-gxl-core.c index c7e3b402b14e..c5ee40196801 100644 --- a/drivers/crypto/amlogic/amlogic-gxl-core.c +++ b/drivers/crypto/amlogic/amlogic-gxl-core.c @@ -20,6 +20,9 @@ #include #include #include +#include +#include +#include #include "amlogic-gxl.h" @@ -173,6 +176,15 @@ int meson_register_algs(struct meson_dev *mc, struct meson_alg_template *algs, return err; } break; + case CRYPTO_ALG_TYPE_AHASH: + err = crypto_engine_register_ahash(&algs[i].alg.ahash); + if (err) { + dev_err(mc->dev, "Fail to register %s\n", + algs[i].alg.ahash.base.halg.base.cra_name); + meson_unregister_algs(mc, algs, count); + return err; + } + break; } } @@ -191,6 +203,9 @@ void meson_unregister_algs(struct meson_dev *mc, struct meson_alg_template *algs case CRYPTO_ALG_TYPE_SKCIPHER: crypto_engine_unregister_skcipher(&algs[i].alg.skcipher); break; + case CRYPTO_ALG_TYPE_AHASH: + crypto_engine_unregister_ahash(&algs[i].alg.ahash); + break; } } } @@ -238,13 +253,20 @@ static int meson_crypto_probe(struct platform_device *pdev) dbgfs_dir = debugfs_create_dir("gxl-crypto", NULL); debugfs_create_file("stats", 0444, dbgfs_dir, mc, &meson_debugfs_fops); - #ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG mc->dbgfs_dir = dbgfs_dir; #endif } + err = meson_hasher_register(mc); + if (err) + goto error_hasher; + return 0; + +error_hasher: + meson_cipher_unregister(mc); + error_flow: meson_free_chanlist(mc, mc->flow_cnt - 1); return err; @@ -267,6 +289,7 @@ static const struct meson_pdata meson_gxl_pdata = { .descs_reg = 0x0, .status_reg = 0x4, .setup_desc_cnt = 3, + .hasher_supported = false, }; static const struct of_device_id meson_crypto_of_match_table[] = { diff --git a/drivers/crypto/amlogic/amlogic-gxl-hasher.c b/drivers/crypto/amlogic/amlogic-gxl-hasher.c new file mode 100644 index 000000000000..1e7e24729120 --- /dev/null +++ b/drivers/crypto/amlogic/amlogic-gxl-hasher.c @@ -0,0 +1,460 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * Hardware asynchronous hasher for Amlogic SoC's. + * + * Copyright (c) 2023, SaluteDevices. All Rights Reserved. + * + * Author: Alexey Romanov + */ + +#include +#include +#include +#include +#include +#include +#include + +#include "amlogic-gxl.h" + +static int meson_sha_init(struct ahash_request *req) +{ + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm); + struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req); + + memset(rctx, 0, sizeof(struct meson_hasher_req_ctx)); + + rctx->flow = meson_get_engine_number(tctx->mc); + rctx->begin_req = true; + + return 0; +} + +static int meson_sha_update(struct ahash_request *req) +{ + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm); + struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req); + struct crypto_engine *engine = tctx->mc->chanlist[rctx->flow].engine; + + return crypto_transfer_hash_request_to_engine(engine, req); +} + +static int meson_sha_final(struct ahash_request *req) +{ + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm); + struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req); + struct crypto_engine *engine = tctx->mc->chanlist[rctx->flow].engine; + + rctx->final_req = true; + + return crypto_transfer_hash_request_to_engine(engine, req); +} + +static int meson_hasher_req_map(struct ahash_request *req) +{ + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm); + struct meson_dev *mc = tctx->mc; + int ret; + + if (!req->nbytes) + return 0; + + ret = dma_map_sg(mc->dev, req->src, sg_nents(req->src), DMA_TO_DEVICE); + if (!ret) { + dev_err(mc->dev, "Cannot DMA MAP request data\n"); + return -ENOMEM; + } + + return 0; +} + +static void meson_hasher_req_unmap(struct ahash_request *req) +{ + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm); + struct meson_dev *mc = tctx->mc; + + if (!req->nbytes) + return; + + dma_unmap_sg(mc->dev, req->src, sg_nents(req->src), DMA_TO_DEVICE); +} + +struct hasher_ctx { + struct crypto_async_request *areq; + + unsigned int tloffset; + unsigned int nbytes; + unsigned int todo; + + dma_addr_t state_addr; + dma_addr_t src_addr; + unsigned int src_offset; + struct scatterlist *src_sg; +}; + +static bool meson_final(struct hasher_ctx *ctx) +{ + struct ahash_request *req = ahash_request_cast(ctx->areq); + struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req); + + return !ctx->nbytes && rctx->final_req; +} + +static int meson_fill_partial_buffer(struct hasher_ctx *ctx, unsigned int len) +{ + struct ahash_request *req = ahash_request_cast(ctx->areq); + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm); + struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req); + struct meson_dev *mc = tctx->mc; + unsigned int blocksize = crypto_ahash_blocksize(tfm); + unsigned int copy; + + if (len) { + copy = min(blocksize - rctx->partial_size, len); + memcpy(rctx->partial + rctx->partial_size, + sg_virt(ctx->src_sg) + ctx->src_offset, copy); + + rctx->partial_size += copy; + ctx->nbytes -= copy; + ctx->src_offset += copy; + } + + if (rctx->partial_size == blocksize || meson_final(ctx)) { + rctx->partial_addr = dma_map_single(mc->dev, + rctx->partial, + rctx->partial_size, + DMA_TO_DEVICE); + if (dma_mapping_error(mc->dev, rctx->partial_addr)) { + dev_err(mc->dev, "Cannot DMA MAP SHA partial buffer\n"); + return -ENOMEM; + } + + rctx->partial_mapped = true; + ctx->todo = rctx->partial_size; + ctx->src_addr = rctx->partial_addr; + } + + return 0; +} + +static unsigned int meson_setup_data_descs(struct hasher_ctx *ctx) +{ + struct ahash_request *req = ahash_request_cast(ctx->areq); + struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req); + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm); + struct meson_dev *mc = tctx->mc; + struct meson_flow *flow = &mc->chanlist[rctx->flow]; + struct hash_alg_common *alg = crypto_hash_alg_common(tfm); + struct meson_alg_template *algt = container_of(alg, + struct meson_alg_template, alg.ahash.base.halg); + struct meson_desc *desc = &flow->tl[ctx->tloffset]; + u32 v; + + ctx->tloffset++; + + v = DESC_OWN | DESC_ENCRYPTION | DESC_OPMODE_SHA | + ctx->todo | algt->blockmode; + if (rctx->begin_req) { + rctx->begin_req = false; + v |= DESC_BEGIN; + } + + if (!ctx->nbytes && rctx->final_req) { + rctx->final_req = false; + v |= DESC_END; + } + + if (!ctx->nbytes || ctx->tloffset == MAXDESC || rctx->partial_mapped) + v |= DESC_LAST; + + desc->t_src = cpu_to_le32(ctx->src_addr); + desc->t_dst = cpu_to_le32(ctx->state_addr); + desc->t_status = cpu_to_le32(v); + + return v & DESC_LAST; +} + +static int meson_kick_hardware(struct hasher_ctx *ctx) +{ + struct ahash_request *req = ahash_request_cast(ctx->areq); + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req); + struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm); + struct meson_dev *mc = tctx->mc; + struct meson_flow *flow = &mc->chanlist[rctx->flow]; + int ret; + + reinit_completion(&flow->complete); + meson_dma_start(mc, rctx->flow); + + ret = wait_for_completion_timeout(&flow->complete, + msecs_to_jiffies(500)); + if (ret == 0) { + dev_err(mc->dev, "DMA timeout for flow %d\n", rctx->flow); + return -EINVAL; + } else if (ret < 0) { + dev_err(mc->dev, "Waiting for DMA completion is failed (%d)\n", ret); + return ret; + } + + if (rctx->partial_mapped) { + dma_unmap_single(mc->dev, rctx->partial_addr, + rctx->partial_size, + DMA_TO_DEVICE); + rctx->partial_size = 0; + rctx->partial_mapped = false; + } + + ctx->tloffset = 0; + + return 0; +} + +static void meson_setup_state_descs(struct hasher_ctx *ctx) +{ + struct ahash_request *req = ahash_request_cast(ctx->areq); + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req); + struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm); + struct meson_dev *mc = tctx->mc; + struct meson_desc *desc; + int i; + + if (ctx->tloffset || rctx->begin_req) + return; + + for (i = 0; i < mc->pdata->setup_desc_cnt; i++) { + int offset = i * 16; + + desc = &mc->chanlist[rctx->flow].tl[ctx->tloffset]; + desc->t_src = cpu_to_le32(ctx->state_addr + offset); + desc->t_dst = cpu_to_le32(offset); + desc->t_status = cpu_to_le32(MESON_SHA_BUFFER_SIZE | + DESC_MODE_KEY | DESC_OWN); + + ctx->tloffset++; + } +} + +static int meson_hasher_do_one_request(struct crypto_engine *engine, void *areq) +{ + struct ahash_request *req = ahash_request_cast(areq); + struct crypto_ahash *tfm = crypto_ahash_reqtfm(req); + struct meson_hasher_tfm_ctx *tctx = crypto_ahash_ctx(tfm); + struct meson_hasher_req_ctx *rctx = ahash_request_ctx(req); + struct meson_dev *mc = tctx->mc; + struct hasher_ctx ctx = { + .tloffset = 0, + .src_offset = 0, + .nbytes = req->nbytes, + .src_sg = req->src, + .areq = areq, + }; + unsigned int blocksize = crypto_ahash_blocksize(tfm); + unsigned int digest_size = crypto_ahash_digestsize(tfm); + bool final_req = rctx->final_req; + int ret; + + ctx.state_addr = dma_map_single(mc->dev, rctx->state, + sizeof(rctx->state), DMA_BIDIRECTIONAL); + ret = dma_mapping_error(mc->dev, ctx.state_addr); + if (ret) { + dev_err(mc->dev, "Cannot DMA MAP SHA state buffer"); + goto fail_map_single; + } + + ret = meson_hasher_req_map(req); + if (ret) + goto fail_map_req; + + for (;;) { + unsigned int len = ctx.src_sg ? + min(sg_dma_len(ctx.src_sg) - ctx.src_offset, ctx.nbytes) : 0; + + ctx.src_addr = 0; + ctx.todo = 0; + + if (!rctx->final_req && !ctx.nbytes) + break; + + meson_setup_state_descs(&ctx); + + if (rctx->partial_size && rctx->partial_size < blocksize) { + ret = meson_fill_partial_buffer(&ctx, len); + if (ret) + goto fail; + } else if (len && len < blocksize) { + memcpy(rctx->partial, sg_virt(ctx.src_sg) + ctx.src_offset, len); + + rctx->partial_size = len; + ctx.nbytes -= len; + ctx.src_offset += len; + } else if (len) { + ctx.src_addr = sg_dma_address(ctx.src_sg) + ctx.src_offset; + ctx.todo = min(rounddown(DESC_MAXLEN, blocksize), + rounddown(len, blocksize)); + ctx.nbytes -= ctx.todo; + ctx.src_offset += ctx.todo; + } + + if (ctx.src_sg && ctx.src_offset == sg_dma_len(ctx.src_sg)) { + ctx.src_offset = 0; + ctx.src_sg = sg_next(ctx.src_sg); + } + + if (!ctx.todo && ctx.nbytes) + continue; + + if (!ctx.todo && !rctx->final_req && !ctx.tloffset) + continue; + + if (meson_setup_data_descs(&ctx)) { + ret = meson_kick_hardware(&ctx); + if (ret) + goto fail; + } + } + +fail: + meson_hasher_req_unmap(req); + +fail_map_req: + dma_unmap_single(mc->dev, ctx.state_addr, sizeof(rctx->state), + DMA_BIDIRECTIONAL); + +fail_map_single: + if (final_req && ret == 0) + memcpy(req->result, rctx->state, digest_size); + + local_bh_disable(); + crypto_finalize_hash_request(engine, req, ret); + local_bh_enable(); + + return ret; +} + +static int meson_hasher_init(struct crypto_tfm *tfm) +{ + struct meson_hasher_tfm_ctx *tctx = crypto_tfm_ctx(tfm); + struct crypto_ahash *atfm = __crypto_ahash_cast(tfm); + struct hash_alg_common *alg = crypto_hash_alg_common(atfm); + struct meson_alg_template *algt = container_of(alg, + struct meson_alg_template, alg.ahash.base.halg); + + crypto_ahash_set_reqsize(atfm, crypto_ahash_statesize(atfm)); + + memset(tctx, 0, sizeof(struct meson_hasher_tfm_ctx)); + + tctx->mc = algt->mc; + + return 0; +} + +static struct meson_alg_template mc_algs[] = { +{ + .type = CRYPTO_ALG_TYPE_AHASH, + .blockmode = DESC_MODE_SHA1, + .alg.ahash.base = { + .halg = { + .base = { + .cra_name = "sha1", + .cra_driver_name = "sha1-gxl", + .cra_priority = 400, + .cra_blocksize = SHA1_BLOCK_SIZE, + .cra_flags = CRYPTO_ALG_ASYNC, + .cra_ctxsize = sizeof(struct meson_hasher_tfm_ctx), + .cra_module = THIS_MODULE, + .cra_alignmask = 0, + .cra_init = meson_hasher_init, + }, + .digestsize = SHA1_DIGEST_SIZE, + .statesize = sizeof(struct meson_hasher_req_ctx), + }, + .init = meson_sha_init, + .update = meson_sha_update, + .final = meson_sha_final, + }, + .alg.ahash.op = { + .do_one_request = meson_hasher_do_one_request, + }, +}, +{ + .type = CRYPTO_ALG_TYPE_AHASH, + .blockmode = DESC_MODE_SHA224, + .alg.ahash.base = { + .halg = { + .base = { + .cra_name = "sha224", + .cra_driver_name = "sha224-gxl", + .cra_priority = 400, + .cra_blocksize = SHA224_BLOCK_SIZE, + .cra_flags = CRYPTO_ALG_ASYNC, + .cra_ctxsize = sizeof(struct meson_hasher_tfm_ctx), + .cra_module = THIS_MODULE, + .cra_alignmask = 0, + .cra_init = meson_hasher_init, + }, + .digestsize = SHA224_DIGEST_SIZE, + .statesize = sizeof(struct meson_hasher_req_ctx), + }, + .init = meson_sha_init, + .update = meson_sha_update, + .final = meson_sha_final, + }, + .alg.ahash.op = { + .do_one_request = meson_hasher_do_one_request, + }, +}, +{ + .type = CRYPTO_ALG_TYPE_AHASH, + .blockmode = DESC_MODE_SHA256, + .alg.ahash.base = { + .halg = { + .base = { + .cra_name = "sha256", + .cra_driver_name = "sha256-gxl", + .cra_priority = 400, + .cra_blocksize = SHA256_BLOCK_SIZE, + .cra_flags = CRYPTO_ALG_ASYNC, + .cra_ctxsize = sizeof(struct meson_hasher_tfm_ctx), + .cra_module = THIS_MODULE, + .cra_alignmask = 0, + .cra_init = meson_hasher_init, + }, + .digestsize = SHA256_DIGEST_SIZE, + .statesize = sizeof(struct meson_hasher_req_ctx), + }, + .init = meson_sha_init, + .update = meson_sha_update, + .final = meson_sha_final, + }, + .alg.ahash.op = { + .do_one_request = meson_hasher_do_one_request, + }, +}, +}; + +int meson_hasher_register(struct meson_dev *mc) +{ + if (!mc->pdata->hasher_supported) { + pr_info("amlogic-gxl-hasher: hasher not supported at current platform"); + return 0; + } + + return meson_register_algs(mc, mc_algs, ARRAY_SIZE(mc_algs)); +} + +void meson_hasher_unregister(struct meson_dev *mc) +{ + if (!mc->pdata->hasher_supported) + return; + + meson_unregister_algs(mc, mc_algs, ARRAY_SIZE(mc_algs)); +} diff --git a/drivers/crypto/amlogic/amlogic-gxl.h b/drivers/crypto/amlogic/amlogic-gxl.h index dc5218064f99..cebdfd81f6e2 100644 --- a/drivers/crypto/amlogic/amlogic-gxl.h +++ b/drivers/crypto/amlogic/amlogic-gxl.h @@ -5,6 +5,7 @@ * Copyright (C) 2018-2019 Corentin LABBE */ #include +#include #include #include #include @@ -23,13 +24,25 @@ #define DESC_OPMODE_ECB (0 << 26) #define DESC_OPMODE_CBC (1 << 26) +#define DESC_OPMODE_SHA (0 << 26) #define DESC_MAXLEN GENMASK(16, 0) +#define DESC_MODE_SHA1 (0x5 << 20) +#define DESC_MODE_SHA224 (0x7 << 20) +#define DESC_MODE_SHA256 (0x6 << 20) + #define DESC_LAST BIT(18) +#define DESC_BEGIN BIT(24) +#define DESC_END BIT(25) #define DESC_ENCRYPTION BIT(28) #define DESC_OWN BIT(31) +#define MESON_SHA_BUFFER_SIZE (SHA256_DIGEST_SIZE + 16) + +#define MESON_SHA_BEGIN BIT(1) +#define MESON_SHA_FINAL BIT(2) + /* * struct meson_desc - Descriptor for DMA operations * Note that without datasheet, some are unknown @@ -83,11 +96,13 @@ struct meson_flow { * @reg_descs: offset to descriptors register * @reg_status: offset to status register * @setup_desc_cnt: number of setup descriptor to configure. + * @hasher_supported: indecates whether hasher is supported. */ struct meson_pdata { u32 descs_reg; u32 status_reg; u32 setup_desc_cnt; + bool hasher_supported; }; /* @@ -141,6 +156,38 @@ struct meson_cipher_tfm_ctx { struct crypto_skcipher *fallback_tfm; }; +/* + * struct meson_hasher_req_ctx - context for a hasher request + * @state: state data + * @partial: partial buffer data. Contains sent data which + * size < blocksize + * @partial_size: size of the partial buffer + * @partial_addr: physical address of partial buffer + * @partial_mapped: indicates is partial buffer currently mapped or not + * @flags: request flags (for example, is this final req or not) + * @flow: the flow to use for this request + */ +struct meson_hasher_req_ctx { + u8 state[SHA256_DIGEST_SIZE + 16] ____cacheline_aligned; + u8 partial[SHA256_BLOCK_SIZE] ____cacheline_aligned; + unsigned int partial_size ____cacheline_aligned; + dma_addr_t partial_addr; + bool partial_mapped; + + bool begin_req; + bool final_req; + int flow; +}; + +/* + * struct meson_hasher_tfm_ctx - context for a hasher TFM + * @enginectx: crypto_engine used by this TFM + * @mc: pointer to the private data of driver handling this TFM + */ +struct meson_hasher_tfm_ctx { + struct meson_dev *mc; +}; + /* * struct meson_alg_template - crypto_alg template * @type: the CRYPTO_ALG_TYPE for this template @@ -155,6 +202,7 @@ struct meson_alg_template { u32 blockmode; union { struct skcipher_engine_alg skcipher; + struct ahash_engine_alg ahash; } alg; struct meson_dev *mc; #ifdef CONFIG_CRYPTO_DEV_AMLOGIC_GXL_DEBUG @@ -178,3 +226,6 @@ int meson_cipher_register(struct meson_dev *mc); void meson_cipher_unregister(struct meson_dev *mc); void meson_cipher_debugfs_show(struct seq_file *seq, void *v); int meson_handle_cipher_request(struct crypto_engine *engine, void *areq); + +int meson_hasher_register(struct meson_dev *mc); +void meson_hasher_unregister(struct meson_dev *mc); -- 2.34.1