Received-SPF: softfail (google.com: domain of transitioning linux-crypto-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
From:   Ard Biesheuvel <ardb@kernel.org>
To:     linux-crypto@vger.kernel.org
Cc:     herbert@gondor.apana.org.au, keescook@chromium.org,
        Ard Biesheuvel <ardb@kernel.org>
Subject: [PATCH 7/8] crypto: ahash - avoid DMA alignment for request structures unless needed
Date:   Wed,  6 Apr 2022 16:27:14 +0200
Message-Id: <20220406142715.2270256-8-ardb@kernel.org>
In-Reply-To: <20220406142715.2270256-1-ardb@kernel.org>
References: <20220406142715.2270256-1-ardb@kernel.org>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

AHASH request structures are currently aligned to minimal DMA alignment
for the arcitecture, which defaults to 128 bytes on arm64. This is
excessive, and rarely needed, i.e., only when doing non-coherent inbound
DMA on the contents of the request context buffer. So let's relax this
requirement, and only use this alignment if the
CRYPTO_ALG_NEED_DMA_ALIGNMENT flag is set by the implementation.

Signed-off-by: Ard Biesheuvel <ardb@kernel.org>
---
 include/crypto/hash.h          |  5 +++--
 include/crypto/internal/hash.h | 10 +++++++++-
 2 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/include/crypto/hash.h b/include/crypto/hash.h
index f140e4643949..cd16c37c38af 100644
--- a/include/crypto/hash.h
+++ b/include/crypto/hash.h
@@ -56,7 +56,7 @@ struct ahash_request {
 	/* This field may only be used by the ahash API code. */
 	void *priv;
 
-	void *__ctx[] CRYPTO_MINALIGN_ATTR;
+	void *__ctx[] CRYPTO_REQ_MINALIGN_ATTR;
 };
 
 /**
@@ -417,7 +417,8 @@ static inline unsigned int crypto_ahash_reqsize(struct crypto_ahash *tfm)
 
 static inline void *ahash_request_ctx(struct ahash_request *req)
 {
-	return req->__ctx;
+	return PTR_ALIGN(&req->__ctx,
+			 crypto_tfm_alg_req_alignmask(req->base.tfm) + 1);
 }
 
 /**
diff --git a/include/crypto/internal/hash.h b/include/crypto/internal/hash.h
index 25806141db59..222c2df009c6 100644
--- a/include/crypto/internal/hash.h
+++ b/include/crypto/internal/hash.h
@@ -143,7 +143,15 @@ static inline struct ahash_alg *__crypto_ahash_alg(struct crypto_alg *alg)
 static inline void crypto_ahash_set_reqsize(struct crypto_ahash *tfm,
 					    unsigned int reqsize)
 {
-	tfm->reqsize = reqsize;
+	unsigned int align = crypto_tfm_alg_req_alignmask(crypto_ahash_tfm(tfm)) + 1;
+
+	/*
+	 * The request structure itself is only aligned to CRYPTO_REQ_MINALIGN,
+	 * so we need to add some headroom, allowing us to return a suitably
+	 * aligned context buffer pointer. We also need to round up the size so
+	 * we don't end up sharing a cacheline at the end of the buffer.
+	 */
+	tfm->reqsize = ALIGN(reqsize, align) + align - CRYPTO_REQ_MINALIGN;
 }
 
 static inline struct crypto_instance *ahash_crypto_instance(
-- 
2.30.2