2016-12-13 14:32:20

by Ilya Lesokhin

Subject: [PATCH] crypto: aesni-intel - RFC4106 can zero copy when !PageHighMem

In the common case of !PageHighMem we can do zero copy crypto
even if sg crosses a pages boundary.

Signed-off-by: Ilya Lesokhin <[email protected]>
---
arch/x86/crypto/aesni-intel_glue.c | 12 ++++++++----
1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/arch/x86/crypto/aesni-intel_glue.c b/arch/x86/crypto/aesni-intel_glue.c
index b1ab0cb..b494ad7 100644
--- a/arch/x86/crypto/aesni-intel_glue.c
+++ b/arch/x86/crypto/aesni-intel_glue.c
@@ -903,9 +903,11 @@ static int helper_rfc4106_encrypt(struct aead_request *req)
*((__be32 *)(iv+12)) = counter;

if (sg_is_last(req->src) &&
- req->src->offset + req->src->length <= PAGE_SIZE &&
+ (!PageHighMem(sg_page(req->src)) ||
+ req->src->offset + req->src->length <= PAGE_SIZE) &&
sg_is_last(req->dst) &&
- req->dst->offset + req->dst->length <= PAGE_SIZE) {
+ (!PageHighMem(sg_page(req->dst)) ||
+ req->dst->offset + req->dst->length <= PAGE_SIZE)) {
one_entry_in_sg = 1;
scatterwalk_start(&src_sg_walk, req->src);
assoc = scatterwalk_map(&src_sg_walk);
@@ -990,9 +992,11 @@ static int helper_rfc4106_decrypt(struct aead_request *req)
*((__be32 *)(iv+12)) = counter;

if (sg_is_last(req->src) &&
- req->src->offset + req->src->length <= PAGE_SIZE &&
+ (!PageHighMem(sg_page(req->src)) ||
+ req->src->offset + req->src->length <= PAGE_SIZE) &&
sg_is_last(req->dst) &&
- req->dst->offset + req->dst->length <= PAGE_SIZE) {
+ (!PageHighMem(sg_page(req->dst)) ||
+ req->dst->offset + req->dst->length <= PAGE_SIZE)) {
one_entry_in_sg = 1;
scatterwalk_start(&src_sg_walk, req->src);
assoc = scatterwalk_map(&src_sg_walk);
--
1.8.3.1

2016-12-13 19:07:58

by Dave Watson

Subject: Re: [PATCH] crypto: aesni-intel - RFC4106 can zero copy when !PageHighMem

On 12/13/16 04:32 PM, Ilya Lesokhin wrote:
> --- a/arch/x86/crypto/aesni-intel_glue.c
> +++ b/arch/x86/crypto/aesni-intel_glue.c
> @@ -903,9 +903,11 @@ static int helper_rfc4106_encrypt(struct aead_request *req)
> *((__be32 *)(iv+12)) = counter;
>
> if (sg_is_last(req->src) &&
> - req->src->offset + req->src->length <= PAGE_SIZE &&
> + (!PageHighMem(sg_page(req->src)) ||
> + req->src->offset + req->src->length <= PAGE_SIZE) &&
> sg_is_last(req->dst) &&
> - req->dst->offset + req->dst->length <= PAGE_SIZE) {
> + (!PageHighMem(sg_page(req->dst)) ||
> + req->dst->offset + req->dst->length <= PAGE_SIZE)) {
> one_entry_in_sg = 1;
> scatterwalk_start(&src_sg_walk, req->src);
> assoc = scatterwalk_map(&src_sg_walk);

I was also experimenting with a similar patch that loosened up the
restrictions here, checking for highmem. Note that you can go even
further and check the AAD, data, and TAG all separately, the current
aesni crypto routines take them as separate buffers. (This might fix
the RFC5288 patch AAD size issue?)

Long term it would be nice to improve the asm routines instead to
support scatter / gather IO and any AAD len, as the newer intel
routines do:

https://github.com/01org/isa-l_crypto/tree/master/aes

2016-12-27 10:05:39

by Herbert Xu

Subject: Re: [PATCH] crypto: aesni-intel - RFC4106 can zero copy when !PageHighMem

On Tue, Dec 13, 2016 at 04:32:06PM +0200, Ilya Lesokhin wrote:
> In the common case of !PageHighMem we can do zero copy crypto
> even if sg crosses a pages boundary.
>
> Signed-off-by: Ilya Lesokhin <[email protected]>

Patch applied. Thanks.
--
Email: Herbert Xu <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt