Received-SPF: pass (google.com: domain of linux-crypto-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
References: <20201025143119.1054168-1-nivedita@alum.mit.edu> <20201025143119.1054168-7-nivedita@alum.mit.edu>
In-Reply-To: <20201025143119.1054168-7-nivedita@alum.mit.edu>
From:   Ard Biesheuvel <ardb@kernel.org>
Date:   Mon, 26 Oct 2020 09:02:10 +0100
Message-ID: <CAMj1kXEiNGnUmo=ZR3=Eij8hnD0kZef8rEcG2H7Q7vq4OeyTSA@mail.gmail.com>
Subject: Re: [PATCH v4 6/6] crypto: lib/sha256 - Unroll LOAD and BLEND loops
To:     Arvind Sankar <nivedita@alum.mit.edu>
Cc:     Herbert Xu <herbert@gondor.apana.org.au>,
        "David S. Miller" <davem@davemloft.net>,
        "linux-crypto@vger.kernel.org" <linux-crypto@vger.kernel.org>,
        Eric Biggers <ebiggers@kernel.org>,
        David Laight <David.Laight@aculab.com>,
        Linux Kernel Mailing List <linux-kernel@vger.kernel.org>,
        Eric Biggers <ebiggers@google.com>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Sun, 25 Oct 2020 at 15:31, Arvind Sankar <nivedita@alum.mit.edu> wrote:
>
> Unrolling the LOAD and BLEND loops improves performance by ~8% on x86_64
> (tested on Broadwell Xeon) while not increasing code size too much.
>
> Signed-off-by: Arvind Sankar <nivedita@alum.mit.edu>
> Reviewed-by: Eric Biggers <ebiggers@google.com>

Acked-by: Ard Biesheuvel <ardb@kernel.org>

> ---
>  lib/crypto/sha256.c | 24 ++++++++++++++++++++----
>  1 file changed, 20 insertions(+), 4 deletions(-)
>
> diff --git a/lib/crypto/sha256.c b/lib/crypto/sha256.c
> index e2e29d9b0ccd..cdef37c05972 100644
> --- a/lib/crypto/sha256.c
> +++ b/lib/crypto/sha256.c
> @@ -76,12 +76,28 @@ static void sha256_transform(u32 *state, const u8 *input, u32 *W)
>         int i;
>
>         /* load the input */
> -       for (i = 0; i < 16; i++)
> -               LOAD_OP(i, W, input);
> +       for (i = 0; i < 16; i += 8) {
> +               LOAD_OP(i + 0, W, input);
> +               LOAD_OP(i + 1, W, input);
> +               LOAD_OP(i + 2, W, input);
> +               LOAD_OP(i + 3, W, input);
> +               LOAD_OP(i + 4, W, input);
> +               LOAD_OP(i + 5, W, input);
> +               LOAD_OP(i + 6, W, input);
> +               LOAD_OP(i + 7, W, input);
> +       }
>
>         /* now blend */
> -       for (i = 16; i < 64; i++)
> -               BLEND_OP(i, W);
> +       for (i = 16; i < 64; i += 8) {
> +               BLEND_OP(i + 0, W);
> +               BLEND_OP(i + 1, W);
> +               BLEND_OP(i + 2, W);
> +               BLEND_OP(i + 3, W);
> +               BLEND_OP(i + 4, W);
> +               BLEND_OP(i + 5, W);
> +               BLEND_OP(i + 6, W);
> +               BLEND_OP(i + 7, W);
> +       }
>
>         /* load the state into our registers */
>         a = state[0];  b = state[1];  c = state[2];  d = state[3];
> --
> 2.26.2
>