BACKPORT, FROMGIT: crypto: arm/chacha20 - limit the preemption-disabled section

To improve responsivesess, disable preemption for each step of the walk
(which is at most PAGE_SIZE) rather than for the entire
encryption/decryption operation.

Suggested-by: Ard Biesheuvel <ard.biesheuvel@linaro.org>
Signed-off-by: Eric Biggers <ebiggers@google.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>
(cherry picked from commit be2830b15b60011845ad701076511e8b93b2fd76
 https://git.kernel.org/pub/scm/linux/kernel/git/herbert/cryptodev-2.6.git master)

Conflicts:
	arch/arm/crypto/chacha20-neon-glue.c

Bug: 112008522
Test: As series, see Ic61c13b53facfd2173065be715a7ee5f3af8760b
Change-Id: I21bfa9c14635e695b128c87df53fea505c3cdd4e
Signed-off-by: Eric Biggers <ebiggers@google.com>

diff --git a/arch/arm/crypto/chacha20-neon-glue.c b/arch/arm/crypto/chacha20-neon-glue.c
index 2647e07..83c1c85 100644 (file)
--- a/arch/arm/crypto/chacha20-neon-glue.c
+++ b/arch/arm/crypto/chacha20-neon-glue.c
@@ -67,29 +67,28 @@ static int chacha20_simd(struct blkcipher_desc *desc, struct scatterlist *dst,
        if (nbytes <= CHACHA_BLOCK_SIZE || !may_use_simd())
                return crypto_chacha_crypt(desc, dst, src, nbytes);
 
-       desc->flags &= ~CRYPTO_TFM_REQ_MAY_SLEEP;
        blkcipher_walk_init(&walk, dst, src, nbytes);
        err = blkcipher_walk_virt_block(desc, &walk, CHACHA_BLOCK_SIZE);
 
        crypto_chacha_init(state, crypto_blkcipher_ctx(desc->tfm), walk.iv);
 
-       kernel_neon_begin();
-
        while (walk.nbytes >= CHACHA_BLOCK_SIZE) {
+               kernel_neon_begin();
                chacha20_dosimd(state, walk.dst.virt.addr, walk.src.virt.addr,
                                rounddown(walk.nbytes, CHACHA_BLOCK_SIZE));
+               kernel_neon_end();
                err = blkcipher_walk_done(desc, &walk,
                                          walk.nbytes % CHACHA_BLOCK_SIZE);
        }
 
        if (walk.nbytes) {
+               kernel_neon_begin();
                chacha20_dosimd(state, walk.dst.virt.addr, walk.src.virt.addr,
                                walk.nbytes);
+               kernel_neon_end();
                err = blkcipher_walk_done(desc, &walk, 0);
        }
 
-       kernel_neon_end();
-
        return err;
 }