From: Aseem Rastogi
crypto/zlib: When zlib_deflate is called with Z_FINISH flag, it should be
considered valid if it returns Z_OK instead of Z_STREAM_END.
zlib_defalte allows callers to call it with Z_FINISH flag multiple times. This
is useful if output buffer did not have enough space first time zlib_deflate
was called with Z_FINISH set. zlib_deflate returns Z_OK in that case and
expects that caller will invoke it again with Z_FINISH set to collect
remaining data. So, return value from zlib_deflate can be Z_OK OR
Z_STREAM_END. zlib_compress_final is modified to allow return value of Z_OK.
Signed-off-by: Aseem Rastogi <[email protected]>
Tested-by: Aseem Rastogi <[email protected]>
Reviewed-by: Vasily Tarasov <[email protected]>
diff --git a/crypto/zlib.c b/crypto/zlib.c
index c301573..c63c520 100644
--- a/crypto/zlib.c
+++ b/crypto/zlib.c
@@ -190,7 +190,7 @@ static int zlib_compress_final(struct crypto_pcomp *tfm,
stream->avail_out = req->avail_out;
ret = zlib_deflate(stream, Z_FINISH);
- if (ret != Z_STREAM_END) {
+ if ((ret != Z_STREAM_END) && (ret != Z_OK)) {
pr_debug("zlib_deflate failed %d\n", ret);
return -EINVAL;
}
On Sun, Dec 20, 2009 at 02:34:09PM -0500, Aseem Rastogi wrote:
> From: Aseem Rastogi
>
> crypto/zlib: When zlib_deflate is called with Z_FINISH flag, it should be
> considered valid if it returns Z_OK instead of Z_STREAM_END.
>
> zlib_defalte allows callers to call it with Z_FINISH flag multiple times. This
> is useful if output buffer did not have enough space first time zlib_deflate
> was called with Z_FINISH set. zlib_deflate returns Z_OK in that case and
> expects that caller will invoke it again with Z_FINISH set to collect
> remaining data. So, return value from zlib_deflate can be Z_OK OR
> Z_STREAM_END. zlib_compress_final is modified to allow return value of Z_OK.
>
> Signed-off-by: Aseem Rastogi <[email protected]>
> Tested-by: Aseem Rastogi <[email protected]>
> Reviewed-by: Vasily Tarasov <[email protected]>
Hmm, it seems that the semantics of the final function isn't
quite right. It was never meant to be called multiple times.
Let me see if we can fix this up without touching the interface.
Thanks,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
On Wed, Dec 23, 2009 at 08:01:57PM +0800, Herbert Xu wrote:
>
> Hmm, it seems that the semantics of the final function isn't
> quite right. It was never meant to be called multiple times.
>
> Let me see if we can fix this up without touching the interface.
How about this patch? So now the final functions are more like
their zlib counterparts. If it runs out of output buffer, it
returns avail_out + 1 and you need to call it again.
diff --git a/crypto/zlib.c b/crypto/zlib.c
index c301573..632b4b4 100644
--- a/crypto/zlib.c
+++ b/crypto/zlib.c
@@ -190,19 +190,23 @@ static int zlib_compress_final(struct crypto_pcomp *tfm,
stream->avail_out = req->avail_out;
ret = zlib_deflate(stream, Z_FINISH);
+ if (ret == Z_OK)
+ ret = req->avail_out + 1;
if (ret != Z_STREAM_END) {
pr_debug("zlib_deflate failed %d\n", ret);
return -EINVAL;
- }
+ } else
+ ret = req->avail_out - stream->avail_out;
- ret = req->avail_out - stream->avail_out;
pr_debug("avail_in %u, avail_out %u (consumed %u, produced %u)\n",
stream->avail_in, stream->avail_out,
req->avail_in - stream->avail_in, ret);
+
req->next_in = stream->next_in;
req->avail_in = stream->avail_in;
req->next_out = stream->next_out;
req->avail_out = stream->avail_out;
+
return ret;
}
@@ -323,15 +327,19 @@ static int zlib_decompress_final(struct crypto_pcomp *tfm,
}
} else
ret = zlib_inflate(stream, Z_FINISH);
- if (ret != Z_STREAM_END) {
+
+ if (ret == Z_OK)
+ ret = req->avail_out + 1;
+ else if (ret != Z_STREAM_END) {
pr_debug("zlib_inflate failed %d\n", ret);
return -EINVAL;
- }
+ } else
+ ret = req->avail_out - stream->avail_out;
- ret = req->avail_out - stream->avail_out;
pr_debug("avail_in %u, avail_out %u (consumed %u, produced %u)\n",
stream->avail_in, stream->avail_out,
req->avail_in - stream->avail_in, ret);
+
req->next_in = stream->next_in;
req->avail_in = stream->avail_in;
req->next_out = stream->next_out;
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt