Hi,
I have a few questions about the "deflate" crypto module (crypto/deflate.c).
1. Why does it support decompression of full chunks only (i.e. all compressed
data has to be passed at once)? Is there a specific reason for that?
Or is it just because so far nobody needed partial decompression?
2. Why was the lazy allocation removed back in 2004?
If you're interested in decompression only, it's a bit wasteful to
allocate 262 KiB of memory for compression and never use it.
Thanks for your answers!
Please CC me, as I'm not subscribed, and
http://vger.kernel.org/vger-lists.html#linux-crypto doesn't list any archives.
With kind regards,
Geert Uytterhoeven
Software Architect
Sony Techsoft Centre
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium
Phone: +32 (0)2 700 8453
Fax: +32 (0)2 700 8622
E-mail: [email protected]
Internet: http://www.sony-europe.com/
Sony Technology and Software Centre Europe
A division of Sony Service Centre (Europe) N.V.
Registered office: Technologielaan 7 · B-1840 Londerzeel · Belgium
VAT BE 0413.825.160 · RPR Brussels
Fortis 293-0376800-10 GEBA-BE-BB
* Geert Uytterhoeven | 2008-06-25 19:26:28 [+0200]:
> Hi,
Hello,
>
>I have a few questions about the "deflate" crypto module (crypto/deflate.c).
>
> 1. Why does it support decompression of full chunks only (i.e. all compressed
> data has to be passed at once)? Is there a specific reason for that?
> Or is it just because so far nobody needed partial decompression?
I guess the latter. The only user is ipcomp_decompress() as far as I can
see. Other zlib users like ppp or jffs2 user zlib directly. However, the
crypto API is the only place in kernel which provides you a generic
interface for compression.
>Please CC me, as I'm not subscribed, and
>http://vger.kernel.org/vger-lists.html#linux-crypto doesn't list any archives.
Try
http://www.mail-archive.com/[email protected]/
or
http://marc.info/?l=linux-crypto-vger&r=1&b=200806&w=2
David: do you thing it is a good idea to add them to the list?
>Geert Uytterhoeven
>Software Architect
Sebastian
From: Sebastian Siewior <[email protected]>
Date: Wed, 25 Jun 2008 21:14:57 +0200
> http://www.mail-archive.com/[email protected]/
> or
> http://marc.info/?l=linux-crypto-vger&r=1&b=200806&w=2
I've added this to the vger list info for linux-crypto
as follows:
--------------------
This is the Linux kernel cryptograpic layer discussion and bug
reporting mailing list.
Archives:
http://www.mail-archive.com/[email protected]/
http://marc.info/?l=linux-crypto-vger
--------------------
It will show up on the web page the next time the cron
jobs run which generate them.
On Wed, 25 Jun 2008, Sebastian Siewior wrote:
> * Geert Uytterhoeven | 2008-06-25 19:26:28 [+0200]:
> >I have a few questions about the "deflate" crypto module (crypto/deflate.c).
> >
> > 1. Why does it support decompression of full chunks only (i.e. all compressed
> > data has to be passed at once)? Is there a specific reason for that?
> > Or is it just because so far nobody needed partial decompression?
> I guess the latter. The only user is ipcomp_decompress() as far as I can
IC.
> see. Other zlib users like ppp or jffs2 user zlib directly. However, the
> crypto API is the only place in kernel which provides you a generic
> interface for compression.
Indeed. So if a hardware-assisted deflate would be available as a crypto
module, "deflate" crypto module users would benefit from it automatically,
right? While direct zlib users are out of luck.
With kind regards,
Geert Uytterhoeven
Software Architect
Sony Techsoft Centre
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium
Phone: +32 (0)2 700 8453
Fax: +32 (0)2 700 8622
E-mail: [email protected]
Internet: http://www.sony-europe.com/
Sony Technology and Software Centre Europe
A division of Sony Service Centre (Europe) N.V.
Registered office: Technologielaan 7 · B-1840 Londerzeel · Belgium
VAT BE 0413.825.160 · RPR Brussels
Fortis 293-0376800-10 GEBA-BE-BB
* Geert Uytterhoeven | 2008-06-26 09:14:51 [+0200]:
>On Wed, 25 Jun 2008, Sebastian Siewior wrote:
>> see. Other zlib users like ppp or jffs2 user zlib directly. However, the
>> crypto API is the only place in kernel which provides you a generic
>> interface for compression.
>
>Indeed. So if a hardware-assisted deflate would be available as a crypto
>module, "deflate" crypto module users would benefit from it automatically,
>right? While direct zlib users are out of luck.
Haven't look at this from that perspective but yes. This is allready the
case for crypto HW.
>With kind regards,
>
>Geert Uytterhoeven
>Software Architect
Sebastian
Geert Uytterhoeven <[email protected]> wrote:
>
> 2. Why was the lazy allocation removed back in 2004?
> If you're interested in decompression only, it's a bit wasteful to
> allocate 262 KiB of memory for compression and never use it.
The burden to save memory was simply moved to the user :)
As Sebastian correctly pointed out, the only user of deflate as
it is is IPComp (there were some talk about using it for file
systems but that never worked out).
As such we cannot sleep when operating on the data. Since we're
allocating more than a page worth of memory, it must be allocated
in a process context. That's why the lazy allocation was removed.
Having said that, the user is free to use lazy allocation if it
is in a sleepable context by simply deferring the allocation of
the tfm.
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
On Thu, 26 Jun 2008, Herbert Xu wrote:
> Geert Uytterhoeven <[email protected]> wrote:
> > 2. Why was the lazy allocation removed back in 2004?
> > If you're interested in decompression only, it's a bit wasteful to
> > allocate 262 KiB of memory for compression and never use it.
>
> The burden to save memory was simply moved to the user :)
>
> As Sebastian correctly pointed out, the only user of deflate as
> it is is IPComp (there were some talk about using it for file
> systems but that never worked out).
Typically file systems seem to use the `zlib data format', while
crypto/deflate.c uses the `raw deflate data format'.
So I guess they would need a new crypto/zlib.c module anyway.
> As such we cannot sleep when operating on the data. Since we're
> allocating more than a page worth of memory, it must be allocated
> in a process context. That's why the lazy allocation was removed.
Thanks, that's a good reason!
> Having said that, the user is free to use lazy allocation if it
> is in a sleepable context by simply deferring the allocation of
> the tfm.
But it also means it's not so easy to avoid allocating the compression buffer
if you need decompression only. Or am I missing something?
Thanks again!
With kind regards,
Geert Uytterhoeven
Software Architect
Sony Techsoft Centre
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium
Phone: +32 (0)2 700 8453
Fax: +32 (0)2 700 8622
E-mail: [email protected]
Internet: http://www.sony-europe.com/
Sony Technology and Software Centre Europe
A division of Sony Service Centre (Europe) N.V.
Registered office: Technologielaan 7 · B-1840 Londerzeel · Belgium
VAT BE 0413.825.160 · RPR Brussels
Fortis 293-0376800-10 GEBA-BE-BB
On Thu, Jun 26, 2008 at 06:07:29PM +0200, Geert Uytterhoeven wrote:
>
> But it also means it's not so easy to avoid allocating the compression buffer
> if you need decompression only. Or am I missing something?
Well there are ways to solve this, e.g., by creating new interfaces
that are compression or decompression only. What application do
you have in mind?
Cheers,
--
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmV>HI~} <[email protected]>
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
On Fri, 27 Jun 2008, Herbert Xu wrote:
> On Thu, Jun 26, 2008 at 06:07:29PM +0200, Geert Uytterhoeven wrote:
> > But it also means it's not so easy to avoid allocating the compression buffer
> > if you need decompression only. Or am I missing something?
>
> Well there are ways to solve this, e.g., by creating new interfaces
> that are compression or decompression only. What application do
> you have in mind?
Decompression for a read-only file system.
With kind regards,
Geert Uytterhoeven
Software Architect
Sony Techsoft Centre
The Corporate Village · Da Vincilaan 7-D1 · B-1935 Zaventem · Belgium
Phone: +32 (0)2 700 8453
Fax: +32 (0)2 700 8622
E-mail: [email protected]
Internet: http://www.sony-europe.com/
Sony Technology and Software Centre Europe
A division of Sony Service Centre (Europe) N.V.
Registered office: Technologielaan 7 · B-1840 Londerzeel · Belgium
VAT BE 0413.825.160 · RPR Brussels
Fortis 293-0376800-10 GEBA-BE-BB