2013-10-24 15:15:28

by Konstantin Tokarev

[permalink] [raw]
Subject: Re: lz4hc compression in UBIFS?



24.10.2013, 18:20, "Konstantin Tokarev" <[email protected]>:
> 23.10.2013, 22:26, "Yann Collet" <[email protected]>:
>
>> ?UBIFS error (pid 4288): ubifs_decompress: cannot decompress 12 bytes,
>> ?(...)
>> ?????????data size ?????12
>> ?????????data:
>> ?????????00000000: 1f 00 01 00 ff e8 50 00 00 00 00 00
>>
>> ?The compressed format is correct.
>> ?It describes a flow of 00, of size ~500.
>>
>> ?So the problem seems more likely on the decompression side.
>>
>> ?Are you sure you are providing "12" as the "input size" parameter ? and that
>> ?the "maximum output size" parameter is correctly provided ? (i.e. >= to
>> ?original data size)
>
> Decompression code in kernel[1] is heavily modified. In particular, lz4_uncompress
> function (used in this case) does not have input size parameter at all,
> while it's present in lz4_uncompress_unknownoutputsize.
>
> [1] lib/lz4/lz4_decompress.c

Attached patch to crypto API wrapper for lz4hc seems to fix the issue. I can save
and read files on LZ4HC-compressed volume, and I'm running on LZ4HC-compressed rootfs,
flashed from mkfs.ubifs generated image (patch by Elie De Brauwer). My software
works correctly.

Unfortunately, on my board LZ4HC-compressed UBIFS volume performs noticeable worse
than LZO, while still faster than zlib. I guess the reason is that CPU is no longer a
bottleneck for my system, but NAND speed.

Thank you all for your help!

--
Regards,
Konstantin


Attachments:
crypto_lz4.patch (852.00 B)

2013-10-28 16:28:03

by Konstantin Tokarev

[permalink] [raw]
Subject: Re: lz4hc compression in UBIFS?



24.10.2013, 19:15, "Konstantin Tokarev" <[email protected]>:
> Attached patch to crypto API wrapper for lz4hc seems to fix the issue. I can save
> and read files on LZ4HC-compressed volume, and I'm running on LZ4HC-compressed rootfs,
> flashed from mkfs.ubifs generated image (patch by Elie De Brauwer). My software
> works correctly.
>
> Unfortunately, on my board LZ4HC-compressed UBIFS volume performs noticeable worse
> than LZO, while still faster than zlib. I guess the reason is that CPU is no longer a
> bottleneck for my system, but NAND speed.
>
> Thank you all for your help!

Can anyone review the correctness of my patch? Looks like API of LZ4 decompressor is
used wrong way in crypto API.

--
Regards,
Konstantin

2013-10-28 16:46:09

by Florian Fainelli

[permalink] [raw]
Subject: Re: lz4hc compression in UBIFS?

2013/10/28 Konstantin Tokarev <[email protected]>:
>
>
> 24.10.2013, 19:15, "Konstantin Tokarev" <[email protected]>:
>> Attached patch to crypto API wrapper for lz4hc seems to fix the issue. I can save
>> and read files on LZ4HC-compressed volume, and I'm running on LZ4HC-compressed rootfs,
>> flashed from mkfs.ubifs generated image (patch by Elie De Brauwer). My software
>> works correctly.
>>
>> Unfortunately, on my board LZ4HC-compressed UBIFS volume performs noticeable worse
>> than LZO, while still faster than zlib. I guess the reason is that CPU is no longer a
>> bottleneck for my system, but NAND speed.
>>
>> Thank you all for your help!
>
> Can anyone review the correctness of my patch? Looks like API of LZ4 decompressor is
> used wrong way in crypto API.

Can you make a formal submission of it? That would probably help reviewing it.
--
Florian