2007-05-29 14:32:32

by Michael-Luke Jones

[permalink] [raw]
Subject: [-mm] Move zlib compression library to common directory

This patch moves the zlib compression and decompression code
to a common directory. The zutil.h header, which is not meant
to be public, is moved from include/linux/ to lib/zlib/.

In addition, the PRESET_DICT definition from zutil.h, used by
fs/jffs2/compr_zlib.c is moved to the 'true' public zlib header
include/linux/zlib.h. This move is documented in comments.

Conditional compile of only inflate or deflate code is
preserved by this patch.

Signed-off-by: Michael-Luke Jones <[email protected]>

diff --git a/fs/jffs2/compr_zlib.c b/fs/jffs2/compr_zlib.c
index 2b87fcc..e2cc6c7 100644
--- a/fs/jffs2/compr_zlib.c
+++ b/fs/jffs2/compr_zlib.c
@@ -16,7 +16,6 @@
#include <linux/kernel.h>
#include <linux/slab.h>
#include <linux/zlib.h>
-#include <linux/zutil.h>
#include "nodelist.h"
#include "compr.h"

diff --git a/include/linux/zlib.h b/include/linux/zlib.h
index 9e3192a..e9d32c5 100644
--- a/include/linux/zlib.h
+++ b/include/linux/zlib.h
@@ -177,6 +177,9 @@ typedef z_stream *z_streamp;
#define Z_DEFLATED 8
/* The deflate compression method (the only one supported in this
version) */

+#define PRESET_DICT 0x20 /* preset dictionary flag in zlib header */
+/* Required by fs/jffs2/compr_zlib.c */
+
/* basic functions */

extern int zlib_deflate_workspacesize (void);
diff --git a/lib/Makefile b/lib/Makefile
index c8c8e20..42c3903 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -46,8 +46,8 @@ obj-$(CONFIG_CRC32) += crc32.o
obj-$(CONFIG_LIBCRC32C) += libcrc32c.o
obj-$(CONFIG_GENERIC_ALLOCATOR) += genalloc.o

-obj-$(CONFIG_ZLIB_INFLATE) += zlib_inflate/
-obj-$(CONFIG_ZLIB_DEFLATE) += zlib_deflate/
+obj-$(CONFIG_ZLIB_INFLATE) += zlib/
+obj-$(CONFIG_ZLIB_DEFLATE) += zlib/
obj-$(CONFIG_REED_SOLOMON) += reed_solomon/

obj-$(CONFIG_TEXTSEARCH) += textsearch.o
diff --git a/lib/zlib_inflate/Makefile b/lib/zlib/Makefile
similarity index 71%
rename from lib/zlib_inflate/Makefile
rename to lib/zlib/Makefile
index bf06548..528d903 100644
--- a/lib/zlib_inflate/Makefile
+++ b/lib/zlib/Makefile
@@ -2,8 +2,8 @@
# This is a modified version of zlib, which does all memory
# allocation ahead of time.
#
-# This is only the decompression, see zlib_deflate for the
-# the compression
+# The compression (deflate) and decompression (inflate) code
+# is present in this folder.
#
# Decompression needs to be serialized for each memory
# allocation.
@@ -17,3 +17,7 @@ obj-$(CONFIG_ZLIB_INFLATE) += zlib_inflate.o

zlib_inflate-objs := inffast.o inflate.o \
inftrees.o inflate_syms.o
+
+obj-$(CONFIG_ZLIB_DEFLATE) += zlib_deflate.o
+
+zlib_deflate-objs := deflate.o deftree.o deflate_syms.o
diff --git a/lib/zlib_deflate/deflate.c b/lib/zlib/deflate.c
similarity index 100%
rename from lib/zlib_deflate/deflate.c
rename to lib/zlib/deflate.c
index c3e4a2b..1137780 100644
--- a/lib/zlib_deflate/deflate.c
+++ b/lib/zlib/deflate.c
@@ -49,10 +49,9 @@
*/

#include <linux/module.h>
-#include <linux/zutil.h>
+#include "zutil.h"
#include "defutil.h"

-
/*
===========================================================================
* Function prototypes.
*/
diff --git a/lib/zlib_deflate/deflate_syms.c b/lib/zlib/deflate_syms.c
similarity index 100%
rename from lib/zlib_deflate/deflate_syms.c
rename to lib/zlib/deflate_syms.c
diff --git a/lib/zlib_deflate/deftree.c b/lib/zlib/deftree.c
similarity index 100%
rename from lib/zlib_deflate/deftree.c
rename to lib/zlib/deftree.c
index ddf3482..d5ce87d 100644
--- a/lib/zlib_deflate/deftree.c
+++ b/lib/zlib/deftree.c
@@ -34,7 +34,7 @@

/* #include "deflate.h" */

-#include <linux/zutil.h>
+#include "zutil.h"
#include "defutil.h"

#ifdef DEBUG_ZLIB
diff --git a/lib/zlib_deflate/defutil.h b/lib/zlib/defutil.h
similarity index 100%
rename from lib/zlib_deflate/defutil.h
rename to lib/zlib/defutil.h
diff --git a/lib/zlib_inflate/inffast.c b/lib/zlib/inffast.c
similarity index 100%
rename from lib/zlib_inflate/inffast.c
rename to lib/zlib/inffast.c
index d84560c..50d3de1 100644
--- a/lib/zlib_inflate/inffast.c
+++ b/lib/zlib/inffast.c
@@ -3,7 +3,7 @@
* For conditions of distribution and use, see copyright notice in zlib.h
*/

-#include <linux/zutil.h>
+#include "zutil.h"
#include "inftrees.h"
#include "inflate.h"
#include "inffast.h"
diff --git a/lib/zlib_inflate/inffast.h b/lib/zlib/inffast.h
similarity index 100%
rename from lib/zlib_inflate/inffast.h
rename to lib/zlib/inffast.h
diff --git a/lib/zlib_inflate/inffixed.h b/lib/zlib/inffixed.h
similarity index 100%
rename from lib/zlib_inflate/inffixed.h
rename to lib/zlib/inffixed.h
diff --git a/lib/zlib_inflate/inflate.c b/lib/zlib/inflate.c
similarity index 100%
rename from lib/zlib_inflate/inflate.c
rename to lib/zlib/inflate.c
index 7e1e311..ba58871 100644
--- a/lib/zlib_inflate/inflate.c
+++ b/lib/zlib/inflate.c
@@ -9,7 +9,7 @@
*
*/

-#include <linux/zutil.h>
+#include "zutil.h"
#include "inftrees.h"
#include "inflate.h"
#include "inffast.h"
diff --git a/lib/zlib_inflate/inflate.h b/lib/zlib/inflate.h
similarity index 100%
rename from lib/zlib_inflate/inflate.h
rename to lib/zlib/inflate.h
diff --git a/lib/zlib_inflate/inflate_syms.c b/lib/zlib/inflate_syms.c
similarity index 100%
rename from lib/zlib_inflate/inflate_syms.c
rename to lib/zlib/inflate_syms.c
diff --git a/lib/zlib_inflate/inftrees.c b/lib/zlib/inftrees.c
similarity index 100%
rename from lib/zlib_inflate/inftrees.c
rename to lib/zlib/inftrees.c
index 3fe6ce5..faf6f75 100644
--- a/lib/zlib_inflate/inftrees.c
+++ b/lib/zlib/inftrees.c
@@ -3,7 +3,7 @@
* For conditions of distribution and use, see copyright notice in zlib.h
*/

-#include <linux/zutil.h>
+#include "zutil.h"
#include "inftrees.h"

#define MAXBITS 15
diff --git a/lib/zlib_inflate/inftrees.h b/lib/zlib/inftrees.h
similarity index 100%
rename from lib/zlib_inflate/inftrees.h
rename to lib/zlib/inftrees.h
diff --git a/lib/zlib_inflate/infutil.h b/lib/zlib/infutil.h
similarity index 100%
rename from lib/zlib_inflate/infutil.h
rename to lib/zlib/infutil.h
diff --git a/lib/zlib/patch-zlib b/lib/zlib/patch-zlib
new file mode 100644
index 0000000..d4a011b
Binary files /dev/null and b/lib/zlib/patch-zlib differ
diff --git a/include/linux/zutil.h b/lib/zlib/zutil.h
similarity index 98%
rename from include/linux/zutil.h
rename to lib/zlib/zutil.h
index 6adfa9a..0467c8c 100644
--- a/include/linux/zutil.h
+++ b/lib/zlib/zutil.h
@@ -32,7 +32,7 @@ typedef unsigned long ulg;
#define MAX_MATCH 258
/* The minimum and maximum match lengths */

-#define PRESET_DICT 0x20 /* preset dictionary flag in zlib header */
+/* PRESET_DICT #define moved to <linux/zlib.h> for JFFS2 */

/* target dependencies */

diff --git a/lib/zlib_deflate/Makefile b/lib/zlib_deflate/Makefile
deleted file mode 100644
index 86275e3..0000000
--- a/lib/zlib_deflate/Makefile
+++ /dev/null
@@ -1,11 +0,0 @@
-#
-# This is a modified version of zlib, which does all memory
-# allocation ahead of time.
-#
-# This is the compression code, see zlib_inflate for the
-# decompression code.
-#
-
-obj-$(CONFIG_ZLIB_DEFLATE) += zlib_deflate.o
-
-zlib_deflate-objs := deflate.o deftree.o deflate_syms.o


2007-05-29 16:31:29

by Michael-Luke Jones

[permalink] [raw]
Subject: [-mm] Move zlib compression library to common directory [TAKE TWO]

This patch moves the zlib compression and decompression code
to a common directory. The zutil.h header, which is not meant
to be public, is moved from include/linux/ to lib/zlib/.

In addition, the PRESET_DICT definition from zutil.h, used by
fs/jffs2/compr_zlib.c is moved to the 'true' public zlib
header include/linux/zlib.h.

Conditional compile of only inflate or deflate code is
preserved by this patch. With a bit of luck, this version
won't be whitespace-damaged. Grumble.

Signed-off-by: Michael-Luke Jones <[email protected]>

diff --git a/fs/jffs2/compr_zlib.c b/fs/jffs2/compr_zlib.c
index 2b87fcc..e2cc6c7 100644
--- a/fs/jffs2/compr_zlib.c
+++ b/fs/jffs2/compr_zlib.c
@@ -16,7 +16,6 @@
#include <linux/kernel.h>
#include <linux/slab.h>
#include <linux/zlib.h>
-#include <linux/zutil.h>
#include "nodelist.h"
#include "compr.h"

diff --git a/include/linux/zlib.h b/include/linux/zlib.h
index 9e3192a..e9d32c5 100644
--- a/include/linux/zlib.h
+++ b/include/linux/zlib.h
@@ -177,6 +177,9 @@ typedef z_stream *z_streamp;
#define Z_DEFLATED 8
/* The deflate compression method (the only one supported in this version) */

+#define PRESET_DICT 0x20 /* preset dictionary flag in zlib header */
+/* Required by fs/jffs2/compr_zlib.c */
+
/* basic functions */

extern int zlib_deflate_workspacesize (void);
diff --git a/lib/Makefile b/lib/Makefile
index c8c8e20..42c3903 100644
--- a/lib/Makefile
+++ b/lib/Makefile
@@ -46,8 +46,8 @@ obj-$(CONFIG_CRC32) += crc32.o
obj-$(CONFIG_LIBCRC32C) += libcrc32c.o
obj-$(CONFIG_GENERIC_ALLOCATOR) += genalloc.o

-obj-$(CONFIG_ZLIB_INFLATE) += zlib_inflate/
-obj-$(CONFIG_ZLIB_DEFLATE) += zlib_deflate/
+obj-$(CONFIG_ZLIB_INFLATE) += zlib/
+obj-$(CONFIG_ZLIB_DEFLATE) += zlib/
obj-$(CONFIG_REED_SOLOMON) += reed_solomon/

obj-$(CONFIG_TEXTSEARCH) += textsearch.o
diff --git a/lib/zlib_inflate/Makefile b/lib/zlib/Makefile
similarity index 71%
rename from lib/zlib_inflate/Makefile
rename to lib/zlib/Makefile
index bf06548..528d903 100644
--- a/lib/zlib_inflate/Makefile
+++ b/lib/zlib/Makefile
@@ -2,8 +2,8 @@
# This is a modified version of zlib, which does all memory
# allocation ahead of time.
#
-# This is only the decompression, see zlib_deflate for the
-# the compression
+# The compression (deflate) and decompression (inflate) code
+# is present in this folder.
#
# Decompression needs to be serialized for each memory
# allocation.
@@ -17,3 +17,7 @@ obj-$(CONFIG_ZLIB_INFLATE) += zlib_inflate.o

zlib_inflate-objs := inffast.o inflate.o \
inftrees.o inflate_syms.o
+
+obj-$(CONFIG_ZLIB_DEFLATE) += zlib_deflate.o
+
+zlib_deflate-objs := deflate.o deftree.o deflate_syms.o
diff --git a/lib/zlib_deflate/deflate.c b/lib/zlib/deflate.c
similarity index 100%
rename from lib/zlib_deflate/deflate.c
rename to lib/zlib/deflate.c
index c3e4a2b..1137780 100644
--- a/lib/zlib_deflate/deflate.c
+++ b/lib/zlib/deflate.c
@@ -49,10 +49,9 @@
*/

#include <linux/module.h>
-#include <linux/zutil.h>
+#include "zutil.h"
#include "defutil.h"

-
/* ===========================================================================
* Function prototypes.
*/
diff --git a/lib/zlib_deflate/deflate_syms.c b/lib/zlib/deflate_syms.c
similarity index 100%
rename from lib/zlib_deflate/deflate_syms.c
rename to lib/zlib/deflate_syms.c
diff --git a/lib/zlib_deflate/deftree.c b/lib/zlib/deftree.c
similarity index 100%
rename from lib/zlib_deflate/deftree.c
rename to lib/zlib/deftree.c
index ddf3482..d5ce87d 100644
--- a/lib/zlib_deflate/deftree.c
+++ b/lib/zlib/deftree.c
@@ -34,7 +34,7 @@

/* #include "deflate.h" */

-#include <linux/zutil.h>
+#include "zutil.h"
#include "defutil.h"

#ifdef DEBUG_ZLIB
diff --git a/lib/zlib_deflate/defutil.h b/lib/zlib/defutil.h
similarity index 100%
rename from lib/zlib_deflate/defutil.h
rename to lib/zlib/defutil.h
diff --git a/lib/zlib_inflate/inffast.c b/lib/zlib/inffast.c
similarity index 100%
rename from lib/zlib_inflate/inffast.c
rename to lib/zlib/inffast.c
index d84560c..50d3de1 100644
--- a/lib/zlib_inflate/inffast.c
+++ b/lib/zlib/inffast.c
@@ -3,7 +3,7 @@
* For conditions of distribution and use, see copyright notice in zlib.h
*/

-#include <linux/zutil.h>
+#include "zutil.h"
#include "inftrees.h"
#include "inflate.h"
#include "inffast.h"
diff --git a/lib/zlib_inflate/inffast.h b/lib/zlib/inffast.h
similarity index 100%
rename from lib/zlib_inflate/inffast.h
rename to lib/zlib/inffast.h
diff --git a/lib/zlib_inflate/inffixed.h b/lib/zlib/inffixed.h
similarity index 100%
rename from lib/zlib_inflate/inffixed.h
rename to lib/zlib/inffixed.h
diff --git a/lib/zlib_inflate/inflate.c b/lib/zlib/inflate.c
similarity index 100%
rename from lib/zlib_inflate/inflate.c
rename to lib/zlib/inflate.c
index 7e1e311..ba58871 100644
--- a/lib/zlib_inflate/inflate.c
+++ b/lib/zlib/inflate.c
@@ -9,7 +9,7 @@
*
*/

-#include <linux/zutil.h>
+#include "zutil.h"
#include "inftrees.h"
#include "inflate.h"
#include "inffast.h"
diff --git a/lib/zlib_inflate/inflate.h b/lib/zlib/inflate.h
similarity index 100%
rename from lib/zlib_inflate/inflate.h
rename to lib/zlib/inflate.h
diff --git a/lib/zlib_inflate/inflate_syms.c b/lib/zlib/inflate_syms.c
similarity index 100%
rename from lib/zlib_inflate/inflate_syms.c
rename to lib/zlib/inflate_syms.c
diff --git a/lib/zlib_inflate/inftrees.c b/lib/zlib/inftrees.c
similarity index 100%
rename from lib/zlib_inflate/inftrees.c
rename to lib/zlib/inftrees.c
index 3fe6ce5..faf6f75 100644
--- a/lib/zlib_inflate/inftrees.c
+++ b/lib/zlib/inftrees.c
@@ -3,7 +3,7 @@
* For conditions of distribution and use, see copyright notice in zlib.h
*/

-#include <linux/zutil.h>
+#include "zutil.h"
#include "inftrees.h"

#define MAXBITS 15
diff --git a/lib/zlib_inflate/inftrees.h b/lib/zlib/inftrees.h
similarity index 100%
rename from lib/zlib_inflate/inftrees.h
rename to lib/zlib/inftrees.h
diff --git a/lib/zlib_inflate/infutil.h b/lib/zlib/infutil.h
similarity index 100%
rename from lib/zlib_inflate/infutil.h
rename to lib/zlib/infutil.h
diff --git a/include/linux/zutil.h b/lib/zlib/zutil.h
similarity index 98%
rename from include/linux/zutil.h
rename to lib/zlib/zutil.h
index 6adfa9a..0467c8c 100644
--- a/include/linux/zutil.h
+++ b/lib/zlib/zutil.h
@@ -32,7 +32,7 @@ typedef unsigned long ulg;
#define MAX_MATCH 258
/* The minimum and maximum match lengths */

-#define PRESET_DICT 0x20 /* preset dictionary flag in zlib header */
+/* PRESET_DICT #define moved to <linux/zlib.h> for JFFS2 */

/* target dependencies */

diff --git a/lib/zlib_deflate/Makefile b/lib/zlib_deflate/Makefile
deleted file mode 100644
index 86275e3..0000000
--- a/lib/zlib_deflate/Makefile
+++ /dev/null
@@ -1,11 +0,0 @@
-#
-# This is a modified version of zlib, which does all memory
-# allocation ahead of time.
-#
-# This is the compression code, see zlib_inflate for the
-# decompression code.
-#
-
-obj-$(CONFIG_ZLIB_DEFLATE) += zlib_deflate.o
-
-zlib_deflate-objs := deflate.o deftree.o deflate_syms.o


Attachments:
zlib-move.patch (6.87 kB)

2007-05-30 17:48:14

by Andrew Morton

[permalink] [raw]
Subject: Re: [-mm] Move zlib compression library to common directory [TAKE TWO]

On Tue, 29 May 2007 17:31:05 +0100
Michael-Luke Jones <[email protected]> wrote:

> This patch moves the zlib compression and decompression code
> to a common directory. The zutil.h header, which is not meant
> to be public, is moved from include/linux/ to lib/zlib/.
>
> In addition, the PRESET_DICT definition from zutil.h, used by
> fs/jffs2/compr_zlib.c is moved to the 'true' public zlib
> header include/linux/zlib.h.
>
> Conditional compile of only inflate or deflate code is
> preserved by this patch. With a bit of luck, this version
> won't be whitespace-damaged. Grumble.

umm, OK, if you say so. I'm not sure I understand the point behind
all this, but I'm sure there is one ;)

> diff --git a/lib/zlib_deflate/deflate.c b/lib/zlib/deflate.c
> similarity index 100%
> rename from lib/zlib_deflate/deflate.c
> rename to lib/zlib/deflate.c
> index c3e4a2b..1137780 100644
> diff --git a/lib/zlib_deflate/deflate_syms.c b/lib/zlib/deflate_syms.c
> similarity index 100%
> rename from lib/zlib_deflate/deflate_syms.c
> rename to lib/zlib/deflate_syms.c
> diff --git a/lib/zlib_deflate/deftree.c b/lib/zlib/deftree.c
> similarity index 100%
> rename from lib/zlib_deflate/deftree.c
> rename to lib/zlib/deftree.c
> index ddf3482..d5ce87d 100644
> diff --git a/lib/zlib_deflate/defutil.h b/lib/zlib/defutil.h
> similarity index 100%
> rename from lib/zlib_deflate/defutil.h
> rename to lib/zlib/defutil.h
> diff --git a/lib/zlib_inflate/inffast.c b/lib/zlib/inffast.c
> similarity index 100%
> rename from lib/zlib_inflate/inffast.c
> rename to lib/zlib/inffast.c
> diff --git a/lib/zlib_inflate/inftrees.h b/lib/zlib/inftrees.h
> similarity index 100%
> rename from lib/zlib_inflate/inftrees.h
> rename to lib/zlib/inftrees.h
> diff --git a/lib/zlib_inflate/infutil.h b/lib/zlib/infutil.h
> similarity index 100%
> rename from lib/zlib_inflate/infutil.h
> rename to lib/zlib/infutil.h
> diff --git a/include/linux/zutil.h b/lib/zlib/zutil.h
> similarity index 98%
> rename from include/linux/zutil.h
> rename to lib/zlib/zutil.h
> index 6adfa9a..0467c8c 100644

Lots of git goodies there - I didn't actually receive the patch: the parts
which move files around were communicated via git metadata, not diffs.
Presumably there's a way to tell git to not do that.

But probably it's best that this change be merged via git anyway. David,
are you interested in handling it?

2007-05-30 18:05:31

by Satyam Sharma

[permalink] [raw]
Subject: Re: [-mm] Move zlib compression library to common directory [TAKE TWO]

Hi Michael,

On 5/29/07, Michael-Luke Jones <[email protected]> wrote:
>
>
> This patch moves the zlib compression and decompression code
> to a common directory. The zutil.h header, which is not meant
> to be public, is moved from include/linux/ to lib/zlib/.

You could remove the (now pointless) /* WARNING: ... */ comment
from (the now truly private) zutil.h too ...

> In addition, the PRESET_DICT definition from zutil.h, used by
> fs/jffs2/compr_zlib.c is moved to the 'true' public zlib
> header include/linux/zlib.h.

Ugh, I wish you had held on from this patch till the original thread
discussing this reached some conclusion ... from Mark's response
there it does seem PRESET_DICT is clearly an implementation
(and not interface) detail -- which means to me that it must continue
to live in zutil.h (the private header) ... but then, this change is
clearly not so critical either, so it doesn't matter much, I guess.

[ Added zlib authors Mark Adler and Jean-Loup Gailly on this
thread too. ]

Satyam

2007-05-30 19:20:50

by Michael-Luke Jones

[permalink] [raw]
Subject: Re: [-mm] Move zlib compression library to common directory [TAKE TWO]

Satyam Sharma wrote:
> Ugh, I wish you had held on from this patch till the original thread
> discussing this reached some conclusion ... from Mark's response
> there it does seem PRESET_DICT is clearly an implementation
> (and not interface) detail -- which means to me that it must continue
> to live in zutil.h (the private header) ... but then, this change is
> clearly not so critical either, so it doesn't matter much, I guess.

Satyam: Sorry about this, I wrote this before the conversation on this
point started, expecting the discussion to happen here. My hope was to
see if this minimal change to export PRESET_DICT was acceptable (if not
ideal). Given that practically all of the implementation details are
currently exported, it's probably an improvement to only export one :)

Andrew: The main reason for this change is to allow a private header to
be moved from include/linux/ to this private folder. It's already been
used apparently inappropriately by JFFS2 compression code. Minimal code
changes, lots of file moving (as I'm sure you can see)...

I managed to get the patch to apply to a fresh tree using 'git apply',
which may well mean that I didn't understand your comment that "the
parts which move files around were communicated via git metadata, not
diffs".

Given that, since I wrote this patch, no-one has actually said anything
positive about it, there's probably not much point picking it up until
people have worked out exactly what to do with JFFS2's inappropriate use
of the public-private zutil.h header. At least it's just JFFS2 :)

> [ Added zlib authors Mark Adler and Jean-Loup Gailly on this
> thread too. ]

Thanks, I should have CCed the authors/maintainers originally.

> Satyam

Michael-Luke

(PS: The stern warning is also present in the other private zlib header
files.)

2007-05-30 20:15:44

by Michael-Luke Jones

[permalink] [raw]
Subject: Re: [-mm] Move zlib compression library to common directory [TAKE TWO]

Andrew Morton wrote:
> Lots of git goodies there - I didn't actually receive the patch: the parts
> which move files around were communicated via git metadata, not diffs.
> Presumably there's a way to tell git to not do that.
>
> But probably it's best that this change be merged via git anyway. David,
> are you interested in handling it?

Update: totally ignore this patch. Even using git-apply something gets
confused and git loses the meaning of the renames, turning them into
file additions and deletions.

A better version will no doubt come along. In the mean time, it looks
like I need to improve my git-foo skills...

M-L