Changelog:
v5 -> v6:
* shove variables in debug.c and in debug.h just have an extern, spotted by Konrad
* update patch description, spotted by Konrad
v4 -> v5:
* fix compile error, reported by Fengguang, Geert
* add check for !is_ephemeral(pool), spotted by Bob
v3 -> v4:
* handle duplication in page_is_zero_filled, spotted by Bob
* fix zcache writeback in dubugfs
* fix pers_pageframes|_max isn't exported in debugfs
* fix static variable defined in debug.h but used in multiple C files
* rebase on Greg's staging-next
v2 -> v3:
* increment/decrement zcache_[eph|pers]_zpages for zero-filled pages, spotted by Dan
* replace "zero" or "zero page" by "zero_filled_page", spotted by Dan
v1 -> v2:
* avoid changing tmem.[ch] entirely, spotted by Dan.
* don't accumulate [eph|pers]pageframe and [eph|pers]zpages for
zero-filled pages, spotted by Dan
* cleanup TODO list
* add Dan Acked-by.
Motivation:
- Seth Jennings points out compress zero-filled pages with LZO(a lossless
data compression algorithm) will waste memory and result in fragmentation.
https://lkml.org/lkml/2012/8/14/347
- Dan Magenheimer add "Support zero-filled pages more efficiently" feature
in zcache TODO list https://lkml.org/lkml/2013/2/13/503
Design:
- For store page, capture zero-filled pages(evicted clean page cache pages and
swap pages), but don't compress them, set pampd which store zpage address to
0x2(since 0x0 and 0x1 has already been ocuppied) to mark special zero-filled
case and take advantage of tmem infrastructure to transform handle-tuple(pool
id, object id, and an index) to a pampd. Twice compress zero-filled pages will
contribute to one zcache_[eph|pers]_pageframes count accumulated.
- For load page, traverse tmem hierachical to transform handle-tuple to pampd
and identify zero-filled case by pampd equal to 0x2 when filesystem reads
file pages or a page needs to be swapped in, then refill the page to zero
and return.
Test:
dd if=/dev/zero of=zerofile bs=1MB count=500
vmtouch -t zerofile
vmtouch -e zerofile
formula:
- fragmentation level = (zcache_[eph|pers]_pageframes * PAGE_SIZE - zcache_[eph|pers]_zbytes)
* 100 / (zcache_[eph|pers]_pageframes * PAGE_SIZE)
- memory zcache occupy = zcache_[eph|pers]_zbytes
Result:
without zero-filled awareness:
- fragmentation level: 98%
- memory zcache occupy: 238MB
with zero-filled awareness:
- fragmentation level: 0%
- memory zcache occupy: 0MB
Wanpeng Li (3):
staging: zcache: fix static variables defined in debug.h but used in
mutiple C files
staging: zcache: introduce zero-filled page stat count
staging: zcache: clean TODO list
drivers/staging/zcache/TODO | 3 +-
drivers/staging/zcache/debug.c | 35 +++++++++++++++
drivers/staging/zcache/debug.h | 79 ++++++++++++++++++++-------------
drivers/staging/zcache/zcache-main.c | 4 ++
4 files changed, 88 insertions(+), 33 deletions(-)
--
1.7.5.4
After commit 95bdaee214 ("zcache: Move debugfs code out of zcache-main.c file")
be merged, most of knods in zcache debugfs just export zero since these variables
are defined in debug.h but are in use in multiple C files zcache-main.c and debug.c,
in this case variables can't be treated as shared variables.
Signed-off-by: Wanpeng Li <[email protected]>
---
drivers/staging/zcache/debug.c | 32 ++++++++++++++++++++
drivers/staging/zcache/debug.h | 62 ++++++++++++++++++++--------------------
2 files changed, 63 insertions(+), 31 deletions(-)
diff --git a/drivers/staging/zcache/debug.c b/drivers/staging/zcache/debug.c
index d2d1fdf..faab2a9 100644
--- a/drivers/staging/zcache/debug.c
+++ b/drivers/staging/zcache/debug.c
@@ -4,6 +4,38 @@
#ifdef CONFIG_ZCACHE_DEBUG
#include <linux/debugfs.h>
+ssize_t zcache_obj_count;
+ssize_t zcache_obj_count_max;
+ssize_t zcache_objnode_count;
+ssize_t zcache_objnode_count_max;
+u64 zcache_eph_zbytes;
+u64 zcache_eph_zbytes_max;
+u64 zcache_pers_zbytes_max;
+ssize_t zcache_eph_pageframes_max;
+ssize_t zcache_pers_pageframes_max;
+ssize_t zcache_pageframes_alloced;
+ssize_t zcache_pageframes_freed;
+ssize_t zcache_eph_zpages;
+ssize_t zcache_eph_zpages_max;
+ssize_t zcache_pers_zpages_max;
+ssize_t zcache_flush_total;
+ssize_t zcache_flush_found;
+ssize_t zcache_flobj_total;
+ssize_t zcache_flobj_found;
+ssize_t zcache_failed_eph_puts;
+ssize_t zcache_failed_pers_puts;
+ssize_t zcache_failed_getfreepages;
+ssize_t zcache_failed_alloc;
+ssize_t zcache_put_to_flush;
+ssize_t zcache_compress_poor;
+ssize_t zcache_mean_compress_poor;
+ssize_t zcache_eph_ate_tail;
+ssize_t zcache_eph_ate_tail_failed;
+ssize_t zcache_pers_ate_eph;
+ssize_t zcache_pers_ate_eph_failed;
+ssize_t zcache_evicted_eph_zpages;
+ssize_t zcache_evicted_eph_pageframes;
+
#define ATTR(x) { .name = #x, .val = &zcache_##x, }
static struct debug_entry {
const char *name;
diff --git a/drivers/staging/zcache/debug.h b/drivers/staging/zcache/debug.h
index 4bbe49b..8ec82d4 100644
--- a/drivers/staging/zcache/debug.h
+++ b/drivers/staging/zcache/debug.h
@@ -3,9 +3,9 @@
#ifdef CONFIG_ZCACHE_DEBUG
/* we try to keep these statistics SMP-consistent */
-static ssize_t zcache_obj_count;
+extern ssize_t zcache_obj_count;
static atomic_t zcache_obj_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_obj_count_max;
+extern ssize_t zcache_obj_count_max;
static inline void inc_zcache_obj_count(void)
{
zcache_obj_count = atomic_inc_return(&zcache_obj_atomic);
@@ -17,9 +17,9 @@ static inline void dec_zcache_obj_count(void)
zcache_obj_count = atomic_dec_return(&zcache_obj_atomic);
BUG_ON(zcache_obj_count < 0);
};
-static ssize_t zcache_objnode_count;
+extern ssize_t zcache_objnode_count;
static atomic_t zcache_objnode_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_objnode_count_max;
+extern ssize_t zcache_objnode_count_max;
static inline void inc_zcache_objnode_count(void)
{
zcache_objnode_count = atomic_inc_return(&zcache_objnode_atomic);
@@ -31,9 +31,9 @@ static inline void dec_zcache_objnode_count(void)
zcache_objnode_count = atomic_dec_return(&zcache_objnode_atomic);
BUG_ON(zcache_objnode_count < 0);
};
-static u64 zcache_eph_zbytes;
+extern u64 zcache_eph_zbytes;
static atomic_long_t zcache_eph_zbytes_atomic = ATOMIC_INIT(0);
-static u64 zcache_eph_zbytes_max;
+extern u64 zcache_eph_zbytes_max;
static inline void inc_zcache_eph_zbytes(unsigned clen)
{
zcache_eph_zbytes = atomic_long_add_return(clen, &zcache_eph_zbytes_atomic);
@@ -46,7 +46,7 @@ static inline void dec_zcache_eph_zbytes(unsigned zsize)
};
extern u64 zcache_pers_zbytes;
static atomic_long_t zcache_pers_zbytes_atomic = ATOMIC_INIT(0);
-static u64 zcache_pers_zbytes_max;
+extern u64 zcache_pers_zbytes_max;
static inline void inc_zcache_pers_zbytes(unsigned clen)
{
zcache_pers_zbytes = atomic_long_add_return(clen, &zcache_pers_zbytes_atomic);
@@ -59,7 +59,7 @@ static inline void dec_zcache_pers_zbytes(unsigned zsize)
}
extern ssize_t zcache_eph_pageframes;
static atomic_t zcache_eph_pageframes_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_eph_pageframes_max;
+extern ssize_t zcache_eph_pageframes_max;
static inline void inc_zcache_eph_pageframes(void)
{
zcache_eph_pageframes = atomic_inc_return(&zcache_eph_pageframes_atomic);
@@ -72,7 +72,7 @@ static inline void dec_zcache_eph_pageframes(void)
};
extern ssize_t zcache_pers_pageframes;
static atomic_t zcache_pers_pageframes_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_pers_pageframes_max;
+extern ssize_t zcache_pers_pageframes_max;
static inline void inc_zcache_pers_pageframes(void)
{
zcache_pers_pageframes = atomic_inc_return(&zcache_pers_pageframes_atomic);
@@ -83,21 +83,21 @@ static inline void dec_zcache_pers_pageframes(void)
{
zcache_pers_pageframes = atomic_dec_return(&zcache_pers_pageframes_atomic);
}
-static ssize_t zcache_pageframes_alloced;
+extern ssize_t zcache_pageframes_alloced;
static atomic_t zcache_pageframes_alloced_atomic = ATOMIC_INIT(0);
static inline void inc_zcache_pageframes_alloced(void)
{
zcache_pageframes_alloced = atomic_inc_return(&zcache_pageframes_alloced_atomic);
};
-static ssize_t zcache_pageframes_freed;
+extern ssize_t zcache_pageframes_freed;
static atomic_t zcache_pageframes_freed_atomic = ATOMIC_INIT(0);
static inline void inc_zcache_pageframes_freed(void)
{
zcache_pageframes_freed = atomic_inc_return(&zcache_pageframes_freed_atomic);
}
-static ssize_t zcache_eph_zpages;
+extern ssize_t zcache_eph_zpages;
static atomic_t zcache_eph_zpages_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_eph_zpages_max;
+extern ssize_t zcache_eph_zpages_max;
static inline void inc_zcache_eph_zpages(void)
{
zcache_eph_zpages = atomic_inc_return(&zcache_eph_zpages_atomic);
@@ -110,7 +110,7 @@ static inline void dec_zcache_eph_zpages(unsigned zpages)
}
extern ssize_t zcache_pers_zpages;
static atomic_t zcache_pers_zpages_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_pers_zpages_max;
+extern ssize_t zcache_pers_zpages_max;
static inline void inc_zcache_pers_zpages(void)
{
zcache_pers_zpages = atomic_inc_return(&zcache_pers_zpages_atomic);
@@ -130,23 +130,23 @@ static inline unsigned long curr_pageframes_count(void)
atomic_read(&zcache_pers_pageframes_atomic);
};
/* but for the rest of these, counting races are ok */
-static ssize_t zcache_flush_total;
-static ssize_t zcache_flush_found;
-static ssize_t zcache_flobj_total;
-static ssize_t zcache_flobj_found;
-static ssize_t zcache_failed_eph_puts;
-static ssize_t zcache_failed_pers_puts;
-static ssize_t zcache_failed_getfreepages;
-static ssize_t zcache_failed_alloc;
-static ssize_t zcache_put_to_flush;
-static ssize_t zcache_compress_poor;
-static ssize_t zcache_mean_compress_poor;
-static ssize_t zcache_eph_ate_tail;
-static ssize_t zcache_eph_ate_tail_failed;
-static ssize_t zcache_pers_ate_eph;
-static ssize_t zcache_pers_ate_eph_failed;
-static ssize_t zcache_evicted_eph_zpages;
-static ssize_t zcache_evicted_eph_pageframes;
+extern ssize_t zcache_flush_total;
+extern ssize_t zcache_flush_found;
+extern ssize_t zcache_flobj_total;
+extern ssize_t zcache_flobj_found;
+extern ssize_t zcache_failed_eph_puts;
+extern ssize_t zcache_failed_pers_puts;
+extern ssize_t zcache_failed_getfreepages;
+extern ssize_t zcache_failed_alloc;
+extern ssize_t zcache_put_to_flush;
+extern ssize_t zcache_compress_poor;
+extern ssize_t zcache_mean_compress_poor;
+extern ssize_t zcache_eph_ate_tail;
+extern ssize_t zcache_eph_ate_tail_failed;
+extern ssize_t zcache_pers_ate_eph;
+extern ssize_t zcache_pers_ate_eph_failed;
+extern ssize_t zcache_evicted_eph_zpages;
+extern ssize_t zcache_evicted_eph_pageframes;
extern ssize_t zcache_last_active_file_pageframes;
extern ssize_t zcache_last_inactive_file_pageframes;
--
1.7.5.4
Cleanup TODO list since support zero-filled pages more efficiently has
already done by this patchset.
Acked-by: Dan Magenheimer <[email protected]>
Signed-off-by: Wanpeng Li <[email protected]>
---
drivers/staging/zcache/TODO | 3 +--
1 files changed, 1 insertions(+), 2 deletions(-)
diff --git a/drivers/staging/zcache/TODO b/drivers/staging/zcache/TODO
index ec9aa11..d0c18fa 100644
--- a/drivers/staging/zcache/TODO
+++ b/drivers/staging/zcache/TODO
@@ -61,5 +61,4 @@ ZCACHE FUTURE NEW FUNCTIONALITY
A. Support zsmalloc as an alternative high-density allocator
(See https://lkml.org/lkml/2013/1/23/511)
-B. Support zero-filled pages more efficiently
-C. Possibly support three zbuds per pageframe when space allows
+B. Possibly support three zbuds per pageframe when space allows
--
1.7.5.4
Introduce zero-filled page statistics to monitor the number of
zero-filled pages.
Acked-by: Dan Magenheimer <[email protected]>
Signed-off-by: Wanpeng Li <[email protected]>
---
drivers/staging/zcache/debug.c | 3 +++
drivers/staging/zcache/debug.h | 17 +++++++++++++++++
drivers/staging/zcache/zcache-main.c | 4 ++++
3 files changed, 24 insertions(+), 0 deletions(-)
diff --git a/drivers/staging/zcache/debug.c b/drivers/staging/zcache/debug.c
index faab2a9..daa2691 100644
--- a/drivers/staging/zcache/debug.c
+++ b/drivers/staging/zcache/debug.c
@@ -35,6 +35,8 @@ ssize_t zcache_pers_ate_eph;
ssize_t zcache_pers_ate_eph_failed;
ssize_t zcache_evicted_eph_zpages;
ssize_t zcache_evicted_eph_pageframes;
+ssize_t zcache_zero_filled_pages;
+ssize_t zcache_zero_filled_pages_max;
#define ATTR(x) { .name = #x, .val = &zcache_##x, }
static struct debug_entry {
@@ -62,6 +64,7 @@ static struct debug_entry {
ATTR(last_inactive_anon_pageframes),
ATTR(eph_nonactive_puts_ignored),
ATTR(pers_nonactive_puts_ignored),
+ ATTR(zero_filled_pages),
#ifdef CONFIG_ZCACHE_WRITEBACK
ATTR(outstanding_writeback_pages),
ATTR(writtenback_pages),
diff --git a/drivers/staging/zcache/debug.h b/drivers/staging/zcache/debug.h
index 8ec82d4..ddad92f 100644
--- a/drivers/staging/zcache/debug.h
+++ b/drivers/staging/zcache/debug.h
@@ -122,6 +122,21 @@ static inline void dec_zcache_pers_zpages(unsigned zpages)
zcache_pers_zpages = atomic_sub_return(zpages, &zcache_pers_zpages_atomic);
}
+extern ssize_t zcache_zero_filled_pages;
+static atomic_t zcache_zero_filled_pages_atomic = ATOMIC_INIT(0);
+extern ssize_t zcache_zero_filled_pages_max;
+static inline void inc_zcache_zero_filled_pages(void)
+{
+ zcache_zero_filled_pages = atomic_inc_return(
+ &zcache_zero_filled_pages_atomic);
+ if (zcache_zero_filled_pages > zcache_zero_filled_pages_max)
+ zcache_zero_filled_pages_max = zcache_zero_filled_pages;
+}
+static inline void dec_zcache_zero_filled_pages(void)
+{
+ zcache_zero_filled_pages = atomic_dec_return(
+ &zcache_zero_filled_pages_atomic);
+}
static inline unsigned long curr_pageframes_count(void)
{
return zcache_pageframes_alloced -
@@ -200,6 +215,8 @@ static inline void inc_zcache_eph_zpages(void) { };
static inline void dec_zcache_eph_zpages(unsigned zpages) { };
static inline void inc_zcache_pers_zpages(void) { };
static inline void dec_zcache_pers_zpages(unsigned zpages) { };
+static inline void inc_zcache_zero_filled_pages(void) { };
+static inline void dec_zcache_zero_filled_pages(void) { };
static inline unsigned long curr_pageframes_count(void)
{
return 0;
diff --git a/drivers/staging/zcache/zcache-main.c b/drivers/staging/zcache/zcache-main.c
index 1994cab..f3de76d 100644
--- a/drivers/staging/zcache/zcache-main.c
+++ b/drivers/staging/zcache/zcache-main.c
@@ -374,6 +374,7 @@ static void *zcache_pampd_eph_create(char *data, size_t size, bool raw,
if (page_is_zero_filled(page)) {
clen = 0;
zero_filled = true;
+ inc_zcache_zero_filled_pages();
goto got_pampd;
}
@@ -440,6 +441,7 @@ static void *zcache_pampd_pers_create(char *data, size_t size, bool raw,
if (page_is_zero_filled(page)) {
clen = 0;
zero_filled = true;
+ inc_zcache_zero_filled_pages();
goto got_pampd;
}
@@ -652,6 +654,7 @@ static int zcache_pampd_get_data_and_free(char *data, size_t *sizep, bool raw,
zpages = 1;
if (!raw)
*sizep = PAGE_SIZE;
+ dec_zcache_zero_filled_pages();
goto zero_fill;
}
@@ -702,6 +705,7 @@ static void zcache_pampd_free(void *pampd, struct tmem_pool *pool,
zero_filled = true;
zsize = 0;
zpages = 1;
+ dec_zcache_zero_filled_pages();
}
if (pampd_is_remote(pampd) && !zero_filled) {
--
1.7.5.4
On Wed, Apr 3, 2013 at 6:16 AM, Wanpeng Li <[email protected]> wrote:
> Introduce zero-filled page statistics to monitor the number of
> zero-filled pages.
>
> Acked-by: Dan Magenheimer <[email protected]>
> Signed-off-by: Wanpeng Li <[email protected]>
Reviewed-by: Konrad Rzeszutek Wilk <[email protected]>
> ---
> drivers/staging/zcache/debug.c | 3 +++
> drivers/staging/zcache/debug.h | 17 +++++++++++++++++
> drivers/staging/zcache/zcache-main.c | 4 ++++
> 3 files changed, 24 insertions(+), 0 deletions(-)
>
> diff --git a/drivers/staging/zcache/debug.c b/drivers/staging/zcache/debug.c
> index faab2a9..daa2691 100644
> --- a/drivers/staging/zcache/debug.c
> +++ b/drivers/staging/zcache/debug.c
> @@ -35,6 +35,8 @@ ssize_t zcache_pers_ate_eph;
> ssize_t zcache_pers_ate_eph_failed;
> ssize_t zcache_evicted_eph_zpages;
> ssize_t zcache_evicted_eph_pageframes;
> +ssize_t zcache_zero_filled_pages;
> +ssize_t zcache_zero_filled_pages_max;
>
> #define ATTR(x) { .name = #x, .val = &zcache_##x, }
> static struct debug_entry {
> @@ -62,6 +64,7 @@ static struct debug_entry {
> ATTR(last_inactive_anon_pageframes),
> ATTR(eph_nonactive_puts_ignored),
> ATTR(pers_nonactive_puts_ignored),
> + ATTR(zero_filled_pages),
> #ifdef CONFIG_ZCACHE_WRITEBACK
> ATTR(outstanding_writeback_pages),
> ATTR(writtenback_pages),
> diff --git a/drivers/staging/zcache/debug.h b/drivers/staging/zcache/debug.h
> index 8ec82d4..ddad92f 100644
> --- a/drivers/staging/zcache/debug.h
> +++ b/drivers/staging/zcache/debug.h
> @@ -122,6 +122,21 @@ static inline void dec_zcache_pers_zpages(unsigned zpages)
> zcache_pers_zpages = atomic_sub_return(zpages, &zcache_pers_zpages_atomic);
> }
>
> +extern ssize_t zcache_zero_filled_pages;
> +static atomic_t zcache_zero_filled_pages_atomic = ATOMIC_INIT(0);
> +extern ssize_t zcache_zero_filled_pages_max;
> +static inline void inc_zcache_zero_filled_pages(void)
> +{
> + zcache_zero_filled_pages = atomic_inc_return(
> + &zcache_zero_filled_pages_atomic);
> + if (zcache_zero_filled_pages > zcache_zero_filled_pages_max)
> + zcache_zero_filled_pages_max = zcache_zero_filled_pages;
> +}
> +static inline void dec_zcache_zero_filled_pages(void)
> +{
> + zcache_zero_filled_pages = atomic_dec_return(
> + &zcache_zero_filled_pages_atomic);
> +}
> static inline unsigned long curr_pageframes_count(void)
> {
> return zcache_pageframes_alloced -
> @@ -200,6 +215,8 @@ static inline void inc_zcache_eph_zpages(void) { };
> static inline void dec_zcache_eph_zpages(unsigned zpages) { };
> static inline void inc_zcache_pers_zpages(void) { };
> static inline void dec_zcache_pers_zpages(unsigned zpages) { };
> +static inline void inc_zcache_zero_filled_pages(void) { };
> +static inline void dec_zcache_zero_filled_pages(void) { };
> static inline unsigned long curr_pageframes_count(void)
> {
> return 0;
> diff --git a/drivers/staging/zcache/zcache-main.c b/drivers/staging/zcache/zcache-main.c
> index 1994cab..f3de76d 100644
> --- a/drivers/staging/zcache/zcache-main.c
> +++ b/drivers/staging/zcache/zcache-main.c
> @@ -374,6 +374,7 @@ static void *zcache_pampd_eph_create(char *data, size_t size, bool raw,
> if (page_is_zero_filled(page)) {
> clen = 0;
> zero_filled = true;
> + inc_zcache_zero_filled_pages();
> goto got_pampd;
> }
>
> @@ -440,6 +441,7 @@ static void *zcache_pampd_pers_create(char *data, size_t size, bool raw,
> if (page_is_zero_filled(page)) {
> clen = 0;
> zero_filled = true;
> + inc_zcache_zero_filled_pages();
> goto got_pampd;
> }
>
> @@ -652,6 +654,7 @@ static int zcache_pampd_get_data_and_free(char *data, size_t *sizep, bool raw,
> zpages = 1;
> if (!raw)
> *sizep = PAGE_SIZE;
> + dec_zcache_zero_filled_pages();
> goto zero_fill;
> }
>
> @@ -702,6 +705,7 @@ static void zcache_pampd_free(void *pampd, struct tmem_pool *pool,
> zero_filled = true;
> zsize = 0;
> zpages = 1;
> + dec_zcache_zero_filled_pages();
> }
>
> if (pampd_is_remote(pampd) && !zero_filled) {
> --
> 1.7.5.4
>
On Wed, Apr 3, 2013 at 6:16 AM, Wanpeng Li <[email protected]> wrote:
> After commit 95bdaee214 ("zcache: Move debugfs code out of zcache-main.c file")
> be merged, most of knods in zcache debugfs just export zero since these variables
> are defined in debug.h but are in use in multiple C files zcache-main.c and debug.c,
> in this case variables can't be treated as shared variables.
>
> Signed-off-by: Wanpeng Li <[email protected]>
Reviewed-by: Konrad Rzeszutek Wilk <[email protected]>
cc Bob
On 04/07/2013 05:03 PM, Wanpeng Li wrote:
> On Wed, Apr 03, 2013 at 06:16:20PM +0800, Wanpeng Li wrote:
>> Changelog:
>> v5 -> v6:
>> * shove variables in debug.c and in debug.h just have an extern, spotted by Konrad
>> * update patch description, spotted by Konrad
>> v4 -> v5:
>> * fix compile error, reported by Fengguang, Geert
>> * add check for !is_ephemeral(pool), spotted by Bob
>> v3 -> v4:
>> * handle duplication in page_is_zero_filled, spotted by Bob
>> * fix zcache writeback in dubugfs
>> * fix pers_pageframes|_max isn't exported in debugfs
>> * fix static variable defined in debug.h but used in multiple C files
>> * rebase on Greg's staging-next
>> v2 -> v3:
>> * increment/decrement zcache_[eph|pers]_zpages for zero-filled pages, spotted by Dan
>> * replace "zero" or "zero page" by "zero_filled_page", spotted by Dan
>> v1 -> v2:
>> * avoid changing tmem.[ch] entirely, spotted by Dan.
>> * don't accumulate [eph|pers]pageframe and [eph|pers]zpages for
>> zero-filled pages, spotted by Dan
>> * cleanup TODO list
>> * add Dan Acked-by.
>>
> Hi Dan,
>
> Some issues against Ramster:
>
> - Ramster who takes advantage of zcache also should support zero-filled
> pages more efficiently, correct? It doesn't handle zero-filled pages well
> currently.
> - Ramster DebugFS counters are exported in /sys/kernel/mm/, but zcache/frontswap/cleancache
> all are exported in /sys/kernel/debug/, should we unify them?
> - If ramster also should move DebugFS counters to a single file like
> zcache do?
>
> If you confirm these issues are make sense to fix, I will start coding. ;-)
>
> Regards,
> Wanpeng Li
>
>> Motivation:
>>
>> - Seth Jennings points out compress zero-filled pages with LZO(a lossless
>> data compression algorithm) will waste memory and result in fragmentation.
>> https://lkml.org/lkml/2012/8/14/347
>> - Dan Magenheimer add "Support zero-filled pages more efficiently" feature
>> in zcache TODO list https://lkml.org/lkml/2013/2/13/503
>>
>> Design:
>>
>> - For store page, capture zero-filled pages(evicted clean page cache pages and
>> swap pages), but don't compress them, set pampd which store zpage address to
>> 0x2(since 0x0 and 0x1 has already been ocuppied) to mark special zero-filled
>> case and take advantage of tmem infrastructure to transform handle-tuple(pool
>> id, object id, and an index) to a pampd. Twice compress zero-filled pages will
>> contribute to one zcache_[eph|pers]_pageframes count accumulated.
>> - For load page, traverse tmem hierachical to transform handle-tuple to pampd
>> and identify zero-filled case by pampd equal to 0x2 when filesystem reads
>> file pages or a page needs to be swapped in, then refill the page to zero
>> and return.
>>
>> Test:
>>
>> dd if=/dev/zero of=zerofile bs=1MB count=500
>> vmtouch -t zerofile
>> vmtouch -e zerofile
>>
>> formula:
>> - fragmentation level = (zcache_[eph|pers]_pageframes * PAGE_SIZE - zcache_[eph|pers]_zbytes)
>> * 100 / (zcache_[eph|pers]_pageframes * PAGE_SIZE)
>> - memory zcache occupy = zcache_[eph|pers]_zbytes
>>
>> Result:
>>
>> without zero-filled awareness:
>> - fragmentation level: 98%
>> - memory zcache occupy: 238MB
>> with zero-filled awareness:
>> - fragmentation level: 0%
>> - memory zcache occupy: 0MB
>>
>> Wanpeng Li (3):
>> staging: zcache: fix static variables defined in debug.h but used in
>> mutiple C files
>> staging: zcache: introduce zero-filled page stat count
>> staging: zcache: clean TODO list
>>
>> drivers/staging/zcache/TODO | 3 +-
>> drivers/staging/zcache/debug.c | 35 +++++++++++++++
>> drivers/staging/zcache/debug.h | 79 ++++++++++++++++++++-------------
>> drivers/staging/zcache/zcache-main.c | 4 ++
>> 4 files changed, 88 insertions(+), 33 deletions(-)
>>
>> --
>> 1.7.5.4
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to [email protected]. For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
> From: Wanpeng Li [mailto:[email protected]]
> Subject: Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
>
> Hi Dan,
>
> Some issues against Ramster:
>
> - Ramster who takes advantage of zcache also should support zero-filled
> pages more efficiently, correct? It doesn't handle zero-filled pages well
> currently.
When you first posted your patchset I took a quick look at ramster
and it looked like your patchset should work for ramster also.
However I didn't actually run ramster to try it so there may
be a bug. If it doesn't work, I would very much appreciate a patch.
> - Ramster DebugFS counters are exported in /sys/kernel/mm/, but zcache/frontswap/cleancache
> all are exported in /sys/kernel/debug/, should we unify them?
That would be great.
> - If ramster also should move DebugFS counters to a single file like
> zcache do?
Sure! I am concerned about Konrad's patches adding debug.c as they
add many global variables. They are only required when ZCACHE_DEBUG
is enabled so they may be ok. If not, adding ramster variables
to debug.c may make the problem worse.
> If you confirm these issues are make sense to fix, I will start coding. ;-)
That would be great. Note that I have a how-to for ramster here:
https://oss.oracle.com/projects/tmem/dist/files/RAMster/HOWTO-120817
If when you are testing you find that this how-to has mistakes,
please let me know. Or feel free to add the (corrected) how-to file
as a patch in your patchset.
Thanks very much, Wanpeng, for your great contributions!
(Ric, since you have expressed interest in ramster, if you try it and
find corrections to the how-to file above, your input would be
very much appreciated also!)
Dan
> From: Dan Magenheimer
> Subject: RE: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
>
> > From: Wanpeng Li [mailto:[email protected]]
> > Subject: Re: [PATCH part2 v6 0/3] staging: zcache: Support zero-filled pages more efficiently
> >
> > Hi Dan,
> >
> > Some issues against Ramster:
> >
>
> Sure! I am concerned about Konrad's patches adding debug.c as they
> add many global variables. They are only required when ZCACHE_DEBUG
> is enabled so they may be ok. If not, adding ramster variables
> to debug.c may make the problem worse.
Oops, I just noticed/remembered that ramster uses BOTH debugfs and sysfs.
The sysfs variables are all currently required, i.e. for configuration
so should not be tied to debugfs or a DEBUG config option. However,
if there is a more acceptable way to implement the function of
those sysfs variables, that would be fine.
Thanks,
Dan