Changelog:
v4 -> v5:
* fix compile error, reported by Fengguang, Geert
* add check for !is_ephemeral(pool), spotted by Bob
v3 -> v4:
* handle duplication in page_is_zero_filled, spotted by Bob
* fix zcache writeback in dubugfs
* fix pers_pageframes|_max isn't exported in debugfs
* fix static variable defined in debug.h but used in multiple C files
* rebase on Greg's staging-next
v2 -> v3:
* increment/decrement zcache_[eph|pers]_zpages for zero-filled pages, spotted by Dan
* replace "zero" or "zero page" by "zero_filled_page", spotted by Dan
v1 -> v2:
* avoid changing tmem.[ch] entirely, spotted by Dan.
* don't accumulate [eph|pers]pageframe and [eph|pers]zpages for
zero-filled pages, spotted by Dan
* cleanup TODO list
* add Dan Acked-by.
Motivation:
- Seth Jennings points out compress zero-filled pages with LZO(a lossless
data compression algorithm) will waste memory and result in fragmentation.
https://lkml.org/lkml/2012/8/14/347
- Dan Magenheimer add "Support zero-filled pages more efficiently" feature
in zcache TODO list https://lkml.org/lkml/2013/2/13/503
Design:
- For store page, capture zero-filled pages(evicted clean page cache pages and
swap pages), but don't compress them, set pampd which store zpage address to
0x2(since 0x0 and 0x1 has already been ocuppied) to mark special zero-filled
case and take advantage of tmem infrastructure to transform handle-tuple(pool
id, object id, and an index) to a pampd. Twice compress zero-filled pages will
contribute to one zcache_[eph|pers]_pageframes count accumulated.
- For load page, traverse tmem hierachical to transform handle-tuple to pampd
and identify zero-filled case by pampd equal to 0x2 when filesystem reads
file pages or a page needs to be swapped in, then refill the page to zero
and return.
Test:
dd if=/dev/zero of=zerofile bs=1MB count=500
vmtouch -t zerofile
vmtouch -e zerofile
formula:
- fragmentation level = (zcache_[eph|pers]_pageframes * PAGE_SIZE - zcache_[eph|pers]_zbytes)
* 100 / (zcache_[eph|pers]_pageframes * PAGE_SIZE)
- memory zcache occupy = zcache_[eph|pers]_zbytes
Result:
without zero-filled awareness:
- fragmentation level: 98%
- memory zcache occupy: 238MB
with zero-filled awareness:
- fragmentation level: 0%
- memory zcache occupy: 0MB
Wanpeng Li (8):
staging: zcache: Support zero-filled pages more efficiently
staging: zcache: zero-filled pages awareness
staging: zcache: handle zcache_[eph|pers]_zpages for zero-filled page
staging: zcache: fix pers_pageframes|_max aren't exported in debugfs
staging: zcache: fix zcache writeback in debugfs
staging: zcache: fix static variables defined in debug.h but used in
mutiple C files
staging: zcache: introduce zero-filled page stat count
staging: zcache: clean TODO list
drivers/staging/zcache/TODO | 3 +-
drivers/staging/zcache/debug.c | 5 +-
drivers/staging/zcache/debug.h | 77 +++++++++++--------
drivers/staging/zcache/zcache-main.c | 141 ++++++++++++++++++++++++++++++++--
4 files changed, 183 insertions(+), 43 deletions(-)
--
1.7.7.6
Increment/decrement zcache_[eph|pers]_zpages for zero-filled pages,
the main point of the counters for zpages and pageframes is to be
able to calculate density == zpages/pageframes. A zero-filled page
becomes a zpage that "compresses" to zero bytes and, as a result,
requires zero pageframes for storage. So the zpages counter should
be increased but the pageframes counter should not.
[Dan Magenheimer <[email protected]>: patch description]
Acked-by: Dan Magenheimer <[email protected]>
Reviewed-by: Konrad Rzeszutek Wilk <[email protected]>
Signed-off-by: Wanpeng Li <[email protected]>
---
drivers/staging/zcache/zcache-main.c | 7 ++++++-
1 files changed, 6 insertions(+), 1 deletions(-)
diff --git a/drivers/staging/zcache/zcache-main.c b/drivers/staging/zcache/zcache-main.c
index 961fbf1..14dcf8a 100644
--- a/drivers/staging/zcache/zcache-main.c
+++ b/drivers/staging/zcache/zcache-main.c
@@ -648,6 +648,8 @@ static int zcache_pampd_get_data_and_free(char *data, size_t *sizep, bool raw,
if (pampd == (void *)ZERO_FILLED) {
handle_zero_filled_page(data);
zero_filled = true;
+ zsize = 0;
+ zpages = 1;
if (!raw)
*sizep = PAGE_SIZE;
goto zero_fill;
@@ -696,8 +698,11 @@ static void zcache_pampd_free(void *pampd, struct tmem_pool *pool,
BUG_ON(preemptible());
- if (pampd == (void *)ZERO_FILLED)
+ if (pampd == (void *)ZERO_FILLED) {
zero_filled = true;
+ zsize = 0;
+ zpages = 1;
+ }
if (pampd_is_remote(pampd) && !zero_filled) {
BUG_ON(!ramster_enabled);
--
1.7.7.6
Introduce zero-filled pages handler to capture and handle zero pages.
Acked-by: Dan Magenheimer <[email protected]>
Signed-off-by: Wanpeng Li <[email protected]>
---
drivers/staging/zcache/zcache-main.c | 27 +++++++++++++++++++++++++++
1 files changed, 27 insertions(+), 0 deletions(-)
diff --git a/drivers/staging/zcache/zcache-main.c b/drivers/staging/zcache/zcache-main.c
index 7a6dd96..a0578d1 100644
--- a/drivers/staging/zcache/zcache-main.c
+++ b/drivers/staging/zcache/zcache-main.c
@@ -277,6 +277,33 @@ static void zcache_obj_free(struct tmem_obj *obj, struct tmem_pool *pool)
kmem_cache_free(zcache_obj_cache, obj);
}
+static bool page_is_zero_filled(void *ptr)
+{
+ unsigned int pos;
+ unsigned long *page;
+
+ page = (unsigned long *)ptr;
+
+ for (pos = 0; pos < PAGE_SIZE / sizeof(*page); pos++) {
+ if (page[pos])
+ return false;
+ }
+
+ return true;
+}
+
+static void handle_zero_filled_page(void *p)
+{
+ void *user_mem;
+ struct page *page = (struct page *)p;
+
+ user_mem = kmap_atomic(page);
+ memset(user_mem, 0, PAGE_SIZE);
+ kunmap_atomic(user_mem);
+
+ flush_dcache_page(page);
+}
+
static struct tmem_hostops zcache_hostops = {
.obj_alloc = zcache_obj_alloc,
.obj_free = zcache_obj_free,
--
1.7.7.6
After commit 95bdaee214 ("zcache: Move debugfs code out of zcache-main.c file")
be merged, most of knods in zcache debugfs just export zero since these variables
are defined in debug.h but use in multiple C files zcache-main.c and debug.c, in
this case variables can't be treated as shared variables.
Signed-off-by: Wanpeng Li <[email protected]>
---
drivers/staging/zcache/debug.h | 62 +++++++++++++++++-----------------
drivers/staging/zcache/zcache-main.c | 31 +++++++++++++++++
2 files changed, 62 insertions(+), 31 deletions(-)
diff --git a/drivers/staging/zcache/debug.h b/drivers/staging/zcache/debug.h
index 4bbe49b..8ec82d4 100644
--- a/drivers/staging/zcache/debug.h
+++ b/drivers/staging/zcache/debug.h
@@ -3,9 +3,9 @@
#ifdef CONFIG_ZCACHE_DEBUG
/* we try to keep these statistics SMP-consistent */
-static ssize_t zcache_obj_count;
+extern ssize_t zcache_obj_count;
static atomic_t zcache_obj_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_obj_count_max;
+extern ssize_t zcache_obj_count_max;
static inline void inc_zcache_obj_count(void)
{
zcache_obj_count = atomic_inc_return(&zcache_obj_atomic);
@@ -17,9 +17,9 @@ static inline void dec_zcache_obj_count(void)
zcache_obj_count = atomic_dec_return(&zcache_obj_atomic);
BUG_ON(zcache_obj_count < 0);
};
-static ssize_t zcache_objnode_count;
+extern ssize_t zcache_objnode_count;
static atomic_t zcache_objnode_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_objnode_count_max;
+extern ssize_t zcache_objnode_count_max;
static inline void inc_zcache_objnode_count(void)
{
zcache_objnode_count = atomic_inc_return(&zcache_objnode_atomic);
@@ -31,9 +31,9 @@ static inline void dec_zcache_objnode_count(void)
zcache_objnode_count = atomic_dec_return(&zcache_objnode_atomic);
BUG_ON(zcache_objnode_count < 0);
};
-static u64 zcache_eph_zbytes;
+extern u64 zcache_eph_zbytes;
static atomic_long_t zcache_eph_zbytes_atomic = ATOMIC_INIT(0);
-static u64 zcache_eph_zbytes_max;
+extern u64 zcache_eph_zbytes_max;
static inline void inc_zcache_eph_zbytes(unsigned clen)
{
zcache_eph_zbytes = atomic_long_add_return(clen, &zcache_eph_zbytes_atomic);
@@ -46,7 +46,7 @@ static inline void dec_zcache_eph_zbytes(unsigned zsize)
};
extern u64 zcache_pers_zbytes;
static atomic_long_t zcache_pers_zbytes_atomic = ATOMIC_INIT(0);
-static u64 zcache_pers_zbytes_max;
+extern u64 zcache_pers_zbytes_max;
static inline void inc_zcache_pers_zbytes(unsigned clen)
{
zcache_pers_zbytes = atomic_long_add_return(clen, &zcache_pers_zbytes_atomic);
@@ -59,7 +59,7 @@ static inline void dec_zcache_pers_zbytes(unsigned zsize)
}
extern ssize_t zcache_eph_pageframes;
static atomic_t zcache_eph_pageframes_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_eph_pageframes_max;
+extern ssize_t zcache_eph_pageframes_max;
static inline void inc_zcache_eph_pageframes(void)
{
zcache_eph_pageframes = atomic_inc_return(&zcache_eph_pageframes_atomic);
@@ -72,7 +72,7 @@ static inline void dec_zcache_eph_pageframes(void)
};
extern ssize_t zcache_pers_pageframes;
static atomic_t zcache_pers_pageframes_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_pers_pageframes_max;
+extern ssize_t zcache_pers_pageframes_max;
static inline void inc_zcache_pers_pageframes(void)
{
zcache_pers_pageframes = atomic_inc_return(&zcache_pers_pageframes_atomic);
@@ -83,21 +83,21 @@ static inline void dec_zcache_pers_pageframes(void)
{
zcache_pers_pageframes = atomic_dec_return(&zcache_pers_pageframes_atomic);
}
-static ssize_t zcache_pageframes_alloced;
+extern ssize_t zcache_pageframes_alloced;
static atomic_t zcache_pageframes_alloced_atomic = ATOMIC_INIT(0);
static inline void inc_zcache_pageframes_alloced(void)
{
zcache_pageframes_alloced = atomic_inc_return(&zcache_pageframes_alloced_atomic);
};
-static ssize_t zcache_pageframes_freed;
+extern ssize_t zcache_pageframes_freed;
static atomic_t zcache_pageframes_freed_atomic = ATOMIC_INIT(0);
static inline void inc_zcache_pageframes_freed(void)
{
zcache_pageframes_freed = atomic_inc_return(&zcache_pageframes_freed_atomic);
}
-static ssize_t zcache_eph_zpages;
+extern ssize_t zcache_eph_zpages;
static atomic_t zcache_eph_zpages_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_eph_zpages_max;
+extern ssize_t zcache_eph_zpages_max;
static inline void inc_zcache_eph_zpages(void)
{
zcache_eph_zpages = atomic_inc_return(&zcache_eph_zpages_atomic);
@@ -110,7 +110,7 @@ static inline void dec_zcache_eph_zpages(unsigned zpages)
}
extern ssize_t zcache_pers_zpages;
static atomic_t zcache_pers_zpages_atomic = ATOMIC_INIT(0);
-static ssize_t zcache_pers_zpages_max;
+extern ssize_t zcache_pers_zpages_max;
static inline void inc_zcache_pers_zpages(void)
{
zcache_pers_zpages = atomic_inc_return(&zcache_pers_zpages_atomic);
@@ -130,23 +130,23 @@ static inline unsigned long curr_pageframes_count(void)
atomic_read(&zcache_pers_pageframes_atomic);
};
/* but for the rest of these, counting races are ok */
-static ssize_t zcache_flush_total;
-static ssize_t zcache_flush_found;
-static ssize_t zcache_flobj_total;
-static ssize_t zcache_flobj_found;
-static ssize_t zcache_failed_eph_puts;
-static ssize_t zcache_failed_pers_puts;
-static ssize_t zcache_failed_getfreepages;
-static ssize_t zcache_failed_alloc;
-static ssize_t zcache_put_to_flush;
-static ssize_t zcache_compress_poor;
-static ssize_t zcache_mean_compress_poor;
-static ssize_t zcache_eph_ate_tail;
-static ssize_t zcache_eph_ate_tail_failed;
-static ssize_t zcache_pers_ate_eph;
-static ssize_t zcache_pers_ate_eph_failed;
-static ssize_t zcache_evicted_eph_zpages;
-static ssize_t zcache_evicted_eph_pageframes;
+extern ssize_t zcache_flush_total;
+extern ssize_t zcache_flush_found;
+extern ssize_t zcache_flobj_total;
+extern ssize_t zcache_flobj_found;
+extern ssize_t zcache_failed_eph_puts;
+extern ssize_t zcache_failed_pers_puts;
+extern ssize_t zcache_failed_getfreepages;
+extern ssize_t zcache_failed_alloc;
+extern ssize_t zcache_put_to_flush;
+extern ssize_t zcache_compress_poor;
+extern ssize_t zcache_mean_compress_poor;
+extern ssize_t zcache_eph_ate_tail;
+extern ssize_t zcache_eph_ate_tail_failed;
+extern ssize_t zcache_pers_ate_eph;
+extern ssize_t zcache_pers_ate_eph_failed;
+extern ssize_t zcache_evicted_eph_zpages;
+extern ssize_t zcache_evicted_eph_pageframes;
extern ssize_t zcache_last_active_file_pageframes;
extern ssize_t zcache_last_inactive_file_pageframes;
diff --git a/drivers/staging/zcache/zcache-main.c b/drivers/staging/zcache/zcache-main.c
index 14dcf8a..e112c1e 100644
--- a/drivers/staging/zcache/zcache-main.c
+++ b/drivers/staging/zcache/zcache-main.c
@@ -145,6 +145,37 @@ ssize_t zcache_pers_zpages;
u64 zcache_pers_zbytes;
ssize_t zcache_eph_pageframes;
ssize_t zcache_pers_pageframes;
+ssize_t zcache_obj_count;
+ssize_t zcache_obj_count_max;
+ssize_t zcache_objnode_count;
+ssize_t zcache_objnode_count_max;
+u64 zcache_eph_zbytes;
+u64 zcache_eph_zbytes_max;
+u64 zcache_pers_zbytes_max;
+ssize_t zcache_eph_pageframes_max;
+ssize_t zcache_pers_pageframes_max;
+ssize_t zcache_pageframes_alloced;
+ssize_t zcache_pageframes_freed;
+ssize_t zcache_eph_zpages;
+ssize_t zcache_eph_zpages_max;
+ssize_t zcache_pers_zpages_max;
+ssize_t zcache_flush_total;
+ssize_t zcache_flush_found;
+ssize_t zcache_flobj_total;
+ssize_t zcache_flobj_found;
+ssize_t zcache_failed_eph_puts;
+ssize_t zcache_failed_pers_puts;
+ssize_t zcache_failed_getfreepages;
+ssize_t zcache_failed_alloc;
+ssize_t zcache_put_to_flush;
+ssize_t zcache_compress_poor;
+ssize_t zcache_mean_compress_poor;
+ssize_t zcache_eph_ate_tail;
+ssize_t zcache_eph_ate_tail_failed;
+ssize_t zcache_pers_ate_eph;
+ssize_t zcache_pers_ate_eph_failed;
+ssize_t zcache_evicted_eph_zpages;
+ssize_t zcache_evicted_eph_pageframes;
/* Used by this code. */
ssize_t zcache_last_active_file_pageframes;
--
1.7.7.6
Compression of zero-filled pages can unneccessarily cause internal
fragmentation, and thus waste memory. This special case can be
optimized.
This patch captures zero-filled pages, and marks their corresponding
zcache backing page entry as zero-filled. Whenever such zero-filled
page is retrieved, we fill the page frame with zero.
Acked-by: Dan Magenheimer <[email protected]>
Reviewed-by: Konrad Rzeszutek Wilk <[email protected]>
Signed-off-by: Wanpeng Li <[email protected]>
---
drivers/staging/zcache/zcache-main.c | 83 ++++++++++++++++++++++++++++-----
1 files changed, 70 insertions(+), 13 deletions(-)
diff --git a/drivers/staging/zcache/zcache-main.c b/drivers/staging/zcache/zcache-main.c
index a0578d1..961fbf1 100644
--- a/drivers/staging/zcache/zcache-main.c
+++ b/drivers/staging/zcache/zcache-main.c
@@ -61,6 +61,12 @@ static inline void frontswap_tmem_exclusive_gets(bool b)
}
#endif
+/*
+ * mark pampd to special value in order that later
+ * retrieve will identify zero-filled pages
+ */
+#define ZERO_FILLED 0x2
+
/* enable (or fix code) when Seth's patches are accepted upstream */
#define zcache_writeback_enabled 0
@@ -277,17 +283,23 @@ static void zcache_obj_free(struct tmem_obj *obj, struct tmem_pool *pool)
kmem_cache_free(zcache_obj_cache, obj);
}
-static bool page_is_zero_filled(void *ptr)
+/*
+ * Compressing zero-filled pages will waste memory and introduce
+ * serious fragmentation, skip it to avoid overhead.
+ */
+static bool page_is_zero_filled(struct page *p)
{
unsigned int pos;
- unsigned long *page;
-
- page = (unsigned long *)ptr;
+ char *page;
+ page = kmap_atomic(p);
for (pos = 0; pos < PAGE_SIZE / sizeof(*page); pos++) {
- if (page[pos])
+ if (page[pos]) {
+ kunmap_atomic(page);
return false;
+ }
}
+ kunmap_atomic(page);
return true;
}
@@ -356,8 +368,15 @@ static void *zcache_pampd_eph_create(char *data, size_t size, bool raw,
{
void *pampd = NULL, *cdata = data;
unsigned clen = size;
+ bool zero_filled = false;
struct page *page = (struct page *)(data), *newpage;
+ if (page_is_zero_filled(page)) {
+ clen = 0;
+ zero_filled = true;
+ goto got_pampd;
+ }
+
if (!raw) {
zcache_compress(page, &cdata, &clen);
if (clen > zbud_max_buddy_size()) {
@@ -397,6 +416,8 @@ got_pampd:
inc_zcache_eph_zpages();
if (ramster_enabled && raw)
ramster_count_foreign_pages(true, 1);
+ if (zero_filled)
+ pampd = (void *)ZERO_FILLED;
out:
return pampd;
}
@@ -406,6 +427,7 @@ static void *zcache_pampd_pers_create(char *data, size_t size, bool raw,
{
void *pampd = NULL, *cdata = data;
unsigned clen = size;
+ bool zero_filled = false;
struct page *page = (struct page *)(data), *newpage;
unsigned long zbud_mean_zsize;
unsigned long curr_pers_zpages, total_zsize;
@@ -414,6 +436,13 @@ static void *zcache_pampd_pers_create(char *data, size_t size, bool raw,
BUG_ON(!ramster_enabled);
goto create_pampd;
}
+
+ if (page_is_zero_filled(page)) {
+ clen = 0;
+ zero_filled = true;
+ goto got_pampd;
+ }
+
curr_pers_zpages = zcache_pers_zpages;
/* FIXME CONFIG_RAMSTER... subtract atomic remote_pers_pages here? */
if (!raw)
@@ -471,6 +500,8 @@ got_pampd:
inc_zcache_pers_zbytes(clen);
if (ramster_enabled && raw)
ramster_count_foreign_pages(false, 1);
+ if (zero_filled)
+ pampd = (void *)ZERO_FILLED;
out:
return pampd;
}
@@ -532,7 +563,8 @@ out:
*/
void zcache_pampd_create_finish(void *pampd, bool eph)
{
- zbud_create_finish((struct zbudref *)pampd, eph);
+ if (pampd != (void *)ZERO_FILLED)
+ zbud_create_finish((struct zbudref *)pampd, eph);
}
/*
@@ -577,6 +609,14 @@ static int zcache_pampd_get_data(char *data, size_t *sizep, bool raw,
BUG_ON(preemptible());
BUG_ON(eph); /* fix later if shared pools get implemented */
BUG_ON(pampd_is_remote(pampd));
+
+ if (pampd == (void *)ZERO_FILLED) {
+ handle_zero_filled_page(data);
+ if (!raw)
+ *sizep = PAGE_SIZE;
+ return 0;
+ }
+
if (raw)
ret = zbud_copy_from_zbud(data, (struct zbudref *)pampd,
sizep, eph);
@@ -597,13 +637,22 @@ static int zcache_pampd_get_data_and_free(char *data, size_t *sizep, bool raw,
void *pampd, struct tmem_pool *pool,
struct tmem_oid *oid, uint32_t index)
{
- int ret;
- bool eph = !is_persistent(pool);
+ int ret = 0;
+ bool eph = !is_persistent(pool), zero_filled = false;
struct page *page = NULL;
unsigned int zsize, zpages;
BUG_ON(preemptible());
BUG_ON(pampd_is_remote(pampd));
+
+ if (pampd == (void *)ZERO_FILLED) {
+ handle_zero_filled_page(data);
+ zero_filled = true;
+ if (!raw)
+ *sizep = PAGE_SIZE;
+ goto zero_fill;
+ }
+
if (raw)
ret = zbud_copy_from_zbud(data, (struct zbudref *)pampd,
sizep, eph);
@@ -615,6 +664,7 @@ static int zcache_pampd_get_data_and_free(char *data, size_t *sizep, bool raw,
}
page = zbud_free_and_delist((struct zbudref *)pampd, eph,
&zsize, &zpages);
+zero_fill:
if (eph) {
if (page)
dec_zcache_eph_pageframes();
@@ -628,7 +678,7 @@ static int zcache_pampd_get_data_and_free(char *data, size_t *sizep, bool raw,
}
if (!is_local_client(pool->client))
ramster_count_foreign_pages(eph, -1);
- if (page)
+ if (page && !zero_filled)
zcache_free_page(page);
return ret;
}
@@ -642,16 +692,22 @@ static void zcache_pampd_free(void *pampd, struct tmem_pool *pool,
{
struct page *page = NULL;
unsigned int zsize, zpages;
+ bool zero_filled = false;
BUG_ON(preemptible());
- if (pampd_is_remote(pampd)) {
+
+ if (pampd == (void *)ZERO_FILLED)
+ zero_filled = true;
+
+ if (pampd_is_remote(pampd) && !zero_filled) {
BUG_ON(!ramster_enabled);
pampd = ramster_pampd_free(pampd, pool, oid, index, acct);
if (pampd == NULL)
return;
}
if (is_ephemeral(pool)) {
- page = zbud_free_and_delist((struct zbudref *)pampd,
+ if (!zero_filled)
+ page = zbud_free_and_delist((struct zbudref *)pampd,
true, &zsize, &zpages);
if (page)
dec_zcache_eph_pageframes();
@@ -659,7 +715,8 @@ static void zcache_pampd_free(void *pampd, struct tmem_pool *pool,
dec_zcache_eph_zbytes(zsize);
/* FIXME CONFIG_RAMSTER... check acct parameter? */
} else {
- page = zbud_free_and_delist((struct zbudref *)pampd,
+ if (!zero_filled)
+ page = zbud_free_and_delist((struct zbudref *)pampd,
false, &zsize, &zpages);
if (page)
dec_zcache_pers_pageframes();
@@ -668,7 +725,7 @@ static void zcache_pampd_free(void *pampd, struct tmem_pool *pool,
}
if (!is_local_client(pool->client))
ramster_count_foreign_pages(is_ephemeral(pool), -1);
- if (page)
+ if (page && !zero_filled)
zcache_free_page(page);
}
--
1.7.7.6
Introduce zero-filled page statistics to monitor the number of
zero-filled pages.
Acked-by: Dan Magenheimer <[email protected]>
Signed-off-by: Wanpeng Li <[email protected]>
---
drivers/staging/zcache/debug.h | 15 +++++++++++++++
drivers/staging/zcache/zcache-main.c | 6 ++++++
2 files changed, 21 insertions(+), 0 deletions(-)
diff --git a/drivers/staging/zcache/debug.h b/drivers/staging/zcache/debug.h
index 8ec82d4..178bf75 100644
--- a/drivers/staging/zcache/debug.h
+++ b/drivers/staging/zcache/debug.h
@@ -122,6 +122,21 @@ static inline void dec_zcache_pers_zpages(unsigned zpages)
zcache_pers_zpages = atomic_sub_return(zpages, &zcache_pers_zpages_atomic);
}
+extern ssize_t zcache_zero_filled_pages;
+static atomic_t zcache_zero_filled_pages_atomic = ATOMIC_INIT(0);
+extern ssize_t zcache_zero_filled_pages_max;
+static inline void inc_zcache_zero_filled_pages(void)
+{
+ zcache_zero_filled_pages = atomic_inc_return(
+ &zcache_zero_filled_pages_atomic);
+ if (zcache_zero_filled_pages > zcache_zero_filled_pages_max)
+ zcache_zero_filled_pages_max = zcache_zero_filled_pages;
+}
+static inline void dec_zcache_zero_filled_pages(void)
+{
+ zcache_zero_filled_pages = atomic_dec_return(
+ &zcache_zero_filled_pages_atomic);
+}
static inline unsigned long curr_pageframes_count(void)
{
return zcache_pageframes_alloced -
diff --git a/drivers/staging/zcache/zcache-main.c b/drivers/staging/zcache/zcache-main.c
index e112c1e..f2bd68e 100644
--- a/drivers/staging/zcache/zcache-main.c
+++ b/drivers/staging/zcache/zcache-main.c
@@ -176,6 +176,8 @@ ssize_t zcache_pers_ate_eph;
ssize_t zcache_pers_ate_eph_failed;
ssize_t zcache_evicted_eph_zpages;
ssize_t zcache_evicted_eph_pageframes;
+ssize_t zcache_zero_filled_pages;
+ssize_t zcache_zero_filled_pages_max;
/* Used by this code. */
ssize_t zcache_last_active_file_pageframes;
@@ -405,6 +407,7 @@ static void *zcache_pampd_eph_create(char *data, size_t size, bool raw,
if (page_is_zero_filled(page)) {
clen = 0;
zero_filled = true;
+ inc_zcache_zero_filled_pages();
goto got_pampd;
}
@@ -471,6 +474,7 @@ static void *zcache_pampd_pers_create(char *data, size_t size, bool raw,
if (page_is_zero_filled(page)) {
clen = 0;
zero_filled = true;
+ inc_zcache_zero_filled_pages();
goto got_pampd;
}
@@ -683,6 +687,7 @@ static int zcache_pampd_get_data_and_free(char *data, size_t *sizep, bool raw,
zpages = 1;
if (!raw)
*sizep = PAGE_SIZE;
+ dec_zcache_zero_filled_pages();
goto zero_fill;
}
@@ -733,6 +738,7 @@ static void zcache_pampd_free(void *pampd, struct tmem_pool *pool,
zero_filled = true;
zsize = 0;
zpages = 1;
+ dec_zcache_zero_filled_pages();
}
if (pampd_is_remote(pampd) && !zero_filled) {
--
1.7.7.6
commit 9c0ad59ef ("zcache/debug: Use an array to initialize/use debugfs attributes")
use an array to initialize/use debugfs attributes, .name = #x, .val = &zcache_##x.
For zcache writeback, this commit set .name = zcache_outstanding_writeback_pages and
.name = zcache_writtenback_pages seperately, however, corresponding .val =
&zcache_zcache_outstanding_writeback_pages and .val = &zcache_zcache_writtenback_pages,
which are not correct.
Signed-off-by: Wanpeng Li <[email protected]>
---
drivers/staging/zcache/debug.c | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/staging/zcache/debug.c b/drivers/staging/zcache/debug.c
index 254dada..d2d1fdf 100644
--- a/drivers/staging/zcache/debug.c
+++ b/drivers/staging/zcache/debug.c
@@ -31,8 +31,8 @@ static struct debug_entry {
ATTR(eph_nonactive_puts_ignored),
ATTR(pers_nonactive_puts_ignored),
#ifdef CONFIG_ZCACHE_WRITEBACK
- ATTR(zcache_outstanding_writeback_pages),
- ATTR(zcache_writtenback_pages),
+ ATTR(outstanding_writeback_pages),
+ ATTR(writtenback_pages),
#endif
};
#undef ATTR
--
1.7.7.6
Cleanup TODO list since support zero-filled pages more efficiently has
already done by this patchset.
Acked-by: Dan Magenheimer <[email protected]>
Signed-off-by: Wanpeng Li <[email protected]>
---
drivers/staging/zcache/TODO | 3 +--
1 files changed, 1 insertions(+), 2 deletions(-)
diff --git a/drivers/staging/zcache/TODO b/drivers/staging/zcache/TODO
index ec9aa11..d0c18fa 100644
--- a/drivers/staging/zcache/TODO
+++ b/drivers/staging/zcache/TODO
@@ -61,5 +61,4 @@ ZCACHE FUTURE NEW FUNCTIONALITY
A. Support zsmalloc as an alternative high-density allocator
(See https://lkml.org/lkml/2013/1/23/511)
-B. Support zero-filled pages more efficiently
-C. Possibly support three zbuds per pageframe when space allows
+B. Possibly support three zbuds per pageframe when space allows
--
1.7.7.6
Before commit 9c0ad59ef ("zcache/debug: Use an array to initialize/use debugfs attributes"),
pers_pageframes|_max are exported in debugfs, but this commit forgot use array export
pers_pageframes|_max. This patch add pers_pageframes|_max back.
Signed-off-by: Wanpeng Li <[email protected]>
---
drivers/staging/zcache/debug.c | 1 +
1 files changed, 1 insertions(+), 0 deletions(-)
diff --git a/drivers/staging/zcache/debug.c b/drivers/staging/zcache/debug.c
index e951c64..254dada 100644
--- a/drivers/staging/zcache/debug.c
+++ b/drivers/staging/zcache/debug.c
@@ -21,6 +21,7 @@ static struct debug_entry {
ATTR(pers_ate_eph), ATTR(pers_ate_eph_failed),
ATTR(evicted_eph_zpages), ATTR(evicted_eph_pageframes),
ATTR(eph_pageframes), ATTR(eph_pageframes_max),
+ ATTR(pers_pageframes), ATTR(pers_pageframes_max),
ATTR(eph_zpages), ATTR(eph_zpages_max),
ATTR(pers_zpages), ATTR(pers_zpages_max),
ATTR(last_active_file_pageframes),
--
1.7.7.6
On Mon, Apr 1, 2013 at 10:46 PM, Wanpeng Li <[email protected]> wrote:
> Before commit 9c0ad59ef ("zcache/debug: Use an array to initialize/use debugfs attributes"),
> pers_pageframes|_max are exported in debugfs, but this commit forgot use array export
> pers_pageframes|_max. This patch add pers_pageframes|_max back.
Duh! Thanks for spotting.
Reviewed-by: Konrad Rzeszutek Wilk <[email protected]>
>
> Signed-off-by: Wanpeng Li <[email protected]>
> ---
> drivers/staging/zcache/debug.c | 1 +
> 1 files changed, 1 insertions(+), 0 deletions(-)
>
> diff --git a/drivers/staging/zcache/debug.c b/drivers/staging/zcache/debug.c
> index e951c64..254dada 100644
> --- a/drivers/staging/zcache/debug.c
> +++ b/drivers/staging/zcache/debug.c
> @@ -21,6 +21,7 @@ static struct debug_entry {
> ATTR(pers_ate_eph), ATTR(pers_ate_eph_failed),
> ATTR(evicted_eph_zpages), ATTR(evicted_eph_pageframes),
> ATTR(eph_pageframes), ATTR(eph_pageframes_max),
> + ATTR(pers_pageframes), ATTR(pers_pageframes_max),
> ATTR(eph_zpages), ATTR(eph_zpages_max),
> ATTR(pers_zpages), ATTR(pers_zpages_max),
> ATTR(last_active_file_pageframes),
> --
> 1.7.7.6
>
On Mon, Apr 1, 2013 at 10:46 PM, Wanpeng Li <[email protected]> wrote:
> commit 9c0ad59ef ("zcache/debug: Use an array to initialize/use debugfs attributes")
> use an array to initialize/use debugfs attributes, .name = #x, .val = &zcache_##x.
> For zcache writeback, this commit set .name = zcache_outstanding_writeback_pages and
> .name = zcache_writtenback_pages seperately, however, corresponding .val =
> &zcache_zcache_outstanding_writeback_pages and .val = &zcache_zcache_writtenback_pages,
> which are not correct.
>
Weird. I recall spotting that when I did the patches, but I wonder how
I missed this.
Ah, now I remember - I did a silly patch by adding in #define
CONFIG_ZCACHE_WRITEBACK
in the zcache-main.c code, but forgot to try it out here.
<sigh>
Thank you for spotting and fixing it.
Reviewed-by: Konrad Rzeszutek Wilk <[email protected]>
> Signed-off-by: Wanpeng Li <[email protected]>
> ---
> drivers/staging/zcache/debug.c | 4 ++--
> 1 files changed, 2 insertions(+), 2 deletions(-)
>
> diff --git a/drivers/staging/zcache/debug.c b/drivers/staging/zcache/debug.c
> index 254dada..d2d1fdf 100644
> --- a/drivers/staging/zcache/debug.c
> +++ b/drivers/staging/zcache/debug.c
> @@ -31,8 +31,8 @@ static struct debug_entry {
> ATTR(eph_nonactive_puts_ignored),
> ATTR(pers_nonactive_puts_ignored),
> #ifdef CONFIG_ZCACHE_WRITEBACK
> - ATTR(zcache_outstanding_writeback_pages),
> - ATTR(zcache_writtenback_pages),
> + ATTR(outstanding_writeback_pages),
> + ATTR(writtenback_pages),
> #endif
> };
> #undef ATTR
> --
> 1.7.7.6
>
On Mon, Apr 1, 2013 at 10:46 PM, Wanpeng Li <[email protected]> wrote:
> After commit 95bdaee214 ("zcache: Move debugfs code out of zcache-main.c file")
> be merged, most of knods in zcache debugfs just export zero since these variables
> are defined in debug.h but use in multiple C files zcache-main.c and debug.c, in
I think you meant: "are in use in multiple.."
> this case variables can't be treated as shared variables.
You could also shove them in debug.c right? That way if you compile
without CONFIG_DEBUGFS
then you won't get warnings from unused variables?
>
> Signed-off-by: Wanpeng Li <[email protected]>
> ---
> drivers/staging/zcache/debug.h | 62 +++++++++++++++++-----------------
> drivers/staging/zcache/zcache-main.c | 31 +++++++++++++++++
> 2 files changed, 62 insertions(+), 31 deletions(-)
>
> diff --git a/drivers/staging/zcache/debug.h b/drivers/staging/zcache/debug.h
> index 4bbe49b..8ec82d4 100644
> --- a/drivers/staging/zcache/debug.h
> +++ b/drivers/staging/zcache/debug.h
> @@ -3,9 +3,9 @@
> #ifdef CONFIG_ZCACHE_DEBUG
>
> /* we try to keep these statistics SMP-consistent */
> -static ssize_t zcache_obj_count;
> +extern ssize_t zcache_obj_count;
> static atomic_t zcache_obj_atomic = ATOMIC_INIT(0);
> -static ssize_t zcache_obj_count_max;
> +extern ssize_t zcache_obj_count_max;
> static inline void inc_zcache_obj_count(void)
> {
> zcache_obj_count = atomic_inc_return(&zcache_obj_atomic);
> @@ -17,9 +17,9 @@ static inline void dec_zcache_obj_count(void)
> zcache_obj_count = atomic_dec_return(&zcache_obj_atomic);
> BUG_ON(zcache_obj_count < 0);
> };
> -static ssize_t zcache_objnode_count;
> +extern ssize_t zcache_objnode_count;
> static atomic_t zcache_objnode_atomic = ATOMIC_INIT(0);
> -static ssize_t zcache_objnode_count_max;
> +extern ssize_t zcache_objnode_count_max;
> static inline void inc_zcache_objnode_count(void)
> {
> zcache_objnode_count = atomic_inc_return(&zcache_objnode_atomic);
> @@ -31,9 +31,9 @@ static inline void dec_zcache_objnode_count(void)
> zcache_objnode_count = atomic_dec_return(&zcache_objnode_atomic);
> BUG_ON(zcache_objnode_count < 0);
> };
> -static u64 zcache_eph_zbytes;
> +extern u64 zcache_eph_zbytes;
> static atomic_long_t zcache_eph_zbytes_atomic = ATOMIC_INIT(0);
> -static u64 zcache_eph_zbytes_max;
> +extern u64 zcache_eph_zbytes_max;
> static inline void inc_zcache_eph_zbytes(unsigned clen)
> {
> zcache_eph_zbytes = atomic_long_add_return(clen, &zcache_eph_zbytes_atomic);
> @@ -46,7 +46,7 @@ static inline void dec_zcache_eph_zbytes(unsigned zsize)
> };
> extern u64 zcache_pers_zbytes;
> static atomic_long_t zcache_pers_zbytes_atomic = ATOMIC_INIT(0);
> -static u64 zcache_pers_zbytes_max;
> +extern u64 zcache_pers_zbytes_max;
> static inline void inc_zcache_pers_zbytes(unsigned clen)
> {
> zcache_pers_zbytes = atomic_long_add_return(clen, &zcache_pers_zbytes_atomic);
> @@ -59,7 +59,7 @@ static inline void dec_zcache_pers_zbytes(unsigned zsize)
> }
> extern ssize_t zcache_eph_pageframes;
> static atomic_t zcache_eph_pageframes_atomic = ATOMIC_INIT(0);
> -static ssize_t zcache_eph_pageframes_max;
> +extern ssize_t zcache_eph_pageframes_max;
> static inline void inc_zcache_eph_pageframes(void)
> {
> zcache_eph_pageframes = atomic_inc_return(&zcache_eph_pageframes_atomic);
> @@ -72,7 +72,7 @@ static inline void dec_zcache_eph_pageframes(void)
> };
> extern ssize_t zcache_pers_pageframes;
> static atomic_t zcache_pers_pageframes_atomic = ATOMIC_INIT(0);
> -static ssize_t zcache_pers_pageframes_max;
> +extern ssize_t zcache_pers_pageframes_max;
> static inline void inc_zcache_pers_pageframes(void)
> {
> zcache_pers_pageframes = atomic_inc_return(&zcache_pers_pageframes_atomic);
> @@ -83,21 +83,21 @@ static inline void dec_zcache_pers_pageframes(void)
> {
> zcache_pers_pageframes = atomic_dec_return(&zcache_pers_pageframes_atomic);
> }
> -static ssize_t zcache_pageframes_alloced;
> +extern ssize_t zcache_pageframes_alloced;
> static atomic_t zcache_pageframes_alloced_atomic = ATOMIC_INIT(0);
> static inline void inc_zcache_pageframes_alloced(void)
> {
> zcache_pageframes_alloced = atomic_inc_return(&zcache_pageframes_alloced_atomic);
> };
> -static ssize_t zcache_pageframes_freed;
> +extern ssize_t zcache_pageframes_freed;
> static atomic_t zcache_pageframes_freed_atomic = ATOMIC_INIT(0);
> static inline void inc_zcache_pageframes_freed(void)
> {
> zcache_pageframes_freed = atomic_inc_return(&zcache_pageframes_freed_atomic);
> }
> -static ssize_t zcache_eph_zpages;
> +extern ssize_t zcache_eph_zpages;
> static atomic_t zcache_eph_zpages_atomic = ATOMIC_INIT(0);
> -static ssize_t zcache_eph_zpages_max;
> +extern ssize_t zcache_eph_zpages_max;
> static inline void inc_zcache_eph_zpages(void)
> {
> zcache_eph_zpages = atomic_inc_return(&zcache_eph_zpages_atomic);
> @@ -110,7 +110,7 @@ static inline void dec_zcache_eph_zpages(unsigned zpages)
> }
> extern ssize_t zcache_pers_zpages;
> static atomic_t zcache_pers_zpages_atomic = ATOMIC_INIT(0);
> -static ssize_t zcache_pers_zpages_max;
> +extern ssize_t zcache_pers_zpages_max;
> static inline void inc_zcache_pers_zpages(void)
> {
> zcache_pers_zpages = atomic_inc_return(&zcache_pers_zpages_atomic);
> @@ -130,23 +130,23 @@ static inline unsigned long curr_pageframes_count(void)
> atomic_read(&zcache_pers_pageframes_atomic);
> };
> /* but for the rest of these, counting races are ok */
> -static ssize_t zcache_flush_total;
> -static ssize_t zcache_flush_found;
> -static ssize_t zcache_flobj_total;
> -static ssize_t zcache_flobj_found;
> -static ssize_t zcache_failed_eph_puts;
> -static ssize_t zcache_failed_pers_puts;
> -static ssize_t zcache_failed_getfreepages;
> -static ssize_t zcache_failed_alloc;
> -static ssize_t zcache_put_to_flush;
> -static ssize_t zcache_compress_poor;
> -static ssize_t zcache_mean_compress_poor;
> -static ssize_t zcache_eph_ate_tail;
> -static ssize_t zcache_eph_ate_tail_failed;
> -static ssize_t zcache_pers_ate_eph;
> -static ssize_t zcache_pers_ate_eph_failed;
> -static ssize_t zcache_evicted_eph_zpages;
> -static ssize_t zcache_evicted_eph_pageframes;
> +extern ssize_t zcache_flush_total;
> +extern ssize_t zcache_flush_found;
> +extern ssize_t zcache_flobj_total;
> +extern ssize_t zcache_flobj_found;
> +extern ssize_t zcache_failed_eph_puts;
> +extern ssize_t zcache_failed_pers_puts;
> +extern ssize_t zcache_failed_getfreepages;
> +extern ssize_t zcache_failed_alloc;
> +extern ssize_t zcache_put_to_flush;
> +extern ssize_t zcache_compress_poor;
> +extern ssize_t zcache_mean_compress_poor;
> +extern ssize_t zcache_eph_ate_tail;
> +extern ssize_t zcache_eph_ate_tail_failed;
> +extern ssize_t zcache_pers_ate_eph;
> +extern ssize_t zcache_pers_ate_eph_failed;
> +extern ssize_t zcache_evicted_eph_zpages;
> +extern ssize_t zcache_evicted_eph_pageframes;
>
> extern ssize_t zcache_last_active_file_pageframes;
> extern ssize_t zcache_last_inactive_file_pageframes;
> diff --git a/drivers/staging/zcache/zcache-main.c b/drivers/staging/zcache/zcache-main.c
> index 14dcf8a..e112c1e 100644
> --- a/drivers/staging/zcache/zcache-main.c
> +++ b/drivers/staging/zcache/zcache-main.c
> @@ -145,6 +145,37 @@ ssize_t zcache_pers_zpages;
> u64 zcache_pers_zbytes;
> ssize_t zcache_eph_pageframes;
> ssize_t zcache_pers_pageframes;
> +ssize_t zcache_obj_count;
> +ssize_t zcache_obj_count_max;
> +ssize_t zcache_objnode_count;
> +ssize_t zcache_objnode_count_max;
> +u64 zcache_eph_zbytes;
> +u64 zcache_eph_zbytes_max;
> +u64 zcache_pers_zbytes_max;
> +ssize_t zcache_eph_pageframes_max;
> +ssize_t zcache_pers_pageframes_max;
> +ssize_t zcache_pageframes_alloced;
> +ssize_t zcache_pageframes_freed;
> +ssize_t zcache_eph_zpages;
> +ssize_t zcache_eph_zpages_max;
> +ssize_t zcache_pers_zpages_max;
> +ssize_t zcache_flush_total;
> +ssize_t zcache_flush_found;
> +ssize_t zcache_flobj_total;
> +ssize_t zcache_flobj_found;
> +ssize_t zcache_failed_eph_puts;
> +ssize_t zcache_failed_pers_puts;
> +ssize_t zcache_failed_getfreepages;
> +ssize_t zcache_failed_alloc;
> +ssize_t zcache_put_to_flush;
> +ssize_t zcache_compress_poor;
> +ssize_t zcache_mean_compress_poor;
> +ssize_t zcache_eph_ate_tail;
> +ssize_t zcache_eph_ate_tail_failed;
> +ssize_t zcache_pers_ate_eph;
> +ssize_t zcache_pers_ate_eph_failed;
> +ssize_t zcache_evicted_eph_zpages;
> +ssize_t zcache_evicted_eph_pageframes;
>
> /* Used by this code. */
> ssize_t zcache_last_active_file_pageframes;
> --
> 1.7.7.6
>
> --- a/drivers/staging/zcache/zcache-main.c
> +++ b/drivers/staging/zcache/zcache-main.c
> @@ -176,6 +176,8 @@ ssize_t zcache_pers_ate_eph;
> ssize_t zcache_pers_ate_eph_failed;
> ssize_t zcache_evicted_eph_zpages;
> ssize_t zcache_evicted_eph_pageframes;
> +ssize_t zcache_zero_filled_pages;
> +ssize_t zcache_zero_filled_pages_max;
Is it possible to shove these in the debug.c file? And in debug.h just
have an extern?