Hello,
This series adds support for multiple compression streams.
The main idea is that different compression algorithms have different
characteristics and zram may benefit when it uses a combination of
algorithms: a default algorithm that is faster but have lower
compression rate and a secondary algorithm that can use higher
compression rate at a price of slower compression/decompression.
There are several use-case for this functionality:
- huge pages re-compression: zstd or deflate can successfully
compress huge pages (~50% of huge pages on my synthetic ChromeOS
tests), IOW pages that lzo was not able to compress.
- idle pages re-compression: idle/cold pages sit in the memory
and we may reduce zsmalloc memory usage if we recompress those
idle pages.
User-space has a number of ways to control the behavior
and impact of zram recompression: what type of pages should be
recompressed, size watermarks, etc. Please refer to documentation
patch.
v5:
-- Addressed (very valuable) review feedback from Minchan
-- Extended documentation
-- We now do chain recompression, tryin algos in order of their priority
-- Support up to 4 compression streams
-- Added named parameters to recomp_algorithm and recompress sysfs knobs
-- Cherry-picked patch from Alexey
-- Store algo priority in mete flags
-- Renamed some fo the flags
-- Added incompressible bit to block state output
-- Added incompressible writeback
-- etc.
v4:
-- added IS_ERR_VALUE patch (Andrew)
-- documented SIZE units (bytes) (Andrew)
-- re-phrased writeback BIO error comment (Andrew)
-- return zs_malloc() error code from zram_recompress()
-- do not lose zram_recompress() error in recompress_store()
-- corrected a typo
-- fixed previous rebase errors
-- rebased the series
v3:
-- conditionally reschedule during recompression loop so that
we don't stall RCU grace periods
-- fixed a false-positive WARN_ON
v2:
-- rebased
-- mark completely incompressible pages (neither default nor secondary
algorithm can compress them) with a new flag so that we don't attempt
to recompress them all the time
Alexey Romanov (1):
zram: add size class equals check into recompression
Sergey Senozhatsky (12):
zram: Preparation for multi-zcomp support
zram: Add recompression algorithm sysfs knob
zram: Factor out WB and non-WB zram read functions
zram: Introduce recompress sysfs knob
zram: Add recompress flag to read_block_state()
zram: Clarify writeback_store() comment
zram: Use IS_ERR_VALUE() to check for zs_malloc() errors
zram: remove redundant checks from zram_recompress()
zram: Add algo parameter support to zram_recompress()
documentation: Add zram recompression documentation
zram: add incompressible writeback
zram: Add incompressible flag to read_block_state()
Documentation/admin-guide/blockdev/zram.rst | 100 +++-
drivers/block/zram/Kconfig | 9 +
drivers/block/zram/zcomp.c | 6 +-
drivers/block/zram/zcomp.h | 2 +-
drivers/block/zram/zram_drv.c | 604 +++++++++++++++++---
drivers/block/zram/zram_drv.h | 22 +-
include/linux/zsmalloc.h | 2 +
mm/zsmalloc.c | 21 +
8 files changed, 685 insertions(+), 81 deletions(-)
--
2.38.1.431.g37b22c650d-goog
Size class index comparison is powerful enough so we can
remove object size comparisons.
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
drivers/block/zram/zram_drv.c | 8 ++------
1 file changed, 2 insertions(+), 6 deletions(-)
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 383d967ef4c7..67b58f2255db 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -1725,9 +1725,7 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page,
comp_len_new);
/* Continue until we make progress */
- if (comp_len_new >= huge_class_size ||
- comp_len_new >= comp_len_old ||
- class_index_new >= class_index_old ||
+ if (class_index_new >= class_index_old ||
(threshold && comp_len_new >= threshold)) {
zcomp_stream_put(zram->comps[prio]);
continue;
@@ -1750,9 +1748,7 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page,
* that would save memory, mark the object as incompressible so that
* we will not try to compress it again.
*/
- if (comp_len_new >= huge_class_size ||
- comp_len_new >= comp_len_old ||
- class_index_new >= class_index_old) {
+ if (class_index_new >= class_index_old) {
zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE);
return 0;
}
--
2.38.1.431.g37b22c650d-goog
Recompression iterates through all the registered secondary
compression algorithms in order of their priorities so that
we have higher chances of finding the algorithm that compresses
a particular page. This, however, may not always be best
approach and sometimes we may want to limit recompression to
only one particular algorithm. For instance, when a higher
priority algorithm uses too much power and device has a
relatively low battery level we may want to limit recompression
to use only a lower priority algorithm, which uses less power.
Introduce algo= parameter support to recompression sysfs knob
so that user-sapce can request recompression with particular
algorithm only:
echo "type=idle algo=zstd" > /sys/block/zramX/recompress
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
drivers/block/zram/zram_drv.c | 54 +++++++++++++++++++++++++++++------
drivers/block/zram/zram_drv.h | 1 +
2 files changed, 46 insertions(+), 9 deletions(-)
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 67b58f2255db..89d25f60b33e 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -1677,6 +1677,7 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page,
unsigned int comp_len_new;
unsigned int class_index_old;
unsigned int class_index_new;
+ u32 num_recomps = 0;
void *src, *dst;
int ret;
@@ -1711,6 +1712,7 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page,
if (prio <= zram_get_priority(zram, index))
continue;
+ num_recomps++;
zstrm = zcomp_stream_get(zram->comps[prio]);
src = kmap_atomic(page);
ret = zcomp_compress(zstrm, src, &comp_len_new);
@@ -1743,13 +1745,19 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page,
if (!zstrm)
return 0;
- /*
- * All secondary algorithms failed to re-compress the page in a way
- * that would save memory, mark the object as incompressible so that
- * we will not try to compress it again.
- */
if (class_index_new >= class_index_old) {
- zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE);
+ /*
+ * Secondary algorithms failed to re-compress the page
+ * in a way that would save memory, mark the object as
+ * incompressible so that we will not try to compress
+ * it again.
+ *
+ * We need to make sure that all secondary algorithms have
+ * failed, so we test if the number of recompressions matches
+ * the number of active secondary algorithms.
+ */
+ if (num_recomps == zram->num_active_comps - 1)
+ zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE);
return 0;
}
@@ -1798,10 +1806,11 @@ static ssize_t recompress_store(struct device *dev,
struct device_attribute *attr,
const char *buf, size_t len)
{
+ u32 prio = ZRAM_SECONDARY_COMP, prio_max = ZRAM_MAX_COMPS;
struct zram *zram = dev_to_zram(dev);
- u32 mode = 0, threshold = 0, prio = ZRAM_SECONDARY_COMP;
unsigned long nr_pages = zram->disksize >> PAGE_SHIFT;
- char *args, *param, *val;
+ char *args, *param, *val, *algo = NULL;
+ u32 mode = 0, threshold = 0;
unsigned long index;
struct page *page;
ssize_t ret;
@@ -1833,6 +1842,11 @@ static ssize_t recompress_store(struct device *dev,
return ret;
continue;
}
+
+ if (!strcmp(param, "algo")) {
+ algo = val;
+ continue;
+ }
}
if (threshold >= PAGE_SIZE)
@@ -1844,6 +1858,26 @@ static ssize_t recompress_store(struct device *dev,
goto release_init_lock;
}
+ if (algo) {
+ bool found = false;
+
+ for (; prio < ZRAM_MAX_COMPS; prio++) {
+ if (!zram->comp_algs[prio])
+ continue;
+
+ if (!strcmp(zram->comp_algs[prio], algo)) {
+ prio_max = min(prio + 1, ZRAM_MAX_COMPS);
+ found = true;
+ break;
+ }
+ }
+
+ if (!found) {
+ ret = -EINVAL;
+ goto release_init_lock;
+ }
+ }
+
page = alloc_page(GFP_KERNEL);
if (!page) {
ret = -ENOMEM;
@@ -1874,7 +1908,7 @@ static ssize_t recompress_store(struct device *dev,
goto next;
err = zram_recompress(zram, index, page, threshold,
- prio, ZRAM_MAX_COMPS);
+ prio, prio_max);
next:
zram_slot_unlock(zram, index);
if (err) {
@@ -2110,6 +2144,7 @@ static void zram_destroy_comps(struct zram *zram)
if (!comp)
continue;
zcomp_destroy(comp);
+ zram->num_active_comps--;
}
}
@@ -2177,6 +2212,7 @@ static ssize_t disksize_store(struct device *dev,
}
zram->comps[prio] = comp;
+ zram->num_active_comps++;
}
zram->disksize = disksize;
set_capacity_and_notify(zram->disk, zram->disksize >> SECTOR_SHIFT);
diff --git a/drivers/block/zram/zram_drv.h b/drivers/block/zram/zram_drv.h
index b80faae76835..473325415a74 100644
--- a/drivers/block/zram/zram_drv.h
+++ b/drivers/block/zram/zram_drv.h
@@ -125,6 +125,7 @@ struct zram {
*/
u64 disksize; /* bytes */
const char *comp_algs[ZRAM_MAX_COMPS];
+ s8 num_active_comps;
/*
* zram is claimed so open request will be failed
*/
--
2.38.1.431.g37b22c650d-goog
From: Alexey Romanov <[email protected]>
It makes no sense for us to recompress the object if it will be in the
same size class. We anyway don't get any memory gain. But, at the same
time, we get a CPU time overhead when inserting this object into zspage
and decompressing it afterwards.
[senozhatsky: rebased and fixed conflicts]
Signed-off-by: Alexey Romanov <[email protected]>
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
drivers/block/zram/zram_drv.c | 11 ++++++++++-
include/linux/zsmalloc.h | 2 ++
mm/zsmalloc.c | 21 +++++++++++++++++++++
3 files changed, 33 insertions(+), 1 deletion(-)
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 9561569273fe..383d967ef4c7 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -1675,6 +1675,8 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page,
unsigned long handle_new;
unsigned int comp_len_old;
unsigned int comp_len_new;
+ unsigned int class_index_old;
+ unsigned int class_index_new;
void *src, *dst;
int ret;
@@ -1693,6 +1695,7 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page,
if (ret)
return ret;
+ class_index_old = zs_lookup_class_index(zram->mem_pool, comp_len_old);
/*
* Iterate the secondary comp algorithms list (in order of priority)
* and try to recompress the page.
@@ -1718,9 +1721,13 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page,
return ret;
}
+ class_index_new = zs_lookup_class_index(zram->mem_pool,
+ comp_len_new);
+
/* Continue until we make progress */
if (comp_len_new >= huge_class_size ||
comp_len_new >= comp_len_old ||
+ class_index_new >= class_index_old ||
(threshold && comp_len_new >= threshold)) {
zcomp_stream_put(zram->comps[prio]);
continue;
@@ -1743,7 +1750,9 @@ static int zram_recompress(struct zram *zram, u32 index, struct page *page,
* that would save memory, mark the object as incompressible so that
* we will not try to compress it again.
*/
- if (comp_len_new >= huge_class_size || comp_len_new >= comp_len_old) {
+ if (comp_len_new >= huge_class_size ||
+ comp_len_new >= comp_len_old ||
+ class_index_new >= class_index_old) {
zram_set_flag(zram, index, ZRAM_INCOMPRESSIBLE);
return 0;
}
diff --git a/include/linux/zsmalloc.h b/include/linux/zsmalloc.h
index 2a430e713ce5..a48cd0ffe57d 100644
--- a/include/linux/zsmalloc.h
+++ b/include/linux/zsmalloc.h
@@ -55,5 +55,7 @@ void zs_unmap_object(struct zs_pool *pool, unsigned long handle);
unsigned long zs_get_total_pages(struct zs_pool *pool);
unsigned long zs_compact(struct zs_pool *pool);
+unsigned int zs_lookup_class_index(struct zs_pool *pool, unsigned int size);
+
void zs_pool_stats(struct zs_pool *pool, struct zs_pool_stats *stats);
#endif
diff --git a/mm/zsmalloc.c b/mm/zsmalloc.c
index d03941cace2c..065744b7e9d8 100644
--- a/mm/zsmalloc.c
+++ b/mm/zsmalloc.c
@@ -1205,6 +1205,27 @@ static bool zspage_full(struct size_class *class, struct zspage *zspage)
return get_zspage_inuse(zspage) == class->objs_per_zspage;
}
+/**
+ * zs_lookup_class_index() - Returns index of the zsmalloc &size_class
+ * that hold objects of the provided size.
+ * @pool: zsmalloc pool to use
+ * @size: object size
+ *
+ * Context: Any context.
+ *
+ * Return: the index of the zsmalloc &size_class that hold objects of the
+ * provided size.
+ */
+unsigned int zs_lookup_class_index(struct zs_pool *pool, unsigned int size)
+{
+ struct size_class *class;
+
+ class = pool->size_class[get_size_class_index(size)];
+
+ return class->index;
+}
+EXPORT_SYMBOL_GPL(zs_lookup_class_index);
+
unsigned long zs_get_total_pages(struct zs_pool *pool)
{
return atomic_long_read(&pool->pages_allocated);
--
2.38.1.431.g37b22c650d-goog
Re-phrase writeback BIO error comment.
Reported-by: Andrew Morton <[email protected]>
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
drivers/block/zram/zram_drv.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 560e2932021e..a5d2ce0bcd5d 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -772,8 +772,12 @@ static ssize_t writeback_store(struct device *dev,
zram_clear_flag(zram, index, ZRAM_IDLE);
zram_slot_unlock(zram, index);
/*
- * Return last IO error unless every IO were
- * not suceeded.
+ * BIO errors are not fatal, we continue and simply
+ * attempt to writeback the remaining objects (pages).
+ * At the same time we need to signal user-space that
+ * some writes (at least one, but also could be all of
+ * them) were not successful and we do so by returning
+ * the most recent BIO error.
*/
ret = err;
continue;
--
2.38.1.431.g37b22c650d-goog
Add a new flag to zram block state that shows if the page
is incompressible: that none of the algorithm (including
secondary ones) could compress it.
Suggested-by: Minchan Kim <[email protected]>
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
Documentation/admin-guide/blockdev/zram.rst | 11 +++++++----
drivers/block/zram/zram_drv.c | 6 ++++--
2 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/Documentation/admin-guide/blockdev/zram.rst b/Documentation/admin-guide/blockdev/zram.rst
index f14c8c2e42f3..e4551579cb12 100644
--- a/Documentation/admin-guide/blockdev/zram.rst
+++ b/Documentation/admin-guide/blockdev/zram.rst
@@ -497,10 +497,11 @@ pages of the process with*pagemap.
If you enable the feature, you could see block state via
/sys/kernel/debug/zram/zram0/block_state". The output is as follows::
- 300 75.033841 .wh..
- 301 63.806904 s....
- 302 63.806919 ..hi.
- 303 62.801919 ....r
+ 300 75.033841 .wh...
+ 301 63.806904 s.....
+ 302 63.806919 ..hi..
+ 303 62.801919 ....r.
+ 304 146.781902 ..hi.n
First column
zram's block index.
@@ -519,6 +520,8 @@ Third column
idle page
r:
recompressed page (secondary compression algorithm)
+ n:
+ none (including secondary) of algorithms could compress it
First line of above example says 300th block is accessed at 75.033841sec
and the block's state is huge so it is written back to the backing
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 2e4ef1ba1973..3447df3ca75e 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -949,14 +949,16 @@ static ssize_t read_block_state(struct file *file, char __user *buf,
ts = ktime_to_timespec64(zram->table[index].ac_time);
copied = snprintf(kbuf + written, count,
- "%12zd %12lld.%06lu %c%c%c%c%c\n",
+ "%12zd %12lld.%06lu %c%c%c%c%c%c\n",
index, (s64)ts.tv_sec,
ts.tv_nsec / NSEC_PER_USEC,
zram_test_flag(zram, index, ZRAM_SAME) ? 's' : '.',
zram_test_flag(zram, index, ZRAM_WB) ? 'w' : '.',
zram_test_flag(zram, index, ZRAM_HUGE) ? 'h' : '.',
zram_test_flag(zram, index, ZRAM_IDLE) ? 'i' : '.',
- zram_get_priority(zram, index) ? 'r' : '.');
+ zram_get_priority(zram, index) ? 'r' : '.',
+ zram_test_flag(zram, index,
+ ZRAM_INCOMPRESSIBLE) ? 'n' : '.');
if (count <= copied) {
zram_slot_unlock(zram, index);
--
2.38.1.431.g37b22c650d-goog
Document user-space visible device attributes that
are enabled by ZRAM_MULTI_COMP.
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
Documentation/admin-guide/blockdev/zram.rst | 81 +++++++++++++++++++++
1 file changed, 81 insertions(+)
diff --git a/Documentation/admin-guide/blockdev/zram.rst b/Documentation/admin-guide/blockdev/zram.rst
index 177a142c3146..d898b7ace33d 100644
--- a/Documentation/admin-guide/blockdev/zram.rst
+++ b/Documentation/admin-guide/blockdev/zram.rst
@@ -401,6 +401,87 @@ budget in next setting is user's job.
If admin wants to measure writeback count in a certain period, they could
know it via /sys/block/zram0/bd_stat's 3rd column.
+recompression
+-------------
+
+With CONFIG_ZRAM_MULTI_COMP, zram can recompress pages using alternative
+(secondary) compression algorithms. The basic idea is that alternative
+compression algorithm can provide better compression ratio at a price of
+(potentially) slower compression/decompression speeds. Alternative compression
+algorithm can, for example, be more successful compressing huge pages (those
+that default algorithm failed to compress). Another application is idle pages
+recompression - pages that are cold and sit in the memory can be recompressed
+using more effective algorithm and, hence, reduce zsmalloc memory usage.
+
+With CONFIG_ZRAM_MULTI_COMP, zram supports up to 4 compression algorithms:
+one primary and up to 3 secondary ones. Primary zram compressor is explained
+in "3) Select compression algorithm", secondary algorithms are configured
+using recomp_algorithm device attribute.
+
+Example:::
+
+ #show supported recompression algorithms
+ cat /sys/block/zramX/recomp_algorithm
+ #1: lzo lzo-rle lz4 lz4hc [zstd]
+ #2: lzo lzo-rle lz4 [lz4hc] zstd
+
+Alternative compression algorithms are sorted by priority. In the example
+above, zstd is used as the first alternative algorithm, which has priority
+of 1, while lz4hc is configured as a compression algorithm with priority 2.
+Alternative compression algorithm's priority is provided during algorithms
+configuration:::
+
+ #select zstd recompression algorithm, priority 1
+ echo "algo=zstd priority=1" > /sys/block/zramX/recomp_algorithm
+
+ #select deflate recompression algorithm, priority 2
+ echo "algo=deflate priority=2" > /sys/block/zramX/recomp_algorithm
+
+Another device attribute that CONFIG_ZRAM_MULTI_COMP enables is recompress,
+which controls recompression.
+
+Examples:::
+
+ #IDLE pages recompression is activated by `idle` mode
+ echo "type=idle" > /sys/block/zramX/recompress
+
+ #HUGE pages recompression is activated by `huge` mode
+ echo "type=huge" > /sys/block/zram0/recompress
+
+ #HUGE_IDLE pages recompression is activated by `huge_idle` mode
+ echo "type=huge_idle" > /sys/block/zramX/recompress
+
+The number of idle pages can be significant, so user-space can pass a size
+threshold (in bytes) to the recompress knob: zram will recompress only pages
+of equal or greater size:::
+
+ #recompress all pages larger than 3000 bytes
+ echo "threshold=3000" > /sys/block/zramX/recompress
+
+ #recompress idle pages larger than 2000 bytes
+ echo "type=idle threshold=2000" > /sys/block/zramX/recompress
+
+Recompression of idle pages requires memory tracking.
+
+During re-compression for every page, that matches re-compression criteria,
+ZRAM iterates the list of registered alternative compression algorithms in
+order of their priorities. ZRAM stops either when re-compression was
+successful (re-compressed object is smaller in size than the original one)
+and matches re-compression criteria (e.g. size threshold) or when there are
+no secondary algorithms left to try. If none of the secondary algorithms can
+successfully re-compressed the page such a page is marked as incompressible,
+so ZRAM will not attempt to re-compress it in the future.
+
+This re-compression behaviour, when it iterates through the list of
+registered compression algorithms, increases our chances of finding the
+algorithm that successfully compresses a particular page. Sometimes, however,
+it is convenient (and sometimes even necessary) to limit recompression to
+only one particular algorithm so that it will not try any other algorithms.
+This can be achieved by providing a algo=NAME parameter:::
+
+ #use zstd algorithm only (if registered)
+ echo "type=huge algo=zstd" > /sys/block/zramX/recompress
+
memory tracking
===============
--
2.38.1.431.g37b22c650d-goog
Add support for incompressible pages writeback:
echo incompressible > /sys/block/zramX/writeback
Signed-off-by: Sergey Senozhatsky <[email protected]>
---
Documentation/admin-guide/blockdev/zram.rst | 7 ++++++-
drivers/block/zram/zram_drv.c | 18 ++++++++++++------
2 files changed, 18 insertions(+), 7 deletions(-)
diff --git a/Documentation/admin-guide/blockdev/zram.rst b/Documentation/admin-guide/blockdev/zram.rst
index d898b7ace33d..f14c8c2e42f3 100644
--- a/Documentation/admin-guide/blockdev/zram.rst
+++ b/Documentation/admin-guide/blockdev/zram.rst
@@ -348,8 +348,13 @@ this can be accomplished with::
echo huge_idle > /sys/block/zramX/writeback
+If a user chooses to writeback only incompressible pages (pages that none of
+algorithms can compress) this can be accomplished with::
+
+ echo incompressible > /sys/block/zramX/writeback
+
If an admin wants to write a specific page in zram device to the backing device,
-they could write a page index into the interface.
+they could write a page index into the interface::
echo "page_index=1251" > /sys/block/zramX/writeback
diff --git a/drivers/block/zram/zram_drv.c b/drivers/block/zram/zram_drv.c
index 89d25f60b33e..2e4ef1ba1973 100644
--- a/drivers/block/zram/zram_drv.c
+++ b/drivers/block/zram/zram_drv.c
@@ -648,10 +648,10 @@ static int read_from_bdev_async(struct zram *zram, struct bio_vec *bvec,
#define PAGE_WB_SIG "page_index="
-#define PAGE_WRITEBACK 0
-#define HUGE_WRITEBACK (1<<0)
-#define IDLE_WRITEBACK (1<<1)
-
+#define PAGE_WRITEBACK 0
+#define HUGE_WRITEBACK (1<<0)
+#define IDLE_WRITEBACK (1<<1)
+#define INCOMPRESSIBLE_WRITEBACK (1<<2)
static ssize_t writeback_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t len)
@@ -672,6 +672,8 @@ static ssize_t writeback_store(struct device *dev,
mode = HUGE_WRITEBACK;
else if (sysfs_streq(buf, "huge_idle"))
mode = IDLE_WRITEBACK | HUGE_WRITEBACK;
+ else if (sysfs_streq(buf, "incompressible"))
+ mode = INCOMPRESSIBLE_WRITEBACK;
else {
if (strncmp(buf, PAGE_WB_SIG, sizeof(PAGE_WB_SIG) - 1))
return -EINVAL;
@@ -734,11 +736,15 @@ static ssize_t writeback_store(struct device *dev,
goto next;
if (mode & IDLE_WRITEBACK &&
- !zram_test_flag(zram, index, ZRAM_IDLE))
+ !zram_test_flag(zram, index, ZRAM_IDLE))
goto next;
if (mode & HUGE_WRITEBACK &&
- !zram_test_flag(zram, index, ZRAM_HUGE))
+ !zram_test_flag(zram, index, ZRAM_HUGE))
goto next;
+ if (mode & INCOMPRESSIBLE_WRITEBACK &&
+ !zram_test_flag(zram, index, ZRAM_INCOMPRESSIBLE))
+ goto next;
+
/*
* Clearing ZRAM_UNDER_WB is duty of caller.
* IOW, zram_free_page never clear it.
--
2.38.1.431.g37b22c650d-goog
On Wed, Nov 09, 2022 at 08:50:34PM +0900, Sergey Senozhatsky wrote:
> Hello,
>
> This series adds support for multiple compression streams.
> The main idea is that different compression algorithms have different
> characteristics and zram may benefit when it uses a combination of
> algorithms: a default algorithm that is faster but have lower
> compression rate and a secondary algorithm that can use higher
> compression rate at a price of slower compression/decompression.
>
> There are several use-case for this functionality:
>
> - huge pages re-compression: zstd or deflate can successfully
> compress huge pages (~50% of huge pages on my synthetic ChromeOS
> tests), IOW pages that lzo was not able to compress.
>
> - idle pages re-compression: idle/cold pages sit in the memory
> and we may reduce zsmalloc memory usage if we recompress those
> idle pages.
>
> User-space has a number of ways to control the behavior
> and impact of zram recompression: what type of pages should be
> recompressed, size watermarks, etc. Please refer to documentation
> patch.
>
> v5:
> -- Addressed (very valuable) review feedback from Minchan
> -- Extended documentation
> -- We now do chain recompression, tryin algos in order of their priority
> -- Support up to 4 compression streams
> -- Added named parameters to recomp_algorithm and recompress sysfs knobs
> -- Cherry-picked patch from Alexey
> -- Store algo priority in mete flags
> -- Renamed some fo the flags
> -- Added incompressible bit to block state output
> -- Added incompressible writeback
> -- etc.
>
> v4:
> -- added IS_ERR_VALUE patch (Andrew)
> -- documented SIZE units (bytes) (Andrew)
> -- re-phrased writeback BIO error comment (Andrew)
> -- return zs_malloc() error code from zram_recompress()
> -- do not lose zram_recompress() error in recompress_store()
> -- corrected a typo
> -- fixed previous rebase errors
> -- rebased the series
>
> v3:
> -- conditionally reschedule during recompression loop so that
> we don't stall RCU grace periods
> -- fixed a false-positive WARN_ON
>
> v2:
> -- rebased
> -- mark completely incompressible pages (neither default nor secondary
> algorithm can compress them) with a new flag so that we don't attempt
> to recompress them all the time
>
> Alexey Romanov (1):
> zram: add size class equals check into recompression
>
> Sergey Senozhatsky (12):
> zram: Preparation for multi-zcomp support
> zram: Add recompression algorithm sysfs knob
> zram: Factor out WB and non-WB zram read functions
> zram: Introduce recompress sysfs knob
> zram: Add recompress flag to read_block_state()
> zram: Clarify writeback_store() comment
> zram: Use IS_ERR_VALUE() to check for zs_malloc() errors
> zram: remove redundant checks from zram_recompress()
> zram: Add algo parameter support to zram_recompress()
> documentation: Add zram recompression documentation
> zram: add incompressible writeback
> zram: Add incompressible flag to read_block_state()
>
> Documentation/admin-guide/blockdev/zram.rst | 100 +++-
> drivers/block/zram/Kconfig | 9 +
> drivers/block/zram/zcomp.c | 6 +-
> drivers/block/zram/zcomp.h | 2 +-
> drivers/block/zram/zram_drv.c | 604 +++++++++++++++++---
> drivers/block/zram/zram_drv.h | 22 +-
> include/linux/zsmalloc.h | 2 +
> mm/zsmalloc.c | 21 +
> 8 files changed, 685 insertions(+), 81 deletions(-)
Acked-by: Minchan Kim <[email protected]>
Thanks!