From: SeongJae Park <[email protected]>
Effects of DAMON and DAMON-based Operation Schemes highly depends on the
configurations. Wrong configurations could even result in unexpected
efficiency degradations. For finding a best configuration, repeating
incremental configuration changes and results measurements, in other
words, online tuning, could be helpful.
Nevertheless, DAMON kernel API supports only restrictive online tuning.
Worse yet, the sysfs-based DAMON user interface doesn't support online
tuning at all. DAMON_RECLAIM also doesn't support online tuning.
This patchset makes the DAMON kernel API, DAMON sysfs interface, and
DAMON_RECLAIM supports online tuning.
Sequence of patches
-------------------
First two patches enhance DAMON online tuning for kernel API users.
Specifically, patch 1 let kernel API users to be able to do DAMON online
tuning without a restriction, and patch 2 makes error handling easier.
Following seven patches (patches 3-9) refactor code for better
readability and easier reuse of code fragments that will be useful for
online tuning support.
Patch 10 introduces DAMON callback based user request handling structure
for DAMON sysfs interface, and patch 11 enables DAMON online tuning via
DAMON sysfs interface. Documentation patch (patch 12) for usage of it
follows.
Patch 13 enables online tuning of DAMON_RECLAIM and finally patch 14
documents the DAMON_RECLAIM online tuning usage.
SeongJae Park (14):
mm/damon/core: add a new callback for watermarks checks
mm/damon/core: finish kdamond as soon as any callback returns an error
mm/damon/vaddr: generalize damon_va_apply_three_regions()
mm/damon/vaddr: move 'damon_set_regions()' to core
mm/damon/vaddr: remove damon_va_apply_three_regions()
mm/damon/sysfs: prohibit multiple physical address space monitoring
targets
mm/damon/sysfs: move targets setup code to a separated function
mm/damon/sysfs: reuse damon_set_regions() for regions setting
mm/damon/sysfs: use enum for 'state' input handling
mm/damon/sysfs: update schemes stat in the kdamond context
mm/damon/sysfs: support online inputs update
Docs/{ABI,admin-guide}/damon: Update for 'state' sysfs file input
keyword, 'commit'
mm/damon/reclaim: support online inputs update
Docs/admin-guide/mm/damon/reclaim: document 'commit_inputs' parameter
.../ABI/testing/sysfs-kernel-mm-damon | 7 +-
.../admin-guide/mm/damon/reclaim.rst | 11 +
Documentation/admin-guide/mm/damon/usage.rst | 9 +-
include/linux/damon.h | 9 +
mm/damon/core.c | 89 +++-
mm/damon/reclaim.c | 95 +++--
mm/damon/sysfs.c | 382 +++++++++++++++---
mm/damon/vaddr-test.h | 6 +-
mm/damon/vaddr.c | 73 +---
9 files changed, 500 insertions(+), 181 deletions(-)
--
2.25.1
From: SeongJae Park <[email protected]>
This commit documents the newly added 'state' sysfs file input keyword,
'commit', which allows online tuning of DAMON contexts.
Signed-off-by: SeongJae Park <[email protected]>
---
Documentation/ABI/testing/sysfs-kernel-mm-damon | 7 ++++---
Documentation/admin-guide/mm/damon/usage.rst | 9 +++++----
2 files changed, 9 insertions(+), 7 deletions(-)
diff --git a/Documentation/ABI/testing/sysfs-kernel-mm-damon b/Documentation/ABI/testing/sysfs-kernel-mm-damon
index fab97ea22569..08b9df323560 100644
--- a/Documentation/ABI/testing/sysfs-kernel-mm-damon
+++ b/Documentation/ABI/testing/sysfs-kernel-mm-damon
@@ -23,9 +23,10 @@ Date: Mar 2022
Contact: SeongJae Park <[email protected]>
Description: Writing 'on' or 'off' to this file makes the kdamond starts or
stops, respectively. Reading the file returns the keywords
- based on the current status. Writing 'update_schemes_stats' to
- the file updates contents of schemes stats files of the
- kdamond.
+ based on the current status. Writing 'commit' to this file
+ makes the kdamond reads the user inputs in the sysfs files
+ except 'state' again. Writing 'update_schemes_stats' to the
+ file updates contents of schemes stats files of the kdamond.
What: /sys/kernel/mm/damon/admin/kdamonds/<K>/pid
Date: Mar 2022
diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin-guide/mm/damon/usage.rst
index 9c67311a79d8..1bb7b72414b2 100644
--- a/Documentation/admin-guide/mm/damon/usage.rst
+++ b/Documentation/admin-guide/mm/damon/usage.rst
@@ -121,10 +121,11 @@ In each kdamond directory, two files (``state`` and ``pid``) and one directory
Reading ``state`` returns ``on`` if the kdamond is currently running, or
``off`` if it is not running. Writing ``on`` or ``off`` makes the kdamond be
-in the state. Writing ``update_schemes_stats`` to ``state`` file updates the
-contents of stats files for each DAMON-based operation scheme of the kdamond.
-For details of the stats, please refer to :ref:`stats section
-<sysfs_schemes_stats>`.
+in the state. Writing ``commit`` to the ``state`` file makes kdamond reads the
+user inputs in the sysfs files except ``state`` file again. Writing
+``update_schemes_stats`` to ``state`` file updates the contents of stats files
+for each DAMON-based operation scheme of the kdamond. For details of the
+stats, please refer to :ref:`stats section <sysfs_schemes_stats>`.
If the state is ``on``, reading ``pid`` shows the pid of the kdamond thread.
--
2.25.1
From: SeongJae Park <[email protected]>
When 'after_sampling()' or 'after_aggregation()' DAMON callbacks return
an error, kdamond continues the remaining loop once. It makes no much
sense to run the remaining part while something wrong already happened.
The context might be corrupted or having invalid data. This commit
therefore makes kdamond skips the remaining works and immediately finish
in the cases.
Signed-off-by: SeongJae Park <[email protected]>
---
mm/damon/core.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/mm/damon/core.c b/mm/damon/core.c
index e28fbc3a1969..18c08e90563e 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -1141,8 +1141,10 @@ static int kdamond_fn(void *data)
if (ctx->ops.prepare_access_checks)
ctx->ops.prepare_access_checks(ctx);
if (ctx->callback.after_sampling &&
- ctx->callback.after_sampling(ctx))
+ ctx->callback.after_sampling(ctx)) {
done = true;
+ continue;
+ }
kdamond_usleep(ctx->sample_interval);
@@ -1154,8 +1156,10 @@ static int kdamond_fn(void *data)
max_nr_accesses / 10,
sz_limit);
if (ctx->callback.after_aggregation &&
- ctx->callback.after_aggregation(ctx))
+ ctx->callback.after_aggregation(ctx)) {
done = true;
+ continue;
+ }
kdamond_apply_schemes(ctx);
kdamond_reset_aggregated(ctx);
kdamond_split_regions(ctx);
--
2.25.1
From: SeongJae Park <[email protected]>
This commit moves 'damon_set_regions()' from vaddr to core, as it is
aimed to be used by not only 'vaddr' but also other parts of DAMON.
Signed-off-by: SeongJae Park <[email protected]>
---
include/linux/damon.h | 2 ++
mm/damon/core.c | 73 +++++++++++++++++++++++++++++++++++++++++++
mm/damon/vaddr.c | 73 -------------------------------------------
3 files changed, 75 insertions(+), 73 deletions(-)
diff --git a/include/linux/damon.h b/include/linux/damon.h
index 6cb5ab5d8e9d..d1e6ee28a2ff 100644
--- a/include/linux/damon.h
+++ b/include/linux/damon.h
@@ -494,6 +494,8 @@ static inline void damon_insert_region(struct damon_region *r,
void damon_add_region(struct damon_region *r, struct damon_target *t);
void damon_destroy_region(struct damon_region *r, struct damon_target *t);
+int damon_set_regions(struct damon_target *t, struct damon_addr_range *ranges,
+ unsigned int nr_ranges);
struct damos *damon_new_scheme(
unsigned long min_sz_region, unsigned long max_sz_region,
diff --git a/mm/damon/core.c b/mm/damon/core.c
index 18c08e90563e..ebfe572f6a85 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -167,6 +167,79 @@ void damon_destroy_region(struct damon_region *r, struct damon_target *t)
damon_free_region(r);
}
+/*
+ * Check whether a region is intersecting an address range
+ *
+ * Returns true if it is.
+ */
+static bool damon_intersect(struct damon_region *r,
+ struct damon_addr_range *re)
+{
+ return !(r->ar.end <= re->start || re->end <= r->ar.start);
+}
+
+/*
+ * damon_set_regions() - Set regions of a target for given address ranges.
+ * @t: the given target.
+ * @ranges: array of new monitoring target ranges.
+ * @nr_ranges: length of @ranges.
+ *
+ * This function adds new regions to, or modify existing regions of a
+ * monitoring target to fit in specific ranges.
+ *
+ * Return: 0 if success, or negative error code otherwise.
+ */
+int damon_set_regions(struct damon_target *t, struct damon_addr_range *ranges,
+ unsigned int nr_ranges)
+{
+ struct damon_region *r, *next;
+ unsigned int i;
+
+ /* Remove regions which are not in the new ranges */
+ damon_for_each_region_safe(r, next, t) {
+ for (i = 0; i < nr_ranges; i++) {
+ if (damon_intersect(r, &ranges[i]))
+ break;
+ }
+ if (i == nr_ranges)
+ damon_destroy_region(r, t);
+ }
+
+ /* Add new regions or resize existing regions to fit in the ranges */
+ for (i = 0; i < nr_ranges; i++) {
+ struct damon_region *first = NULL, *last, *newr;
+ struct damon_addr_range *range;
+
+ range = &ranges[i];
+ /* Get the first/last regions intersecting with the range */
+ damon_for_each_region(r, t) {
+ if (damon_intersect(r, range)) {
+ if (!first)
+ first = r;
+ last = r;
+ }
+ if (r->ar.start >= range->end)
+ break;
+ }
+ if (!first) {
+ /* no region intersects with this range */
+ newr = damon_new_region(
+ ALIGN_DOWN(range->start,
+ DAMON_MIN_REGION),
+ ALIGN(range->end, DAMON_MIN_REGION));
+ if (!newr)
+ return -ENOMEM;
+ damon_insert_region(newr, damon_prev_region(r), r, t);
+ } else {
+ /* resize intersecting regions to fit in this range */
+ first->ar.start = ALIGN_DOWN(range->start,
+ DAMON_MIN_REGION);
+ last->ar.end = ALIGN(range->end, DAMON_MIN_REGION);
+ }
+ }
+ return 0;
+}
+
struct damos *damon_new_scheme(
unsigned long min_sz_region, unsigned long max_sz_region,
unsigned int min_nr_accesses, unsigned int max_nr_accesses,
diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
index 6879dd7a6ca5..c1ddd598f087 100644
--- a/mm/damon/vaddr.c
+++ b/mm/damon/vaddr.c
@@ -286,79 +286,6 @@ static void damon_va_init(struct damon_ctx *ctx)
* Functions for the dynamic monitoring target regions update
*/
-/*
- * Check whether a region is intersecting an address range
- *
- * Returns true if it is.
- */
-static bool damon_intersect(struct damon_region *r,
- struct damon_addr_range *re)
-{
- return !(r->ar.end <= re->start || re->end <= r->ar.start);
-}
-
-/*
- * damon_set_regions() - Set regions of a target for given address ranges.
- * @t: the given target.
- * @ranges: array of new monitoring target ranges.
- * @nr_ranges: length of @ranges.
- *
- * This function adds new regions to, or modify existing regions of a
- * monitoring target to fit in specific ranges.
- *
- * Return: 0 if success, or negative error code otherwise.
- */
-static int damon_set_regions(struct damon_target *t,
- struct damon_addr_range *ranges, unsigned int nr_ranges)
-{
- struct damon_region *r, *next;
- unsigned int i;
-
- /* Remove regions which are not in the new ranges */
- damon_for_each_region_safe(r, next, t) {
- for (i = 0; i < nr_ranges; i++) {
- if (damon_intersect(r, &ranges[i]))
- break;
- }
- if (i == nr_ranges)
- damon_destroy_region(r, t);
- }
-
- /* Add new regions or resize existing regions to fit in the ranges */
- for (i = 0; i < nr_ranges; i++) {
- struct damon_region *first = NULL, *last, *newr;
- struct damon_addr_range *range;
-
- range = &ranges[i];
- /* Get the first/last regions intersecting with the range */
- damon_for_each_region(r, t) {
- if (damon_intersect(r, range)) {
- if (!first)
- first = r;
- last = r;
- }
- if (r->ar.start >= range->end)
- break;
- }
- if (!first) {
- /* no region intersects with this range */
- newr = damon_new_region(
- ALIGN_DOWN(range->start,
- DAMON_MIN_REGION),
- ALIGN(range->end, DAMON_MIN_REGION));
- if (!newr)
- return -ENOMEM;
- damon_insert_region(newr, damon_prev_region(r), r, t);
- } else {
- /* resize intersecting regions to fit in this range */
- first->ar.start = ALIGN_DOWN(range->start,
- DAMON_MIN_REGION);
- last->ar.end = ALIGN(range->end, DAMON_MIN_REGION);
- }
- }
- return 0;
-}
-
/*
* Update damon regions for the three big regions of the given target
*
--
2.25.1
From: SeongJae Park <[email protected]>
Currently, DAMON sysfs interface doesn't provide a way for adjusting
DAMON input parameters while it is turned on. Therefore, users who want
to reconfigure DAMON need to stop DAMON and restart. This means all the
monitoring results that accumulated so far, which could be useful,
should be flushed. This would be inefficient for many cases.
For an example, let's suppose a sysadmin was running a DAMON-based
Operation Scheme to find memory regions not accessed for more than 5
mins and page out the regions. If it turns out the 5 mins threshold was
too long and therefore the sysadmin wants to reduce it to 4 mins, the
sysadmin should turn off DAMON, restart it, and wait for at least 4 more
minutes so that DAMON can find the cold memory regions, even though
DAMON was knowing there are regions that not accessed for 4 mins at the
time of shutdown.
This commit makes DAMON sysfs interface to support online DAMON input
parameters updates by adding a new input keyword for the 'state' DAMON
sysfs file, 'commit'. Writing the keyword to the 'state' file while the
corresponding kdamond is running makes the kdamond to read the sysfs
file values again and update the DAMON context.
Signed-off-by: SeongJae Park <[email protected]>
---
mm/damon/sysfs.c | 99 +++++++++++++++++++++++++++++++++++++++++++-----
1 file changed, 90 insertions(+), 9 deletions(-)
diff --git a/mm/damon/sysfs.c b/mm/damon/sysfs.c
index 864a215ff809..23030611ae7d 100644
--- a/mm/damon/sysfs.c
+++ b/mm/damon/sysfs.c
@@ -2061,6 +2061,8 @@ enum damon_sysfs_cmd {
DAMON_SYSFS_CMD_ON,
/* @DAMON_SYSFS_CMD_OFF: Turn the kdamond off. */
DAMON_SYSFS_CMD_OFF,
+ /* @DAMON_SYSFS_CMD_COMMIT: Update kdamond inputs. */
+ DAMON_SYSFS_CMD_COMMIT,
/*
* @DAMON_SYSFS_CMD_UPDATE_SCHEMES_STATS: Update scheme stats sysfs
* files.
@@ -2076,6 +2078,7 @@ enum damon_sysfs_cmd {
static const char * const damon_sysfs_cmd_strs[] = {
"on",
"off",
+ "commit",
"update_schemes_stats",
};
@@ -2194,6 +2197,39 @@ static int damon_sysfs_add_target(struct damon_sysfs_target *sys_target,
return err;
}
+/*
+ * Search a target in a context that corresponds to the sysfs target input.
+ *
+ * Return: pointer to the target if found, NULL if not found, or negative
+ * error code if the search failed.
+ */
+static struct damon_target *damon_sysfs_existing_target(
+ struct damon_sysfs_target *sys_target, struct damon_ctx *ctx)
+{
+ struct pid *pid;
+ struct damon_target *t;
+
+ if (ctx->ops.id == DAMON_OPS_PADDR) {
+ /* Up to only one target for paddr could exist */
+ damon_for_each_target(t, ctx)
+ return t;
+ return NULL;
+ }
+
+ /* ops.id should be DAMON_OPS_VADDR or DAMON_OPS_FVADDR */
+ pid = find_get_pid(sys_target->pid);
+ if (!pid)
+ return ERR_PTR(-EINVAL);
+ damon_for_each_target(t, ctx) {
+ if (t->pid == pid) {
+ put_pid(pid);
+ return t;
+ }
+ }
+ put_pid(pid);
+ return NULL;
+}
+
static int damon_sysfs_set_targets(struct damon_ctx *ctx,
struct damon_sysfs_targets *sysfs_targets)
{
@@ -2204,8 +2240,15 @@ static int damon_sysfs_set_targets(struct damon_ctx *ctx,
return -EINVAL;
for (i = 0; i < sysfs_targets->nr; i++) {
- err = damon_sysfs_add_target(
- sysfs_targets->targets_arr[i], ctx);
+ struct damon_sysfs_target *st = sysfs_targets->targets_arr[i];
+ struct damon_target *t = damon_sysfs_existing_target(st, ctx);
+
+ if (IS_ERR(t))
+ return PTR_ERR(t);
+ if (!t)
+ err = damon_sysfs_add_target(st, ctx);
+ else
+ err = damon_sysfs_set_regions(t, st->regions);
if (err)
return err;
}
@@ -2308,6 +2351,48 @@ static int damon_sysfs_upd_schemes_stats(struct damon_sysfs_kdamond *kdamond)
return 0;
}
+static inline bool damon_sysfs_kdamond_running(
+ struct damon_sysfs_kdamond *kdamond)
+{
+ return kdamond->damon_ctx &&
+ damon_sysfs_ctx_running(kdamond->damon_ctx);
+}
+
+/*
+ * damon_sysfs_commit_input() - Commit user inputs to a running kdamond.
+ * @kdamond: The kobject wrapper for the associated kdamond.
+ *
+ * If the sysfs input is wrong, the kdamond will be terminated.
+ */
+static int damon_sysfs_commit_input(struct damon_sysfs_kdamond *kdamond)
+{
+ struct damon_ctx *ctx = kdamond->damon_ctx;
+ struct damon_sysfs_context *sys_ctx;
+ int err = 0;
+
+ if (!damon_sysfs_kdamond_running(kdamond))
+ return -EINVAL;
+ /* TODO: Support multiple contexts per kdamond */
+ if (kdamond->contexts->nr != 1)
+ return -EINVAL;
+
+ sys_ctx = kdamond->contexts->contexts_arr[0];
+
+ err = damon_select_ops(ctx, sys_ctx->ops_id);
+ if (err)
+ return err;
+ err = damon_sysfs_set_attrs(ctx, sys_ctx->attrs);
+ if (err)
+ return err;
+ err = damon_sysfs_set_targets(ctx, sys_ctx->targets);
+ if (err)
+ return err;
+ err = damon_sysfs_set_schemes(ctx, sys_ctx->schemes);
+ if (err)
+ return err;
+ return err;
+}
+
/*
* damon_sysfs_cmd_request_callback() - DAMON callback for handling requests.
* @c: The DAMON context of the callback.
@@ -2330,6 +2415,9 @@ static int damon_sysfs_cmd_request_callback(struct damon_ctx *c)
case DAMON_SYSFS_CMD_UPDATE_SCHEMES_STATS:
err = damon_sysfs_upd_schemes_stats(kdamond);
break;
+ case DAMON_SYSFS_CMD_COMMIT:
+ err = damon_sysfs_commit_input(kdamond);
+ break;
default:
break;
}
@@ -2414,13 +2502,6 @@ static int damon_sysfs_turn_damon_off(struct damon_sysfs_kdamond *kdamond)
*/
}
-static inline bool damon_sysfs_kdamond_running(
- struct damon_sysfs_kdamond *kdamond)
-{
- return kdamond->damon_ctx &&
- damon_sysfs_ctx_running(kdamond->damon_ctx);
-}
-
/*
* damon_sysfs_handle_cmd() - Handle a command for a specific kdamond.
* @cmd: The command to handle.
--
2.25.1