From: SeongJae Park <[email protected]>
Changes from Previous Version (v33)
===================================
Compared to the v33
(https://lore.kernel.org/linux-mm/[email protected]/),
this version contains below minor changes.
- Rebase on latest -mm tree (v5.14-rc1-mmots-2021-07-15-18-47)
- Remove unnecessary asterisks from the MAINTAINERS file update (Joe Perches)
Now all patches of this patchset has at least one 'Reviewed-by:' or 'Acked-by:'
tags. Andrew, could you please consider merging this into the -mm tree?
Introduction
============
DAMON is a data access monitoring framework for the Linux kernel. The core
mechanisms of DAMON called 'region based sampling' and 'adaptive regions
adjustment' (refer to 'mechanisms.rst' in the 11th patch of this patchset for
the detail) make it
- accurate (The monitored information is useful for DRAM level memory
management. It might not appropriate for Cache-level accuracy, though.),
- light-weight (The monitoring overhead is low enough to be applied online
while making no impact on the performance of the target workloads.), and
- scalable (the upper-bound of the instrumentation overhead is controllable
regardless of the size of target workloads.).
Using this framework, therefore, several memory management mechanisms such as
reclamation and THP can be optimized to aware real data access patterns.
Experimental access pattern aware memory management optimization works that
incurring high instrumentation overhead will be able to have another try.
Though DAMON is for kernel subsystems, it can be easily exposed to the user
space by writing a DAMON-wrapper kernel subsystem. Then, user space users who
have some special workloads will be able to write personalized tools or
applications for deeper understanding and specialized optimizations of their
systems.
DAMON is also merged in two public Amazon Linux kernel trees that based on
v5.4.y[1] and v5.10.y[2].
[1] https://github.com/amazonlinux/linux/tree/amazon-5.4.y/master/mm/damon
[2] https://github.com/amazonlinux/linux/tree/amazon-5.10.y/master/mm/damon
Long-term Plan
--------------
DAMON is a part of a project called Data Access-aware Operating System (DAOS).
As the name implies, I want to improve the performance and efficiency of
systems using fine-grained data access patterns. The optimizations are for
both kernel and user spaces. I will therefore modify or create kernel
subsystems, export some of those to user space and implement user space library
/ tools. Below shows the layers and components for the project.
---------------------------------------------------------------------------
Primitives: PTE Accessed bit, PG_idle, rmap, (Intel CMT), ...
Framework: DAMON
Features: DAMOS, virtual addr, physical addr, ...
Applications: DAMON-debugfs, (DARC), ...
^^^^^^^^^^^^^^^^^^^^^^^ KERNEL SPACE ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Raw Interface: debugfs, (sysfs), (damonfs), tracepoints, (sys_damon), ...
vvvvvvvvvvvvvvvvvvvvvvv USER SPACE vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
Library: (libdamon), ...
Tools: DAMO, (perf), ...
---------------------------------------------------------------------------
The components in parentheses or marked as '...' are not implemented yet but in
the future plan. IOW, those are the TODO tasks of DAOS project. For more
detail, please refer to the plans:
https://lore.kernel.org/linux-mm/[email protected]/
Evaluations
===========
We evaluated DAMON's overhead, monitoring quality and usefulness using 24
realistic workloads on my QEMU/KVM based virtual machine running a kernel that
v24 DAMON patchset is applied.
DAMON is lightweight. It increases system memory usage by 0.39% and slows
target workloads down by 1.16%.
DAMON is accurate and useful for memory management optimizations. An
experimental DAMON-based operation scheme for THP, namely 'ethp', removes
76.15% of THP memory overheads while preserving 51.25% of THP speedup. Another
experimental DAMON-based 'proactive reclamation' implementation, 'prcl',
reduces 93.38% of residential sets and 23.63% of system memory footprint while
incurring only 1.22% runtime overhead in the best case (parsec3/freqmine).
NOTE that the experimental THP optimization and proactive reclamation are not
for production but only for proof of concepts.
Please refer to the official document[1] or "Documentation/admin-guide/mm: Add
a document for DAMON" patch in this patchset for detailed evaluation setup and
results.
[1] https://damonitor.github.io/doc/html/latest-damon/admin-guide/mm/damon/eval.html
Real-world User Story
=====================
In summary, DAMON has used on production systems and proved its usefulness.
DAMON as a profiler
-------------------
We analyzed characteristics of a large scale production systems of our
customers using DAMON. The systems utilize 70GB DRAM and 36 CPUs. From this,
we were able to find interesting things below.
There were obviously different access pattern under idle workload and active
workload. Under the idle workload, it accessed large memory regions with low
frequency, while the active workload accessed small memory regions with high
freuqnecy.
DAMON found a 7GB memory region that showing obviously high access frequency
under the active workload. We believe this is the performance-effective
working set and need to be protected.
There was a 4KB memory region that showing highest access frequency under not
only active but also idle workloads. We think this must be a hottest code
section like thing that should never be paged out.
For this analysis, DAMON used only 0.3-1% of single CPU time. Because we used
recording-based analysis, it consumed about 3-12 MB of disk space per 20
minutes. This is only small amount of disk space, but we can further reduce
the disk usage by using non-recording-based DAMON features. I'd like to argue
that only DAMON can do such detailed analysis (finding 4KB highest region in
70GB memory) with the light overhead.
DAMON as a system optimization tool
-----------------------------------
We also found below potential performance problems on the systems and made
DAMON-based solutions.
The system doesn't want to make the workload suffer from the page reclamation
and thus it utilizes enough DRAM but no swap device. However, we found the
system is actively reclaiming file-backed pages, because the system has
intensive file IO. The file IO turned out to be not performance critical for
the workload, but the customer wanted to ensure performance critical
file-backed pages like code section to not mistakenly be evicted.
Using direct IO should or `mlock()` would be a straightforward solution, but
modifying the user space code is not easy for the customer. Alternatively, we
could use DAMON-based operation scheme[1]. By using it, we can ask DAMON to
track access frequency of each region and make
'process_madvise(MADV_WILLNEED)[2]' call for regions having specific size and
access frequency for a time interval.
We also found the system is having high number of TLB misses. We tried
'always' THP enabled policy and it greatly reduced TLB misses, but the page
reclamation also been more frequent due to the THP internal fragmentation
caused memory bloat. We could try another DAMON-based operation scheme that
applies 'MADV_HUGEPAGE' to memory regions having >=2MB size and high access
frequency, while applying 'MADV_NOHUGEPAGE' to regions having <2MB size and low
access frequency.
We do not own the systems so we only reported the analysis results and possible
optimization solutions to the customers. The customers satisfied about the
analysis results and promised to try the optimization guides.
[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-api/[email protected]/
Comparison with Idle Page Tracking
==================================
Idle Page Tracking allows users to set and read idleness of pages using a
bitmap file which represents each page with each bit of the file. One
recommended usage of it is working set size detection. Users can do that by
1. find PFN of each page for workloads in interest,
2. set all the pages as idle by doing writes to the bitmap file,
3. wait until the workload accesses its working set, and
4. read the idleness of the pages again and count pages became not idle.
NOTE: While Idle Page Tracking is for user space users, DAMON is primarily
designed for kernel subsystems though it can easily exposed to the user space.
Hence, this section only assumes such user space use of DAMON.
For what use cases Idle Page Tracking would be better?
------------------------------------------------------
1. Flexible usecases other than hotness monitoring.
Because Idle Page Tracking allows users to control the primitive (Page
idleness) by themselves, Idle Page Tracking users can do anything they want.
Meanwhile, DAMON is primarily designed to monitor the hotness of each memory
region. For this, DAMON asks users to provide sampling interval and
aggregation interval. For the reason, there could be some use case that using
Idle Page Tracking is simpler.
2. Physical memory monitoring.
Idle Page Tracking receives PFN range as input, so natively supports physical
memory monitoring.
DAMON is designed to be extensible for multiple address spaces and use cases by
implementing and using primitives for the given use case. Therefore, by
theory, DAMON has no limitation in the type of target address space as long as
primitives for the given address space exists. However, the default primitives
introduced by this patchset supports only virtual address spaces.
Therefore, for physical memory monitoring, you should implement your own
primitives and use it, or simply use Idle Page Tracking.
Nonetheless, RFC patchsets[1] for the physical memory address space primitives
is already available. It also supports user memory same to Idle Page Tracking.
[1] https://lore.kernel.org/linux-mm/[email protected]/
For what use cases DAMON is better?
-----------------------------------
1. Hotness Monitoring.
Idle Page Tracking let users know only if a page frame is accessed or not. For
hotness check, the user should write more code and use more memory. DAMON do
that by itself.
2. Low Monitoring Overhead
DAMON receives user's monitoring request with one step and then provide the
results. So, roughly speaking, DAMON require only O(1) user/kernel context
switches.
In case of Idle Page Tracking, however, because the interface receives
contiguous page frames, the number of user/kernel context switches increases as
the monitoring target becomes complex and huge. As a result, the context
switch overhead could be not negligible.
Moreover, DAMON is born to handle with the monitoring overhead. Because the
core mechanism is pure logical, Idle Page Tracking users might be able to
implement the mechanism on thier own, but it would be time consuming and the
user/kernel context switching will still more frequent than that of DAMON.
Also, the kernel subsystems cannot use the logic in this case.
3. Page granularity working set size detection.
Until v22 of this patchset, this was categorized as the thing Idle Page
Tracking could do better, because DAMON basically maintains additional metadata
for each of the monitoring target regions. So, in the page granularity working
set size detection use case, DAMON would incur (number of monitoring target
pages * size of metadata) memory overhead. Size of the single metadata item is
about 54 bytes, so assuming 4KB pages, about 1.3% of monitoring target pages
will be additionally used.
All essential metadata for Idle Page Tracking are embedded in 'struct page' and
page table entries. Therefore, in this use case, only one counter variable for
working set size accounting is required if Idle Page Tracking is used.
There are more details to consider, but roughly speaking, this is true in most
cases.
However, the situation changed from v23. Now DAMON supports arbitrary types of
monitoring targets, which don't use the metadata. Using that, DAMON can do the
working set size detection with no additional space overhead but less
user-kernel context switch. A first draft for the implementation of monitoring
primitives for this usage is available in a DAMON development tree[1]. An RFC
patchset for it based on this patchset will also be available soon.
From v24, the arbitrary type support is dropped from this patchset because this
patchset doesn't introduce real use of the type. You can still get it from the
DAMON development tree[2], though.
[1] https://github.com/sjp38/linux/tree/damon/pgidle_hack
[2] https://github.com/sjp38/linux/tree/damon/master
4. More future usecases
While Idle Page Tracking has tight coupling with base primitives (PG_Idle and
page table Accessed bits), DAMON is designed to be extensible for many use
cases and address spaces. If you need some special address type or want to use
special h/w access check primitives, you can write your own primitives for that
and configure DAMON to use those. Therefore, if your use case could be changed
a lot in future, using DAMON could be better.
Can I use both Idle Page Tracking and DAMON?
--------------------------------------------
Yes, though using them concurrently for overlapping memory regions could result
in interference to each other. Nevertheless, such use case would be rare or
makes no sense at all. Even in the case, the noise would bot be really
significant. So, you can choose whatever you want depending on the
characteristics of your use cases.
More Information
================
We prepared a showcase web site[1] that you can get more information. There
are
- the official documentations[2],
- the heatmap format dynamic access pattern of various realistic workloads for
heap area[3], mmap()-ed area[4], and stack[5] area,
- the dynamic working set size distribution[6] and chronological working set
size changes[7], and
- the latest performance test results[8].
[1] https://damonitor.github.io/_index
[2] https://damonitor.github.io/doc/html/latest-damon
[3] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.0.png.html
[4] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.1.png.html
[5] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.2.png.html
[6] https://damonitor.github.io/test/result/visual/latest/rec.wss_sz.png.html
[7] https://damonitor.github.io/test/result/visual/latest/rec.wss_time.png.html
[8] https://damonitor.github.io/test/result/perf/latest/html/index.html
Baseline and Complete Git Trees
===============================
The patches are based on the latest -mm tree, specifically
v5.14-rc1-mmots-2021-07-15-18-47 of https://github.com/hnaz/linux-mm. You can
also clone the complete git tree:
$ git clone git://github.com/sjp38/linux -b damon/patches/v34
The web is also available:
https://github.com/sjp38/linux/releases/tag/damon/patches/v34
Development Trees
-----------------
There are a couple of trees for entire DAMON patchset series and
features for future release.
- For latest release: https://github.com/sjp38/linux/tree/damon/master
- For next release: https://github.com/sjp38/linux/tree/damon/next
Long-term Support Trees
-----------------------
For people who want to test DAMON but using LTS kernels, there are another
couple of trees based on two latest LTS kernels respectively and containing the
'damon/master' backports.
- For v5.4.y: https://github.com/sjp38/linux/tree/damon/for-v5.4.y
- For v5.10.y: https://github.com/sjp38/linux/tree/damon/for-v5.10.y
Amazon Linux Kernel Trees
-------------------------
DAMON is also merged in two public Amazon Linux kernel trees that based on
v5.4.y[1] and v5.10.y[2].
[1] https://github.com/amazonlinux/linux/tree/amazon-5.4.y/master/mm/damon
[2] https://github.com/amazonlinux/linux/tree/amazon-5.10.y/master/mm/damon
Git Tree for Diff of Patches
============================
For easy review of diff between different versions of each patch, I prepared a
git tree containing all versions of the DAMON patchset series:
https://github.com/sjp38/damon-patches
You can clone it and use 'diff' for easy review of changes between different
versions of the patchset. For example:
$ git clone https://github.com/sjp38/damon-patches && cd damon-patches
$ diff -u damon/v33 damon/v34
Sequence Of Patches
===================
First three patches implement the core logics of DAMON. The 1st patch
introduces basic sampling based hotness monitoring for arbitrary types of
targets. Following two patches implement the core mechanisms for control of
overhead and accuracy, namely regions based sampling (patch 2) and adaptive
regions adjustment (patch 3).
Now the essential parts of DAMON is complete, but it cannot work unless someone
provides monitoring primitives for a specific use case. The following two
patches make it just work for virtual address spaces monitoring. The 4th patch
makes 'PG_idle' can be used by DAMON and the 5th patch implements the virtual
memory address space specific monitoring primitives using page table Accessed
bits and the 'PG_idle' page flag.
Now DAMON just works for virtual address space monitoring via the kernel space
api. To let the user space users can use DAMON, following four patches add
interfaces for them. The 6th patch adds a tracepoint for monitoring results.
The 7th patch implements a DAMON application kernel module, namely damon-dbgfs,
that simply wraps DAMON and exposes DAMON interface to the user space via the
debugfs interface. The 8th patch further exports pid of monitoring thread
(kdamond) to user space for easier cpu usage accounting, and the 9th patch
makes the debugfs interface to support multiple contexts.
Three patches for maintainability follows. The 10th patch adds documentations
for both the user space and the kernel space. The 11th patch provides unit
tests (based on the kunit) while the 12th patch adds user space tests (based on
the kselftest).
Finally, the last patch (13th) updates the MAINTAINERS file.
Patch History
=============
Changes from v33
(https://lore.kernel.org/linux-mm/[email protected]/),
- Rebase on latest -mm tree (v5.14-rc1-mmots-2021-07-15-18-47)
- Remove unnecessary asterisks from the MAINTAINERS file update (Joe Perches)
Changes from v32
(https://lore.kernel.org/linux-mm/[email protected]/)
- Rebase on latest mainline (7d0fc5c62385)
- Collect 'Acked-by:' tags from Shakeel Butt
Chages from v31
(https://lore.kernel.org/linux-mm/[email protected]/)
- Rebase on latest -mm tree (v5.13-rc7-mmots-2021-06-24-20-54)
- Add 'Acked-by:' tags from Shakeel Butt
- Use 'kthread_run()' (Shakeel Butt)
- Change default 'update_interval' to 60 seconds (Shakeel Butt)
- Utilize 'nr_regions' field in each 'damon_target' object (Shakeel Butt)
- Remove unused parameters in some functions (Shakeel Butt)
- Use variable name 'ctx' for 'damon_ctx' (Shakeel Butt)
- Make 'dbgfs' to completely manage pid reference counting (Shakeel Butt)
- Remove '.owner' of debugfs files (Shakeel Butt)
Changes from v30
(https://lore.kernel.org/linux-mm/[email protected]/)
- Rebase on latest -mm tree (v5.13-rc6-mmots-2021-06-16-22-17)
- selftest: Fix wrong file content comparison (Markus Boehme)
- Collect 'Reviewed-by:' tags from Markus
Changes from v29
(https://lore.kernel.org/linux-mm/[email protected]/)
- Rebase on latest -mm tree (v5.13-rc6-mmots-2021-06-15-20-28)
- Remove unnecessary documents
- Wordsmith commit message for PAGE_IDLE separation (Amit Shah)
- selftests: Fix shellcheck warnings and cleanup (Maximilian Heyne)
- Wordsmith the document (Markus Boehme)
- Fix a typo in comments (Fernand Sieber)
- Collect 'Reviewed-by:' tags from "Fernand Sieber <[email protected]>"
Please refer to the v29 patchset to get older history.
SeongJae Park (13):
mm: Introduce Data Access MONitor (DAMON)
mm/damon/core: Implement region-based sampling
mm/damon: Adaptively adjust regions
mm/idle_page_tracking: Make PG_idle reusable
mm/damon: Implement primitives for the virtual memory address spaces
mm/damon: Add a tracepoint
mm/damon: Implement a debugfs-based user space interface
mm/damon/dbgfs: Export kdamond pid to the user space
mm/damon/dbgfs: Support multiple contexts
Documentation: Add documents for DAMON
mm/damon: Add kunit tests
mm/damon: Add user space selftests
MAINTAINERS: Update for DAMON
Documentation/admin-guide/mm/damon/index.rst | 15 +
Documentation/admin-guide/mm/damon/start.rst | 114 +++
Documentation/admin-guide/mm/damon/usage.rst | 112 +++
Documentation/admin-guide/mm/index.rst | 1 +
Documentation/vm/damon/api.rst | 20 +
Documentation/vm/damon/design.rst | 166 ++++
Documentation/vm/damon/faq.rst | 51 ++
Documentation/vm/damon/index.rst | 30 +
Documentation/vm/index.rst | 1 +
MAINTAINERS | 11 +
include/linux/damon.h | 268 +++++++
include/linux/page-flags.h | 4 +-
include/linux/page_ext.h | 2 +-
include/linux/page_idle.h | 6 +-
include/trace/events/damon.h | 43 ++
include/trace/events/mmflags.h | 2 +-
mm/Kconfig | 10 +
mm/Makefile | 1 +
mm/damon/Kconfig | 69 ++
mm/damon/Makefile | 5 +
mm/damon/core-test.h | 253 ++++++
mm/damon/core.c | 720 ++++++++++++++++++
mm/damon/dbgfs-test.h | 126 +++
mm/damon/dbgfs.c | 624 +++++++++++++++
mm/damon/vaddr-test.h | 329 ++++++++
mm/damon/vaddr.c | 613 +++++++++++++++
mm/page_ext.c | 12 +-
mm/page_idle.c | 10 -
tools/testing/selftests/damon/Makefile | 7 +
.../selftests/damon/_chk_dependency.sh | 28 +
.../testing/selftests/damon/debugfs_attrs.sh | 75 ++
31 files changed, 3710 insertions(+), 18 deletions(-)
create mode 100644 Documentation/admin-guide/mm/damon/index.rst
create mode 100644 Documentation/admin-guide/mm/damon/start.rst
create mode 100644 Documentation/admin-guide/mm/damon/usage.rst
create mode 100644 Documentation/vm/damon/api.rst
create mode 100644 Documentation/vm/damon/design.rst
create mode 100644 Documentation/vm/damon/faq.rst
create mode 100644 Documentation/vm/damon/index.rst
create mode 100644 include/linux/damon.h
create mode 100644 include/trace/events/damon.h
create mode 100644 mm/damon/Kconfig
create mode 100644 mm/damon/Makefile
create mode 100644 mm/damon/core-test.h
create mode 100644 mm/damon/core.c
create mode 100644 mm/damon/dbgfs-test.h
create mode 100644 mm/damon/dbgfs.c
create mode 100644 mm/damon/vaddr-test.h
create mode 100644 mm/damon/vaddr.c
create mode 100644 tools/testing/selftests/damon/Makefile
create mode 100644 tools/testing/selftests/damon/_chk_dependency.sh
create mode 100755 tools/testing/selftests/damon/debugfs_attrs.sh
--
2.17.1
From: SeongJae Park <[email protected]>
This commit adds a simple user space tests for DAMON. The tests are
using kselftest framework.
Signed-off-by: SeongJae Park <[email protected]>
Reviewed-by: Markus Boehme <[email protected]>
---
tools/testing/selftests/damon/Makefile | 7 ++
.../selftests/damon/_chk_dependency.sh | 28 +++++++
.../testing/selftests/damon/debugfs_attrs.sh | 75 +++++++++++++++++++
3 files changed, 110 insertions(+)
create mode 100644 tools/testing/selftests/damon/Makefile
create mode 100644 tools/testing/selftests/damon/_chk_dependency.sh
create mode 100755 tools/testing/selftests/damon/debugfs_attrs.sh
diff --git a/tools/testing/selftests/damon/Makefile b/tools/testing/selftests/damon/Makefile
new file mode 100644
index 000000000000..8a3f2cd9fec0
--- /dev/null
+++ b/tools/testing/selftests/damon/Makefile
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0
+# Makefile for damon selftests
+
+TEST_FILES = _chk_dependency.sh
+TEST_PROGS = debugfs_attrs.sh
+
+include ../lib.mk
diff --git a/tools/testing/selftests/damon/_chk_dependency.sh b/tools/testing/selftests/damon/_chk_dependency.sh
new file mode 100644
index 000000000000..0189db81550b
--- /dev/null
+++ b/tools/testing/selftests/damon/_chk_dependency.sh
@@ -0,0 +1,28 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+# Kselftest framework requirement - SKIP code is 4.
+ksft_skip=4
+
+DBGFS=/sys/kernel/debug/damon
+
+if [ $EUID -ne 0 ];
+then
+ echo "Run as root"
+ exit $ksft_skip
+fi
+
+if [ ! -d "$DBGFS" ]
+then
+ echo "$DBGFS not found"
+ exit $ksft_skip
+fi
+
+for f in attrs target_ids monitor_on
+do
+ if [ ! -f "$DBGFS/$f" ]
+ then
+ echo "$f not found"
+ exit 1
+ fi
+done
diff --git a/tools/testing/selftests/damon/debugfs_attrs.sh b/tools/testing/selftests/damon/debugfs_attrs.sh
new file mode 100755
index 000000000000..bfabb19dc0d3
--- /dev/null
+++ b/tools/testing/selftests/damon/debugfs_attrs.sh
@@ -0,0 +1,75 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0
+
+test_write_result() {
+ file=$1
+ content=$2
+ orig_content=$3
+ expect_reason=$4
+ expected=$5
+
+ echo "$content" > "$file"
+ if [ $? -ne "$expected" ]
+ then
+ echo "writing $content to $file doesn't return $expected"
+ echo "expected because: $expect_reason"
+ echo "$orig_content" > "$file"
+ exit 1
+ fi
+}
+
+test_write_succ() {
+ test_write_result "$1" "$2" "$3" "$4" 0
+}
+
+test_write_fail() {
+ test_write_result "$1" "$2" "$3" "$4" 1
+}
+
+test_content() {
+ file=$1
+ orig_content=$2
+ expected=$3
+ expect_reason=$4
+
+ content=$(cat "$file")
+ if [ "$content" != "$expected" ]
+ then
+ echo "reading $file expected $expected but $content"
+ echo "expected because: $expect_reason"
+ echo "$orig_content" > "$file"
+ exit 1
+ fi
+}
+
+source ./_chk_dependency.sh
+
+# Test attrs file
+# ===============
+
+file="$DBGFS/attrs"
+orig_content=$(cat "$file")
+
+test_write_succ "$file" "1 2 3 4 5" "$orig_content" "valid input"
+test_write_fail "$file" "1 2 3 4" "$orig_content" "no enough fields"
+test_write_fail "$file" "1 2 3 5 4" "$orig_content" \
+ "min_nr_regions > max_nr_regions"
+test_content "$file" "$orig_content" "1 2 3 4 5" "successfully written"
+echo "$orig_content" > "$file"
+
+# Test target_ids file
+# ====================
+
+file="$DBGFS/target_ids"
+orig_content=$(cat "$file")
+
+test_write_succ "$file" "1 2 3 4" "$orig_content" "valid input"
+test_write_succ "$file" "1 2 abc 4" "$orig_content" "still valid input"
+test_content "$file" "$orig_content" "1 2" "non-integer was there"
+test_write_succ "$file" "abc 2 3" "$orig_content" "the file allows wrong input"
+test_content "$file" "$orig_content" "" "wrong input written"
+test_write_succ "$file" "" "$orig_content" "empty input"
+test_content "$file" "$orig_content" "" "empty input written"
+echo "$orig_content" > "$file"
+
+echo "PASS"
--
2.17.1
From: SeongJae Park <[email protected]>
Even somehow the initial monitoring target regions are well constructed
to fulfill the assumption (pages in same region have similar access
frequencies), the data access pattern can be dynamically changed. This
will result in low monitoring quality. To keep the assumption as much
as possible, DAMON adaptively merges and splits each region based on
their access frequency.
For each ``aggregation interval``, it compares the access frequencies of
adjacent regions and merges those if the frequency difference is small.
Then, after it reports and clears the aggregated access frequency of
each region, it splits each region into two or three regions if the
total number of regions will not exceed the user-specified maximum
number of regions after the split.
In this way, DAMON provides its best-effort quality and minimal overhead
while keeping the upper-bound overhead that users set.
Signed-off-by: SeongJae Park <[email protected]>
Reviewed-by: Leonard Foerster <[email protected]>
Reviewed-by: Fernand Sieber <[email protected]>
Acked-by: Shakeel Butt <[email protected]>
---
include/linux/damon.h | 30 ++++--
mm/damon/core.c | 224 ++++++++++++++++++++++++++++++++++++++++--
2 files changed, 237 insertions(+), 17 deletions(-)
diff --git a/include/linux/damon.h b/include/linux/damon.h
index 67db309ad61b..ce2a84b26cd7 100644
--- a/include/linux/damon.h
+++ b/include/linux/damon.h
@@ -12,6 +12,9 @@
#include <linux/time64.h>
#include <linux/types.h>
+/* Minimal region size. Every damon_region is aligned by this. */
+#define DAMON_MIN_REGION PAGE_SIZE
+
/**
* struct damon_addr_range - Represents an address region of [@start, @end).
* @start: Start address of the region (inclusive).
@@ -39,6 +42,7 @@ struct damon_region {
/**
* struct damon_target - Represents a monitoring target.
* @id: Unique identifier for this target.
+ * @nr_regions: Number of monitoring target regions of this target.
* @regions_list: Head of the monitoring target regions of this target.
* @list: List head for siblings.
*
@@ -50,6 +54,7 @@ struct damon_region {
*/
struct damon_target {
unsigned long id;
+ unsigned int nr_regions;
struct list_head regions_list;
struct list_head list;
};
@@ -85,6 +90,8 @@ struct damon_ctx;
* prepared for the next access check.
* @check_accesses should check the accesses to each region that made after the
* last preparation and update the number of observed accesses of each region.
+ * It should also return max number of observed accesses that made as a result
+ * of its update. The value will be used for regions adjustment threshold.
* @reset_aggregated should reset the access monitoring results that aggregated
* by @check_accesses.
* @target_valid should check whether the target is still valid for the
@@ -95,7 +102,7 @@ struct damon_primitive {
void (*init)(struct damon_ctx *context);
void (*update)(struct damon_ctx *context);
void (*prepare_access_checks)(struct damon_ctx *context);
- void (*check_accesses)(struct damon_ctx *context);
+ unsigned int (*check_accesses)(struct damon_ctx *context);
void (*reset_aggregated)(struct damon_ctx *context);
bool (*target_valid)(void *target);
void (*cleanup)(struct damon_ctx *context);
@@ -172,7 +179,9 @@ struct damon_callback {
* @primitive: Set of monitoring primitives for given use cases.
* @callback: Set of callbacks for monitoring events notifications.
*
- * @region_targets: Head of monitoring targets (&damon_target) list.
+ * @min_nr_regions: The minimum number of adaptive monitoring regions.
+ * @max_nr_regions: The maximum number of adaptive monitoring regions.
+ * @adaptive_targets: Head of monitoring targets (&damon_target) list.
*/
struct damon_ctx {
unsigned long sample_interval;
@@ -191,7 +200,9 @@ struct damon_ctx {
struct damon_primitive primitive;
struct damon_callback callback;
- struct list_head region_targets;
+ unsigned long min_nr_regions;
+ unsigned long max_nr_regions;
+ struct list_head adaptive_targets;
};
#define damon_next_region(r) \
@@ -207,28 +218,31 @@ struct damon_ctx {
list_for_each_entry_safe(r, next, &t->regions_list, list)
#define damon_for_each_target(t, ctx) \
- list_for_each_entry(t, &(ctx)->region_targets, list)
+ list_for_each_entry(t, &(ctx)->adaptive_targets, list)
#define damon_for_each_target_safe(t, next, ctx) \
- list_for_each_entry_safe(t, next, &(ctx)->region_targets, list)
+ list_for_each_entry_safe(t, next, &(ctx)->adaptive_targets, list)
#ifdef CONFIG_DAMON
struct damon_region *damon_new_region(unsigned long start, unsigned long end);
inline void damon_insert_region(struct damon_region *r,
- struct damon_region *prev, struct damon_region *next);
+ struct damon_region *prev, struct damon_region *next,
+ struct damon_target *t);
void damon_add_region(struct damon_region *r, struct damon_target *t);
-void damon_destroy_region(struct damon_region *r);
+void damon_destroy_region(struct damon_region *r, struct damon_target *t);
struct damon_target *damon_new_target(unsigned long id);
void damon_add_target(struct damon_ctx *ctx, struct damon_target *t);
void damon_free_target(struct damon_target *t);
void damon_destroy_target(struct damon_target *t);
+unsigned int damon_nr_regions(struct damon_target *t);
struct damon_ctx *damon_new_ctx(void);
void damon_destroy_ctx(struct damon_ctx *ctx);
int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int,
- unsigned long aggr_int, unsigned long primitive_upd_int);
+ unsigned long aggr_int, unsigned long primitive_upd_int,
+ unsigned long min_nr_reg, unsigned long max_nr_reg);
int damon_start(struct damon_ctx **ctxs, int nr_ctxs);
int damon_stop(struct damon_ctx **ctxs, int nr_ctxs);
diff --git a/mm/damon/core.c b/mm/damon/core.c
index 947486a150ce..28a2c78914fa 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -10,8 +10,12 @@
#include <linux/damon.h>
#include <linux/delay.h>
#include <linux/kthread.h>
+#include <linux/random.h>
#include <linux/slab.h>
+/* Get a random number in [l, r) */
+#define damon_rand(l, r) (l + prandom_u32_max(r - l))
+
static DEFINE_MUTEX(damon_lock);
static int nr_running_ctxs;
@@ -40,19 +44,23 @@ struct damon_region *damon_new_region(unsigned long start, unsigned long end)
* Add a region between two other regions
*/
inline void damon_insert_region(struct damon_region *r,
- struct damon_region *prev, struct damon_region *next)
+ struct damon_region *prev, struct damon_region *next,
+ struct damon_target *t)
{
__list_add(&r->list, &prev->list, &next->list);
+ t->nr_regions++;
}
void damon_add_region(struct damon_region *r, struct damon_target *t)
{
list_add_tail(&r->list, &t->regions_list);
+ t->nr_regions++;
}
-static void damon_del_region(struct damon_region *r)
+static void damon_del_region(struct damon_region *r, struct damon_target *t)
{
list_del(&r->list);
+ t->nr_regions--;
}
static void damon_free_region(struct damon_region *r)
@@ -60,9 +68,9 @@ static void damon_free_region(struct damon_region *r)
kfree(r);
}
-void damon_destroy_region(struct damon_region *r)
+void damon_destroy_region(struct damon_region *r, struct damon_target *t)
{
- damon_del_region(r);
+ damon_del_region(r, t);
damon_free_region(r);
}
@@ -80,6 +88,7 @@ struct damon_target *damon_new_target(unsigned long id)
return NULL;
t->id = id;
+ t->nr_regions = 0;
INIT_LIST_HEAD(&t->regions_list);
return t;
@@ -87,7 +96,7 @@ struct damon_target *damon_new_target(unsigned long id)
void damon_add_target(struct damon_ctx *ctx, struct damon_target *t)
{
- list_add_tail(&t->list, &ctx->region_targets);
+ list_add_tail(&t->list, &ctx->adaptive_targets);
}
static void damon_del_target(struct damon_target *t)
@@ -110,6 +119,11 @@ void damon_destroy_target(struct damon_target *t)
damon_free_target(t);
}
+unsigned int damon_nr_regions(struct damon_target *t)
+{
+ return t->nr_regions;
+}
+
struct damon_ctx *damon_new_ctx(void)
{
struct damon_ctx *ctx;
@@ -127,7 +141,10 @@ struct damon_ctx *damon_new_ctx(void)
mutex_init(&ctx->kdamond_lock);
- INIT_LIST_HEAD(&ctx->region_targets);
+ ctx->min_nr_regions = 10;
+ ctx->max_nr_regions = 1000;
+
+ INIT_LIST_HEAD(&ctx->adaptive_targets);
return ctx;
}
@@ -157,6 +174,8 @@ void damon_destroy_ctx(struct damon_ctx *ctx)
* @sample_int: time interval between samplings
* @aggr_int: time interval between aggregations
* @primitive_upd_int: time interval between monitoring primitive updates
+ * @min_nr_reg: minimal number of regions
+ * @max_nr_reg: maximum number of regions
*
* This function should not be called while the kdamond is running.
* Every time interval is in micro-seconds.
@@ -164,15 +183,49 @@ void damon_destroy_ctx(struct damon_ctx *ctx)
* Return: 0 on success, negative error code otherwise.
*/
int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int,
- unsigned long aggr_int, unsigned long primitive_upd_int)
+ unsigned long aggr_int, unsigned long primitive_upd_int,
+ unsigned long min_nr_reg, unsigned long max_nr_reg)
{
+ if (min_nr_reg < 3) {
+ pr_err("min_nr_regions (%lu) must be at least 3\n",
+ min_nr_reg);
+ return -EINVAL;
+ }
+ if (min_nr_reg > max_nr_reg) {
+ pr_err("invalid nr_regions. min (%lu) > max (%lu)\n",
+ min_nr_reg, max_nr_reg);
+ return -EINVAL;
+ }
+
ctx->sample_interval = sample_int;
ctx->aggr_interval = aggr_int;
ctx->primitive_update_interval = primitive_upd_int;
+ ctx->min_nr_regions = min_nr_reg;
+ ctx->max_nr_regions = max_nr_reg;
return 0;
}
+/* Returns the size upper limit for each monitoring region */
+static unsigned long damon_region_sz_limit(struct damon_ctx *ctx)
+{
+ struct damon_target *t;
+ struct damon_region *r;
+ unsigned long sz = 0;
+
+ damon_for_each_target(t, ctx) {
+ damon_for_each_region(r, t)
+ sz += r->ar.end - r->ar.start;
+ }
+
+ if (ctx->min_nr_regions)
+ sz /= ctx->min_nr_regions;
+ if (sz < DAMON_MIN_REGION)
+ sz = DAMON_MIN_REGION;
+
+ return sz;
+}
+
static bool damon_kdamond_running(struct damon_ctx *ctx)
{
bool running;
@@ -339,6 +392,150 @@ static void kdamond_reset_aggregated(struct damon_ctx *c)
}
}
+#define sz_damon_region(r) (r->ar.end - r->ar.start)
+
+/*
+ * Merge two adjacent regions into one region
+ */
+static void damon_merge_two_regions(struct damon_target *t,
+ struct damon_region *l, struct damon_region *r)
+{
+ unsigned long sz_l = sz_damon_region(l), sz_r = sz_damon_region(r);
+
+ l->nr_accesses = (l->nr_accesses * sz_l + r->nr_accesses * sz_r) /
+ (sz_l + sz_r);
+ l->ar.end = r->ar.end;
+ damon_destroy_region(r, t);
+}
+
+#define diff_of(a, b) (a > b ? a - b : b - a)
+
+/*
+ * Merge adjacent regions having similar access frequencies
+ *
+ * t target affected by this merge operation
+ * thres '->nr_accesses' diff threshold for the merge
+ * sz_limit size upper limit of each region
+ */
+static void damon_merge_regions_of(struct damon_target *t, unsigned int thres,
+ unsigned long sz_limit)
+{
+ struct damon_region *r, *prev = NULL, *next;
+
+ damon_for_each_region_safe(r, next, t) {
+ if (prev && prev->ar.end == r->ar.start &&
+ diff_of(prev->nr_accesses, r->nr_accesses) <= thres &&
+ sz_damon_region(prev) + sz_damon_region(r) <= sz_limit)
+ damon_merge_two_regions(t, prev, r);
+ else
+ prev = r;
+ }
+}
+
+/*
+ * Merge adjacent regions having similar access frequencies
+ *
+ * threshold '->nr_accesses' diff threshold for the merge
+ * sz_limit size upper limit of each region
+ *
+ * This function merges monitoring target regions which are adjacent and their
+ * access frequencies are similar. This is for minimizing the monitoring
+ * overhead under the dynamically changeable access pattern. If a merge was
+ * unnecessarily made, later 'kdamond_split_regions()' will revert it.
+ */
+static void kdamond_merge_regions(struct damon_ctx *c, unsigned int threshold,
+ unsigned long sz_limit)
+{
+ struct damon_target *t;
+
+ damon_for_each_target(t, c)
+ damon_merge_regions_of(t, threshold, sz_limit);
+}
+
+/*
+ * Split a region in two
+ *
+ * r the region to be split
+ * sz_r size of the first sub-region that will be made
+ */
+static void damon_split_region_at(struct damon_ctx *ctx,
+ struct damon_target *t, struct damon_region *r,
+ unsigned long sz_r)
+{
+ struct damon_region *new;
+
+ new = damon_new_region(r->ar.start + sz_r, r->ar.end);
+ if (!new)
+ return;
+
+ r->ar.end = new->ar.start;
+
+ damon_insert_region(new, r, damon_next_region(r), t);
+}
+
+/* Split every region in the given target into 'nr_subs' regions */
+static void damon_split_regions_of(struct damon_ctx *ctx,
+ struct damon_target *t, int nr_subs)
+{
+ struct damon_region *r, *next;
+ unsigned long sz_region, sz_sub = 0;
+ int i;
+
+ damon_for_each_region_safe(r, next, t) {
+ sz_region = r->ar.end - r->ar.start;
+
+ for (i = 0; i < nr_subs - 1 &&
+ sz_region > 2 * DAMON_MIN_REGION; i++) {
+ /*
+ * Randomly select size of left sub-region to be at
+ * least 10 percent and at most 90% of original region
+ */
+ sz_sub = ALIGN_DOWN(damon_rand(1, 10) *
+ sz_region / 10, DAMON_MIN_REGION);
+ /* Do not allow blank region */
+ if (sz_sub == 0 || sz_sub >= sz_region)
+ continue;
+
+ damon_split_region_at(ctx, t, r, sz_sub);
+ sz_region = sz_sub;
+ }
+ }
+}
+
+/*
+ * Split every target region into randomly-sized small regions
+ *
+ * This function splits every target region into random-sized small regions if
+ * current total number of the regions is equal or smaller than half of the
+ * user-specified maximum number of regions. This is for maximizing the
+ * monitoring accuracy under the dynamically changeable access patterns. If a
+ * split was unnecessarily made, later 'kdamond_merge_regions()' will revert
+ * it.
+ */
+static void kdamond_split_regions(struct damon_ctx *ctx)
+{
+ struct damon_target *t;
+ unsigned int nr_regions = 0;
+ static unsigned int last_nr_regions;
+ int nr_subregions = 2;
+
+ damon_for_each_target(t, ctx)
+ nr_regions += damon_nr_regions(t);
+
+ if (nr_regions > ctx->max_nr_regions / 2)
+ return;
+
+ /* Maybe the middle of the region has different access frequency */
+ if (last_nr_regions == nr_regions &&
+ nr_regions < ctx->max_nr_regions / 3)
+ nr_subregions = 3;
+
+ damon_for_each_target(t, ctx)
+ damon_split_regions_of(ctx, t, nr_subregions);
+
+ last_nr_regions = nr_regions;
+}
+
/*
* Check whether it is time to check and apply the target monitoring regions
*
@@ -395,6 +592,8 @@ static int kdamond_fn(void *data)
struct damon_ctx *ctx = (struct damon_ctx *)data;
struct damon_target *t;
struct damon_region *r, *next;
+ unsigned int max_nr_accesses = 0;
+ unsigned long sz_limit = 0;
mutex_lock(&ctx->kdamond_lock);
pr_info("kdamond (%d) starts\n", ctx->kdamond->pid);
@@ -405,6 +604,8 @@ static int kdamond_fn(void *data)
if (ctx->callback.before_start && ctx->callback.before_start(ctx))
set_kdamond_stop(ctx);
+ sz_limit = damon_region_sz_limit(ctx);
+
while (!kdamond_need_stop(ctx)) {
if (ctx->primitive.prepare_access_checks)
ctx->primitive.prepare_access_checks(ctx);
@@ -415,13 +616,17 @@ static int kdamond_fn(void *data)
usleep_range(ctx->sample_interval, ctx->sample_interval + 1);
if (ctx->primitive.check_accesses)
- ctx->primitive.check_accesses(ctx);
+ max_nr_accesses = ctx->primitive.check_accesses(ctx);
if (kdamond_aggregate_interval_passed(ctx)) {
+ kdamond_merge_regions(ctx,
+ max_nr_accesses / 10,
+ sz_limit);
if (ctx->callback.after_aggregation &&
ctx->callback.after_aggregation(ctx))
set_kdamond_stop(ctx);
kdamond_reset_aggregated(ctx);
+ kdamond_split_regions(ctx);
if (ctx->primitive.reset_aggregated)
ctx->primitive.reset_aggregated(ctx);
}
@@ -429,11 +634,12 @@ static int kdamond_fn(void *data)
if (kdamond_need_update_primitive(ctx)) {
if (ctx->primitive.update)
ctx->primitive.update(ctx);
+ sz_limit = damon_region_sz_limit(ctx);
}
}
damon_for_each_target(t, ctx) {
damon_for_each_region_safe(r, next, t)
- damon_destroy_region(r);
+ damon_destroy_region(r, t);
}
if (ctx->callback.before_terminate &&
--
2.17.1
From: SeongJae Park <[email protected]>
This commit adds a tracepoint for DAMON. It traces the monitoring
results of each region for each aggregation interval. Using this, DAMON
can easily integrated with tracepoints supporting tools such as perf.
Signed-off-by: SeongJae Park <[email protected]>
Reviewed-by: Leonard Foerster <[email protected]>
Reviewed-by: Steven Rostedt (VMware) <[email protected]>
Reviewed-by: Fernand Sieber <[email protected]>
Acked-by: Shakeel Butt <[email protected]>
---
include/trace/events/damon.h | 43 ++++++++++++++++++++++++++++++++++++
mm/damon/core.c | 7 +++++-
2 files changed, 49 insertions(+), 1 deletion(-)
create mode 100644 include/trace/events/damon.h
diff --git a/include/trace/events/damon.h b/include/trace/events/damon.h
new file mode 100644
index 000000000000..2f422f4f1fb9
--- /dev/null
+++ b/include/trace/events/damon.h
@@ -0,0 +1,43 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM damon
+
+#if !defined(_TRACE_DAMON_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_DAMON_H
+
+#include <linux/damon.h>
+#include <linux/types.h>
+#include <linux/tracepoint.h>
+
+TRACE_EVENT(damon_aggregated,
+
+ TP_PROTO(struct damon_target *t, struct damon_region *r,
+ unsigned int nr_regions),
+
+ TP_ARGS(t, r, nr_regions),
+
+ TP_STRUCT__entry(
+ __field(unsigned long, target_id)
+ __field(unsigned int, nr_regions)
+ __field(unsigned long, start)
+ __field(unsigned long, end)
+ __field(unsigned int, nr_accesses)
+ ),
+
+ TP_fast_assign(
+ __entry->target_id = t->id;
+ __entry->nr_regions = nr_regions;
+ __entry->start = r->ar.start;
+ __entry->end = r->ar.end;
+ __entry->nr_accesses = r->nr_accesses;
+ ),
+
+ TP_printk("target_id=%lu nr_regions=%u %lu-%lu: %u",
+ __entry->target_id, __entry->nr_regions,
+ __entry->start, __entry->end, __entry->nr_accesses)
+);
+
+#endif /* _TRACE_DAMON_H */
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
diff --git a/mm/damon/core.c b/mm/damon/core.c
index 28a2c78914fa..ee24d64e8019 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -13,6 +13,9 @@
#include <linux/random.h>
#include <linux/slab.h>
+#define CREATE_TRACE_POINTS
+#include <trace/events/damon.h>
+
/* Get a random number in [l, r) */
#define damon_rand(l, r) (l + prandom_u32_max(r - l))
@@ -387,8 +390,10 @@ static void kdamond_reset_aggregated(struct damon_ctx *c)
damon_for_each_target(t, c) {
struct damon_region *r;
- damon_for_each_region(r, t)
+ damon_for_each_region(r, t) {
+ trace_damon_aggregated(t, r, damon_nr_regions(t));
r->nr_accesses = 0;
+ }
}
}
--
2.17.1
From: SeongJae Park <[email protected]>
This commit adds kunit based unit tests for the core and the virtual
address spaces monitoring primitives of DAMON.
Signed-off-by: SeongJae Park <[email protected]>
Reviewed-by: Brendan Higgins <[email protected]>
---
mm/damon/Kconfig | 36 +++++
mm/damon/core-test.h | 253 ++++++++++++++++++++++++++++++++
mm/damon/core.c | 7 +
mm/damon/dbgfs-test.h | 126 ++++++++++++++++
mm/damon/dbgfs.c | 2 +
mm/damon/vaddr-test.h | 329 ++++++++++++++++++++++++++++++++++++++++++
mm/damon/vaddr.c | 7 +
7 files changed, 760 insertions(+)
create mode 100644 mm/damon/core-test.h
create mode 100644 mm/damon/dbgfs-test.h
create mode 100644 mm/damon/vaddr-test.h
diff --git a/mm/damon/Kconfig b/mm/damon/Kconfig
index 72f1683ba0ee..455995152697 100644
--- a/mm/damon/Kconfig
+++ b/mm/damon/Kconfig
@@ -12,6 +12,18 @@ config DAMON
See https://damonitor.github.io/doc/html/latest-damon/index.html for
more information.
+config DAMON_KUNIT_TEST
+ bool "Test for damon" if !KUNIT_ALL_TESTS
+ depends on DAMON && KUNIT=y
+ default KUNIT_ALL_TESTS
+ help
+ This builds the DAMON Kunit test suite.
+
+ For more information on KUnit and unit tests in general, please refer
+ to the KUnit documentation.
+
+ If unsure, say N.
+
config DAMON_VADDR
bool "Data access monitoring primitives for virtual address spaces"
depends on DAMON && MMU
@@ -21,6 +33,18 @@ config DAMON_VADDR
This builds the default data access monitoring primitives for DAMON
that works for virtual address spaces.
+config DAMON_VADDR_KUNIT_TEST
+ bool "Test for DAMON primitives" if !KUNIT_ALL_TESTS
+ depends on DAMON_VADDR && KUNIT=y
+ default KUNIT_ALL_TESTS
+ help
+ This builds the DAMON virtual addresses primitives Kunit test suite.
+
+ For more information on KUnit and unit tests in general, please refer
+ to the KUnit documentation.
+
+ If unsure, say N.
+
config DAMON_DBGFS
bool "DAMON debugfs interface"
depends on DAMON_VADDR && DEBUG_FS
@@ -30,4 +54,16 @@ config DAMON_DBGFS
If unsure, say N.
+config DAMON_DBGFS_KUNIT_TEST
+ bool "Test for damon debugfs interface" if !KUNIT_ALL_TESTS
+ depends on DAMON_DBGFS && KUNIT=y
+ default KUNIT_ALL_TESTS
+ help
+ This builds the DAMON debugfs interface Kunit test suite.
+
+ For more information on KUnit and unit tests in general, please refer
+ to the KUnit documentation.
+
+ If unsure, say N.
+
endmenu
diff --git a/mm/damon/core-test.h b/mm/damon/core-test.h
new file mode 100644
index 000000000000..c938a9c34e6c
--- /dev/null
+++ b/mm/damon/core-test.h
@@ -0,0 +1,253 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Data Access Monitor Unit Tests
+ *
+ * Copyright 2019 Amazon.com, Inc. or its affiliates. All rights reserved.
+ *
+ * Author: SeongJae Park <[email protected]>
+ */
+
+#ifdef CONFIG_DAMON_KUNIT_TEST
+
+#ifndef _DAMON_CORE_TEST_H
+#define _DAMON_CORE_TEST_H
+
+#include <kunit/test.h>
+
+static void damon_test_regions(struct kunit *test)
+{
+ struct damon_region *r;
+ struct damon_target *t;
+
+ r = damon_new_region(1, 2);
+ KUNIT_EXPECT_EQ(test, 1ul, r->ar.start);
+ KUNIT_EXPECT_EQ(test, 2ul, r->ar.end);
+ KUNIT_EXPECT_EQ(test, 0u, r->nr_accesses);
+
+ t = damon_new_target(42);
+ KUNIT_EXPECT_EQ(test, 0u, damon_nr_regions(t));
+
+ damon_add_region(r, t);
+ KUNIT_EXPECT_EQ(test, 1u, damon_nr_regions(t));
+
+ damon_del_region(r, t);
+ KUNIT_EXPECT_EQ(test, 0u, damon_nr_regions(t));
+
+ damon_free_target(t);
+}
+
+static unsigned int nr_damon_targets(struct damon_ctx *ctx)
+{
+ struct damon_target *t;
+ unsigned int nr_targets = 0;
+
+ damon_for_each_target(t, ctx)
+ nr_targets++;
+
+ return nr_targets;
+}
+
+static void damon_test_target(struct kunit *test)
+{
+ struct damon_ctx *c = damon_new_ctx();
+ struct damon_target *t;
+
+ t = damon_new_target(42);
+ KUNIT_EXPECT_EQ(test, 42ul, t->id);
+ KUNIT_EXPECT_EQ(test, 0u, nr_damon_targets(c));
+
+ damon_add_target(c, t);
+ KUNIT_EXPECT_EQ(test, 1u, nr_damon_targets(c));
+
+ damon_destroy_target(t);
+ KUNIT_EXPECT_EQ(test, 0u, nr_damon_targets(c));
+
+ damon_destroy_ctx(c);
+}
+
+/*
+ * Test kdamond_reset_aggregated()
+ *
+ * DAMON checks access to each region and aggregates this information as the
+ * access frequency of each region. In detail, it increases '->nr_accesses' of
+ * regions that an access has confirmed. 'kdamond_reset_aggregated()' flushes
+ * the aggregated information ('->nr_accesses' of each regions) to the result
+ * buffer. As a result of the flushing, the '->nr_accesses' of regions are
+ * initialized to zero.
+ */
+static void damon_test_aggregate(struct kunit *test)
+{
+ struct damon_ctx *ctx = damon_new_ctx();
+ unsigned long target_ids[] = {1, 2, 3};
+ unsigned long saddr[][3] = {{10, 20, 30}, {5, 42, 49}, {13, 33, 55} };
+ unsigned long eaddr[][3] = {{15, 27, 40}, {31, 45, 55}, {23, 44, 66} };
+ unsigned long accesses[][3] = {{42, 95, 84}, {10, 20, 30}, {0, 1, 2} };
+ struct damon_target *t;
+ struct damon_region *r;
+ int it, ir;
+
+ damon_set_targets(ctx, target_ids, 3);
+
+ it = 0;
+ damon_for_each_target(t, ctx) {
+ for (ir = 0; ir < 3; ir++) {
+ r = damon_new_region(saddr[it][ir], eaddr[it][ir]);
+ r->nr_accesses = accesses[it][ir];
+ damon_add_region(r, t);
+ }
+ it++;
+ }
+ kdamond_reset_aggregated(ctx);
+ it = 0;
+ damon_for_each_target(t, ctx) {
+ ir = 0;
+ /* '->nr_accesses' should be zeroed */
+ damon_for_each_region(r, t) {
+ KUNIT_EXPECT_EQ(test, 0u, r->nr_accesses);
+ ir++;
+ }
+ /* regions should be preserved */
+ KUNIT_EXPECT_EQ(test, 3, ir);
+ it++;
+ }
+ /* targets also should be preserved */
+ KUNIT_EXPECT_EQ(test, 3, it);
+
+ damon_destroy_ctx(ctx);
+}
+
+static void damon_test_split_at(struct kunit *test)
+{
+ struct damon_ctx *c = damon_new_ctx();
+ struct damon_target *t;
+ struct damon_region *r;
+
+ t = damon_new_target(42);
+ r = damon_new_region(0, 100);
+ damon_add_region(r, t);
+ damon_split_region_at(c, t, r, 25);
+ KUNIT_EXPECT_EQ(test, r->ar.start, 0ul);
+ KUNIT_EXPECT_EQ(test, r->ar.end, 25ul);
+
+ r = damon_next_region(r);
+ KUNIT_EXPECT_EQ(test, r->ar.start, 25ul);
+ KUNIT_EXPECT_EQ(test, r->ar.end, 100ul);
+
+ damon_free_target(t);
+ damon_destroy_ctx(c);
+}
+
+static void damon_test_merge_two(struct kunit *test)
+{
+ struct damon_target *t;
+ struct damon_region *r, *r2, *r3;
+ int i;
+
+ t = damon_new_target(42);
+ r = damon_new_region(0, 100);
+ r->nr_accesses = 10;
+ damon_add_region(r, t);
+ r2 = damon_new_region(100, 300);
+ r2->nr_accesses = 20;
+ damon_add_region(r2, t);
+
+ damon_merge_two_regions(t, r, r2);
+ KUNIT_EXPECT_EQ(test, r->ar.start, 0ul);
+ KUNIT_EXPECT_EQ(test, r->ar.end, 300ul);
+ KUNIT_EXPECT_EQ(test, r->nr_accesses, 16u);
+
+ i = 0;
+ damon_for_each_region(r3, t) {
+ KUNIT_EXPECT_PTR_EQ(test, r, r3);
+ i++;
+ }
+ KUNIT_EXPECT_EQ(test, i, 1);
+
+ damon_free_target(t);
+}
+
+static struct damon_region *__nth_region_of(struct damon_target *t, int idx)
+{
+ struct damon_region *r;
+ unsigned int i = 0;
+
+ damon_for_each_region(r, t) {
+ if (i++ == idx)
+ return r;
+ }
+
+ return NULL;
+}
+
+static void damon_test_merge_regions_of(struct kunit *test)
+{
+ struct damon_target *t;
+ struct damon_region *r;
+ unsigned long sa[] = {0, 100, 114, 122, 130, 156, 170, 184};
+ unsigned long ea[] = {100, 112, 122, 130, 156, 170, 184, 230};
+ unsigned int nrs[] = {0, 0, 10, 10, 20, 30, 1, 2};
+
+ unsigned long saddrs[] = {0, 114, 130, 156, 170};
+ unsigned long eaddrs[] = {112, 130, 156, 170, 230};
+ int i;
+
+ t = damon_new_target(42);
+ for (i = 0; i < ARRAY_SIZE(sa); i++) {
+ r = damon_new_region(sa[i], ea[i]);
+ r->nr_accesses = nrs[i];
+ damon_add_region(r, t);
+ }
+
+ damon_merge_regions_of(t, 9, 9999);
+ /* 0-112, 114-130, 130-156, 156-170 */
+ KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 5u);
+ for (i = 0; i < 5; i++) {
+ r = __nth_region_of(t, i);
+ KUNIT_EXPECT_EQ(test, r->ar.start, saddrs[i]);
+ KUNIT_EXPECT_EQ(test, r->ar.end, eaddrs[i]);
+ }
+ damon_free_target(t);
+}
+
+static void damon_test_split_regions_of(struct kunit *test)
+{
+ struct damon_ctx *c = damon_new_ctx();
+ struct damon_target *t;
+ struct damon_region *r;
+
+ t = damon_new_target(42);
+ r = damon_new_region(0, 22);
+ damon_add_region(r, t);
+ damon_split_regions_of(c, t, 2);
+ KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 2u);
+ damon_free_target(t);
+
+ t = damon_new_target(42);
+ r = damon_new_region(0, 220);
+ damon_add_region(r, t);
+ damon_split_regions_of(c, t, 4);
+ KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 4u);
+ damon_free_target(t);
+ damon_destroy_ctx(c);
+}
+
+static struct kunit_case damon_test_cases[] = {
+ KUNIT_CASE(damon_test_target),
+ KUNIT_CASE(damon_test_regions),
+ KUNIT_CASE(damon_test_aggregate),
+ KUNIT_CASE(damon_test_split_at),
+ KUNIT_CASE(damon_test_merge_two),
+ KUNIT_CASE(damon_test_merge_regions_of),
+ KUNIT_CASE(damon_test_split_regions_of),
+ {},
+};
+
+static struct kunit_suite damon_test_suite = {
+ .name = "damon",
+ .test_cases = damon_test_cases,
+};
+kunit_test_suite(damon_test_suite);
+
+#endif /* _DAMON_CORE_TEST_H */
+
+#endif /* CONFIG_DAMON_KUNIT_TEST */
diff --git a/mm/damon/core.c b/mm/damon/core.c
index 59033488402e..30e9211f494a 100644
--- a/mm/damon/core.c
+++ b/mm/damon/core.c
@@ -16,6 +16,11 @@
#define CREATE_TRACE_POINTS
#include <trace/events/damon.h>
+#ifdef CONFIG_DAMON_KUNIT_TEST
+#undef DAMON_MIN_REGION
+#define DAMON_MIN_REGION 1
+#endif
+
/* Get a random number in [l, r) */
#define damon_rand(l, r) (l + prandom_u32_max(r - l))
@@ -711,3 +716,5 @@ static int kdamond_fn(void *data)
do_exit(0);
}
+
+#include "core-test.h"
diff --git a/mm/damon/dbgfs-test.h b/mm/damon/dbgfs-test.h
new file mode 100644
index 000000000000..930e83bceef0
--- /dev/null
+++ b/mm/damon/dbgfs-test.h
@@ -0,0 +1,126 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * DAMON Debugfs Interface Unit Tests
+ *
+ * Author: SeongJae Park <[email protected]>
+ */
+
+#ifdef CONFIG_DAMON_DBGFS_KUNIT_TEST
+
+#ifndef _DAMON_DBGFS_TEST_H
+#define _DAMON_DBGFS_TEST_H
+
+#include <kunit/test.h>
+
+static void damon_dbgfs_test_str_to_target_ids(struct kunit *test)
+{
+ char *question;
+ unsigned long *answers;
+ unsigned long expected[] = {12, 35, 46};
+ ssize_t nr_integers = 0, i;
+
+ question = "123";
+ answers = str_to_target_ids(question, strnlen(question, 128),
+ &nr_integers);
+ KUNIT_EXPECT_EQ(test, (ssize_t)1, nr_integers);
+ KUNIT_EXPECT_EQ(test, 123ul, answers[0]);
+ kfree(answers);
+
+ question = "123abc";
+ answers = str_to_target_ids(question, strnlen(question, 128),
+ &nr_integers);
+ KUNIT_EXPECT_EQ(test, (ssize_t)1, nr_integers);
+ KUNIT_EXPECT_EQ(test, 123ul, answers[0]);
+ kfree(answers);
+
+ question = "a123";
+ answers = str_to_target_ids(question, strnlen(question, 128),
+ &nr_integers);
+ KUNIT_EXPECT_EQ(test, (ssize_t)0, nr_integers);
+ kfree(answers);
+
+ question = "12 35";
+ answers = str_to_target_ids(question, strnlen(question, 128),
+ &nr_integers);
+ KUNIT_EXPECT_EQ(test, (ssize_t)2, nr_integers);
+ for (i = 0; i < nr_integers; i++)
+ KUNIT_EXPECT_EQ(test, expected[i], answers[i]);
+ kfree(answers);
+
+ question = "12 35 46";
+ answers = str_to_target_ids(question, strnlen(question, 128),
+ &nr_integers);
+ KUNIT_EXPECT_EQ(test, (ssize_t)3, nr_integers);
+ for (i = 0; i < nr_integers; i++)
+ KUNIT_EXPECT_EQ(test, expected[i], answers[i]);
+ kfree(answers);
+
+ question = "12 35 abc 46";
+ answers = str_to_target_ids(question, strnlen(question, 128),
+ &nr_integers);
+ KUNIT_EXPECT_EQ(test, (ssize_t)2, nr_integers);
+ for (i = 0; i < 2; i++)
+ KUNIT_EXPECT_EQ(test, expected[i], answers[i]);
+ kfree(answers);
+
+ question = "";
+ answers = str_to_target_ids(question, strnlen(question, 128),
+ &nr_integers);
+ KUNIT_EXPECT_EQ(test, (ssize_t)0, nr_integers);
+ kfree(answers);
+
+ question = "\n";
+ answers = str_to_target_ids(question, strnlen(question, 128),
+ &nr_integers);
+ KUNIT_EXPECT_EQ(test, (ssize_t)0, nr_integers);
+ kfree(answers);
+}
+
+static void damon_dbgfs_test_set_targets(struct kunit *test)
+{
+ struct damon_ctx *ctx = dbgfs_new_ctx();
+ unsigned long ids[] = {1, 2, 3};
+ char buf[64];
+
+ /* Make DAMON consider target id as plain number */
+ ctx->primitive.target_valid = NULL;
+ ctx->primitive.cleanup = NULL;
+
+ damon_set_targets(ctx, ids, 3);
+ sprint_target_ids(ctx, buf, 64);
+ KUNIT_EXPECT_STREQ(test, (char *)buf, "1 2 3\n");
+
+ damon_set_targets(ctx, NULL, 0);
+ sprint_target_ids(ctx, buf, 64);
+ KUNIT_EXPECT_STREQ(test, (char *)buf, "\n");
+
+ damon_set_targets(ctx, (unsigned long []){1, 2}, 2);
+ sprint_target_ids(ctx, buf, 64);
+ KUNIT_EXPECT_STREQ(test, (char *)buf, "1 2\n");
+
+ damon_set_targets(ctx, (unsigned long []){2}, 1);
+ sprint_target_ids(ctx, buf, 64);
+ KUNIT_EXPECT_STREQ(test, (char *)buf, "2\n");
+
+ damon_set_targets(ctx, NULL, 0);
+ sprint_target_ids(ctx, buf, 64);
+ KUNIT_EXPECT_STREQ(test, (char *)buf, "\n");
+
+ dbgfs_destroy_ctx(ctx);
+}
+
+static struct kunit_case damon_test_cases[] = {
+ KUNIT_CASE(damon_dbgfs_test_str_to_target_ids),
+ KUNIT_CASE(damon_dbgfs_test_set_targets),
+ {},
+};
+
+static struct kunit_suite damon_test_suite = {
+ .name = "damon-dbgfs",
+ .test_cases = damon_test_cases,
+};
+kunit_test_suite(damon_test_suite);
+
+#endif /* _DAMON_TEST_H */
+
+#endif /* CONFIG_DAMON_KUNIT_TEST */
diff --git a/mm/damon/dbgfs.c b/mm/damon/dbgfs.c
index cd2cc0a9c52a..d84593924a55 100644
--- a/mm/damon/dbgfs.c
+++ b/mm/damon/dbgfs.c
@@ -620,3 +620,5 @@ static int __init damon_dbgfs_init(void)
}
module_init(damon_dbgfs_init);
+
+#include "dbgfs-test.h"
diff --git a/mm/damon/vaddr-test.h b/mm/damon/vaddr-test.h
new file mode 100644
index 000000000000..1f5c13257dba
--- /dev/null
+++ b/mm/damon/vaddr-test.h
@@ -0,0 +1,329 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * Data Access Monitor Unit Tests
+ *
+ * Copyright 2019 Amazon.com, Inc. or its affiliates. All rights reserved.
+ *
+ * Author: SeongJae Park <[email protected]>
+ */
+
+#ifdef CONFIG_DAMON_VADDR_KUNIT_TEST
+
+#ifndef _DAMON_VADDR_TEST_H
+#define _DAMON_VADDR_TEST_H
+
+#include <kunit/test.h>
+
+static void __link_vmas(struct vm_area_struct *vmas, ssize_t nr_vmas)
+{
+ int i, j;
+ unsigned long largest_gap, gap;
+
+ if (!nr_vmas)
+ return;
+
+ for (i = 0; i < nr_vmas - 1; i++) {
+ vmas[i].vm_next = &vmas[i + 1];
+
+ vmas[i].vm_rb.rb_left = NULL;
+ vmas[i].vm_rb.rb_right = &vmas[i + 1].vm_rb;
+
+ largest_gap = 0;
+ for (j = i; j < nr_vmas; j++) {
+ if (j == 0)
+ continue;
+ gap = vmas[j].vm_start - vmas[j - 1].vm_end;
+ if (gap > largest_gap)
+ largest_gap = gap;
+ }
+ vmas[i].rb_subtree_gap = largest_gap;
+ }
+ vmas[i].vm_next = NULL;
+ vmas[i].vm_rb.rb_right = NULL;
+ vmas[i].rb_subtree_gap = 0;
+}
+
+/*
+ * Test __damon_va_three_regions() function
+ *
+ * In case of virtual memory address spaces monitoring, DAMON converts the
+ * complex and dynamic memory mappings of each target task to three
+ * discontiguous regions which cover every mapped areas. However, the three
+ * regions should not include the two biggest unmapped areas in the original
+ * mapping, because the two biggest areas are normally the areas between 1)
+ * heap and the mmap()-ed regions, and 2) the mmap()-ed regions and stack.
+ * Because these two unmapped areas are very huge but obviously never accessed,
+ * covering the region is just a waste.
+ *
+ * '__damon_va_three_regions() receives an address space of a process. It
+ * first identifies the start of mappings, end of mappings, and the two biggest
+ * unmapped areas. After that, based on the information, it constructs the
+ * three regions and returns. For more detail, refer to the comment of
+ * 'damon_init_regions_of()' function definition in 'mm/damon.c' file.
+ *
+ * For example, suppose virtual address ranges of 10-20, 20-25, 200-210,
+ * 210-220, 300-305, and 307-330 (Other comments represent this mappings in
+ * more short form: 10-20-25, 200-210-220, 300-305, 307-330) of a process are
+ * mapped. To cover every mappings, the three regions should start with 10,
+ * and end with 305. The process also has three unmapped areas, 25-200,
+ * 220-300, and 305-307. Among those, 25-200 and 220-300 are the biggest two
+ * unmapped areas, and thus it should be converted to three regions of 10-25,
+ * 200-220, and 300-330.
+ */
+static void damon_test_three_regions_in_vmas(struct kunit *test)
+{
+ struct damon_addr_range regions[3] = {0,};
+ /* 10-20-25, 200-210-220, 300-305, 307-330 */
+ struct vm_area_struct vmas[] = {
+ (struct vm_area_struct) {.vm_start = 10, .vm_end = 20},
+ (struct vm_area_struct) {.vm_start = 20, .vm_end = 25},
+ (struct vm_area_struct) {.vm_start = 200, .vm_end = 210},
+ (struct vm_area_struct) {.vm_start = 210, .vm_end = 220},
+ (struct vm_area_struct) {.vm_start = 300, .vm_end = 305},
+ (struct vm_area_struct) {.vm_start = 307, .vm_end = 330},
+ };
+
+ __link_vmas(vmas, 6);
+
+ __damon_va_three_regions(&vmas[0], regions);
+
+ KUNIT_EXPECT_EQ(test, 10ul, regions[0].start);
+ KUNIT_EXPECT_EQ(test, 25ul, regions[0].end);
+ KUNIT_EXPECT_EQ(test, 200ul, regions[1].start);
+ KUNIT_EXPECT_EQ(test, 220ul, regions[1].end);
+ KUNIT_EXPECT_EQ(test, 300ul, regions[2].start);
+ KUNIT_EXPECT_EQ(test, 330ul, regions[2].end);
+}
+
+static struct damon_region *__nth_region_of(struct damon_target *t, int idx)
+{
+ struct damon_region *r;
+ unsigned int i = 0;
+
+ damon_for_each_region(r, t) {
+ if (i++ == idx)
+ return r;
+ }
+
+ return NULL;
+}
+
+/*
+ * Test 'damon_va_apply_three_regions()'
+ *
+ * test kunit object
+ * regions an array containing start/end addresses of current
+ * monitoring target regions
+ * nr_regions the number of the addresses in 'regions'
+ * three_regions The three regions that need to be applied now
+ * expected start/end addresses of monitoring target regions that
+ * 'three_regions' are applied
+ * nr_expected the number of addresses in 'expected'
+ *
+ * The memory mapping of the target processes changes dynamically. To follow
+ * the change, DAMON periodically reads the mappings, simplifies it to the
+ * three regions, and updates the monitoring target regions to fit in the three
+ * regions. The update of current target regions is the role of
+ * 'damon_va_apply_three_regions()'.
+ *
+ * This test passes the given target regions and the new three regions that
+ * need to be applied to the function and check whether it updates the regions
+ * as expected.
+ */
+static void damon_do_test_apply_three_regions(struct kunit *test,
+ unsigned long *regions, int nr_regions,
+ struct damon_addr_range *three_regions,
+ unsigned long *expected, int nr_expected)
+{
+ struct damon_ctx *ctx = damon_new_ctx();
+ struct damon_target *t;
+ struct damon_region *r;
+ int i;
+
+ t = damon_new_target(42);
+ for (i = 0; i < nr_regions / 2; i++) {
+ r = damon_new_region(regions[i * 2], regions[i * 2 + 1]);
+ damon_add_region(r, t);
+ }
+ damon_add_target(ctx, t);
+
+ damon_va_apply_three_regions(t, three_regions);
+
+ for (i = 0; i < nr_expected / 2; i++) {
+ r = __nth_region_of(t, i);
+ KUNIT_EXPECT_EQ(test, r->ar.start, expected[i * 2]);
+ KUNIT_EXPECT_EQ(test, r->ar.end, expected[i * 2 + 1]);
+ }
+
+ damon_destroy_ctx(ctx);
+}
+
+/*
+ * This function test most common case where the three big regions are only
+ * slightly changed. Target regions should adjust their boundary (10-20-30,
+ * 50-55, 70-80, 90-100) to fit with the new big regions or remove target
+ * regions (57-79) that now out of the three regions.
+ */
+static void damon_test_apply_three_regions1(struct kunit *test)
+{
+ /* 10-20-30, 50-55-57-59, 70-80-90-100 */
+ unsigned long regions[] = {10, 20, 20, 30, 50, 55, 55, 57, 57, 59,
+ 70, 80, 80, 90, 90, 100};
+ /* 5-27, 45-55, 73-104 */
+ struct damon_addr_range new_three_regions[3] = {
+ (struct damon_addr_range){.start = 5, .end = 27},
+ (struct damon_addr_range){.start = 45, .end = 55},
+ (struct damon_addr_range){.start = 73, .end = 104} };
+ /* 5-20-27, 45-55, 73-80-90-104 */
+ unsigned long expected[] = {5, 20, 20, 27, 45, 55,
+ 73, 80, 80, 90, 90, 104};
+
+ damon_do_test_apply_three_regions(test, regions, ARRAY_SIZE(regions),
+ new_three_regions, expected, ARRAY_SIZE(expected));
+}
+
+/*
+ * Test slightly bigger change. Similar to above, but the second big region
+ * now require two target regions (50-55, 57-59) to be removed.
+ */
+static void damon_test_apply_three_regions2(struct kunit *test)
+{
+ /* 10-20-30, 50-55-57-59, 70-80-90-100 */
+ unsigned long regions[] = {10, 20, 20, 30, 50, 55, 55, 57, 57, 59,
+ 70, 80, 80, 90, 90, 100};
+ /* 5-27, 56-57, 65-104 */
+ struct damon_addr_range new_three_regions[3] = {
+ (struct damon_addr_range){.start = 5, .end = 27},
+ (struct damon_addr_range){.start = 56, .end = 57},
+ (struct damon_addr_range){.start = 65, .end = 104} };
+ /* 5-20-27, 56-57, 65-80-90-104 */
+ unsigned long expected[] = {5, 20, 20, 27, 56, 57,
+ 65, 80, 80, 90, 90, 104};
+
+ damon_do_test_apply_three_regions(test, regions, ARRAY_SIZE(regions),
+ new_three_regions, expected, ARRAY_SIZE(expected));
+}
+
+/*
+ * Test a big change. The second big region has totally freed and mapped to
+ * different area (50-59 -> 61-63). The target regions which were in the old
+ * second big region (50-55-57-59) should be removed and new target region
+ * covering the second big region (61-63) should be created.
+ */
+static void damon_test_apply_three_regions3(struct kunit *test)
+{
+ /* 10-20-30, 50-55-57-59, 70-80-90-100 */
+ unsigned long regions[] = {10, 20, 20, 30, 50, 55, 55, 57, 57, 59,
+ 70, 80, 80, 90, 90, 100};
+ /* 5-27, 61-63, 65-104 */
+ struct damon_addr_range new_three_regions[3] = {
+ (struct damon_addr_range){.start = 5, .end = 27},
+ (struct damon_addr_range){.start = 61, .end = 63},
+ (struct damon_addr_range){.start = 65, .end = 104} };
+ /* 5-20-27, 61-63, 65-80-90-104 */
+ unsigned long expected[] = {5, 20, 20, 27, 61, 63,
+ 65, 80, 80, 90, 90, 104};
+
+ damon_do_test_apply_three_regions(test, regions, ARRAY_SIZE(regions),
+ new_three_regions, expected, ARRAY_SIZE(expected));
+}
+
+/*
+ * Test another big change. Both of the second and third big regions (50-59
+ * and 70-100) has totally freed and mapped to different area (30-32 and
+ * 65-68). The target regions which were in the old second and third big
+ * regions should now be removed and new target regions covering the new second
+ * and third big regions should be crated.
+ */
+static void damon_test_apply_three_regions4(struct kunit *test)
+{
+ /* 10-20-30, 50-55-57-59, 70-80-90-100 */
+ unsigned long regions[] = {10, 20, 20, 30, 50, 55, 55, 57, 57, 59,
+ 70, 80, 80, 90, 90, 100};
+ /* 5-7, 30-32, 65-68 */
+ struct damon_addr_range new_three_regions[3] = {
+ (struct damon_addr_range){.start = 5, .end = 7},
+ (struct damon_addr_range){.start = 30, .end = 32},
+ (struct damon_addr_range){.start = 65, .end = 68} };
+ /* expect 5-7, 30-32, 65-68 */
+ unsigned long expected[] = {5, 7, 30, 32, 65, 68};
+
+ damon_do_test_apply_three_regions(test, regions, ARRAY_SIZE(regions),
+ new_three_regions, expected, ARRAY_SIZE(expected));
+}
+
+static void damon_test_split_evenly(struct kunit *test)
+{
+ struct damon_ctx *c = damon_new_ctx();
+ struct damon_target *t;
+ struct damon_region *r;
+ unsigned long i;
+
+ KUNIT_EXPECT_EQ(test, damon_va_evenly_split_region(NULL, NULL, 5),
+ -EINVAL);
+
+ t = damon_new_target(42);
+ r = damon_new_region(0, 100);
+ KUNIT_EXPECT_EQ(test, damon_va_evenly_split_region(t, r, 0), -EINVAL);
+
+ damon_add_region(r, t);
+ KUNIT_EXPECT_EQ(test, damon_va_evenly_split_region(t, r, 10), 0);
+ KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 10u);
+
+ i = 0;
+ damon_for_each_region(r, t) {
+ KUNIT_EXPECT_EQ(test, r->ar.start, i++ * 10);
+ KUNIT_EXPECT_EQ(test, r->ar.end, i * 10);
+ }
+ damon_free_target(t);
+
+ t = damon_new_target(42);
+ r = damon_new_region(5, 59);
+ damon_add_region(r, t);
+ KUNIT_EXPECT_EQ(test, damon_va_evenly_split_region(t, r, 5), 0);
+ KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 5u);
+
+ i = 0;
+ damon_for_each_region(r, t) {
+ if (i == 4)
+ break;
+ KUNIT_EXPECT_EQ(test, r->ar.start, 5 + 10 * i++);
+ KUNIT_EXPECT_EQ(test, r->ar.end, 5 + 10 * i);
+ }
+ KUNIT_EXPECT_EQ(test, r->ar.start, 5 + 10 * i);
+ KUNIT_EXPECT_EQ(test, r->ar.end, 59ul);
+ damon_free_target(t);
+
+ t = damon_new_target(42);
+ r = damon_new_region(5, 6);
+ damon_add_region(r, t);
+ KUNIT_EXPECT_EQ(test, damon_va_evenly_split_region(t, r, 2), -EINVAL);
+ KUNIT_EXPECT_EQ(test, damon_nr_regions(t), 1u);
+
+ damon_for_each_region(r, t) {
+ KUNIT_EXPECT_EQ(test, r->ar.start, 5ul);
+ KUNIT_EXPECT_EQ(test, r->ar.end, 6ul);
+ }
+ damon_free_target(t);
+ damon_destroy_ctx(c);
+}
+
+static struct kunit_case damon_test_cases[] = {
+ KUNIT_CASE(damon_test_three_regions_in_vmas),
+ KUNIT_CASE(damon_test_apply_three_regions1),
+ KUNIT_CASE(damon_test_apply_three_regions2),
+ KUNIT_CASE(damon_test_apply_three_regions3),
+ KUNIT_CASE(damon_test_apply_three_regions4),
+ KUNIT_CASE(damon_test_split_evenly),
+ {},
+};
+
+static struct kunit_suite damon_test_suite = {
+ .name = "damon-primitives",
+ .test_cases = damon_test_cases,
+};
+kunit_test_suite(damon_test_suite);
+
+#endif /* _DAMON_VADDR_TEST_H */
+
+#endif /* CONFIG_DAMON_VADDR_KUNIT_TEST */
diff --git a/mm/damon/vaddr.c b/mm/damon/vaddr.c
index 91d4a95cedf7..e5c2c3164749 100644
--- a/mm/damon/vaddr.c
+++ b/mm/damon/vaddr.c
@@ -15,6 +15,11 @@
#include <linux/sched/mm.h>
#include <linux/slab.h>
+#ifdef CONFIG_DAMON_VADDR_KUNIT_TEST
+#undef DAMON_MIN_REGION
+#define DAMON_MIN_REGION 1
+#endif
+
/* Get a random number in [l, r) */
#define damon_rand(l, r) (l + prandom_u32_max(r - l))
@@ -604,3 +609,5 @@ void damon_va_set_primitives(struct damon_ctx *ctx)
ctx->primitive.target_valid = damon_va_target_valid;
ctx->primitive.cleanup = NULL;
}
+
+#include "vaddr-test.h"
--
2.17.1
From: SeongJae Park <[email protected]>
This commit adds documents for DAMON under
`Documentation/admin-guide/mm/damon/` and `Documentation/vm/damon/`.
Signed-off-by: SeongJae Park <[email protected]>
Reviewed-by: Fernand Sieber <[email protected]>
Reviewed-by: Markus Boehme <[email protected]>
---
Documentation/admin-guide/mm/damon/index.rst | 15 ++
Documentation/admin-guide/mm/damon/start.rst | 114 +++++++++++++
Documentation/admin-guide/mm/damon/usage.rst | 112 +++++++++++++
Documentation/admin-guide/mm/index.rst | 1 +
Documentation/vm/damon/api.rst | 20 +++
Documentation/vm/damon/design.rst | 166 +++++++++++++++++++
Documentation/vm/damon/faq.rst | 51 ++++++
Documentation/vm/damon/index.rst | 30 ++++
Documentation/vm/index.rst | 1 +
9 files changed, 510 insertions(+)
create mode 100644 Documentation/admin-guide/mm/damon/index.rst
create mode 100644 Documentation/admin-guide/mm/damon/start.rst
create mode 100644 Documentation/admin-guide/mm/damon/usage.rst
create mode 100644 Documentation/vm/damon/api.rst
create mode 100644 Documentation/vm/damon/design.rst
create mode 100644 Documentation/vm/damon/faq.rst
create mode 100644 Documentation/vm/damon/index.rst
diff --git a/Documentation/admin-guide/mm/damon/index.rst b/Documentation/admin-guide/mm/damon/index.rst
new file mode 100644
index 000000000000..8c5dde3a5754
--- /dev/null
+++ b/Documentation/admin-guide/mm/damon/index.rst
@@ -0,0 +1,15 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+========================
+Monitoring Data Accesses
+========================
+
+:doc:`DAMON </vm/damon/index>` allows light-weight data access monitoring.
+Using DAMON, users can analyze the memory access patterns of their systems and
+optimize those.
+
+.. toctree::
+ :maxdepth: 2
+
+ start
+ usage
diff --git a/Documentation/admin-guide/mm/damon/start.rst b/Documentation/admin-guide/mm/damon/start.rst
new file mode 100644
index 000000000000..d5eb89a8fc38
--- /dev/null
+++ b/Documentation/admin-guide/mm/damon/start.rst
@@ -0,0 +1,114 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+===============
+Getting Started
+===============
+
+This document briefly describes how you can use DAMON by demonstrating its
+default user space tool. Please note that this document describes only a part
+of its features for brevity. Please refer to :doc:`usage` for more details.
+
+
+TL; DR
+======
+
+Follow the commands below to monitor and visualize the memory access pattern of
+your workload. ::
+
+ # # build the kernel with CONFIG_DAMON_*=y, install it, and reboot
+ # mount -t debugfs none /sys/kernel/debug/
+ # git clone https://github.com/awslabs/damo
+ # ./damo/damo record $(pidof <your workload>)
+ # ./damo/damo report heat --plot_ascii
+
+The final command draws the access heatmap of ``<your workload>``. The heatmap
+shows which memory region (x-axis) is accessed when (y-axis) and how frequently
+(number; the higher the more accesses have been observed). ::
+
+ 111111111111111111111111111111111111111111111111111111110000
+ 111121111111111111111111111111211111111111111111111111110000
+ 000000000000000000000000000000000000000000000000001555552000
+ 000000000000000000000000000000000000000000000222223555552000
+ 000000000000000000000000000000000000000011111677775000000000
+ 000000000000000000000000000000000000000488888000000000000000
+ 000000000000000000000000000000000177888400000000000000000000
+ 000000000000000000000000000046666522222100000000000000000000
+ 000000000000000000000014444344444300000000000000000000000000
+ 000000000000000002222245555510000000000000000000000000000000
+ # access_frequency: 0 1 2 3 4 5 6 7 8 9
+ # x-axis: space (140286319947776-140286426374096: 101.496 MiB)
+ # y-axis: time (605442256436361-605479951866441: 37.695430s)
+ # resolution: 60x10 (1.692 MiB and 3.770s for each character)
+
+
+Prerequisites
+=============
+
+Kernel
+------
+
+You should first ensure your system is running on a kernel built with
+``CONFIG_DAMON_*=y``.
+
+
+User Space Tool
+---------------
+
+For the demonstration, we will use the default user space tool for DAMON,
+called DAMON Operator (DAMO). It is available at
+https://github.com/awslabs/damo. The examples below assume that ``damo`` is on
+your ``$PATH``. It's not mandatory, though.
+
+Because DAMO is using the debugfs interface (refer to :doc:`usage` for the
+detail) of DAMON, you should ensure debugfs is mounted. Mount it manually as
+below::
+
+ # mount -t debugfs none /sys/kernel/debug/
+
+or append the following line to your ``/etc/fstab`` file so that your system
+can automatically mount debugfs upon booting::
+
+ debugfs /sys/kernel/debug debugfs defaults 0 0
+
+
+Recording Data Access Patterns
+==============================
+
+The commands below record the memory access patterns of a program and save the
+monitoring results to a file. ::
+
+ $ git clone https://github.com/sjp38/masim
+ $ cd masim; make; ./masim ./configs/zigzag.cfg &
+ $ sudo damo record -o damon.data $(pidof masim)
+
+The first two lines of the commands download an artificial memory access
+generator program and run it in the background. The generator will repeatedly
+access two 100 MiB sized memory regions one by one. You can substitute this
+with your real workload. The last line asks ``damo`` to record the access
+pattern in the ``damon.data`` file.
+
+
+Visualizing Recorded Patterns
+=============================
+
+The following three commands visualize the recorded access patterns and save
+the results as separate image files. ::
+
+ $ damo report heats --heatmap access_pattern_heatmap.png
+ $ damo report wss --range 0 101 1 --plot wss_dist.png
+ $ damo report wss --range 0 101 1 --sortby time --plot wss_chron_change.png
+
+- ``access_pattern_heatmap.png`` will visualize the data access pattern in a
+ heatmap, showing which memory region (y-axis) got accessed when (x-axis)
+ and how frequently (color).
+- ``wss_dist.png`` will show the distribution of the working set size.
+- ``wss_chron_change.png`` will show how the working set size has
+ chronologically changed.
+
+You can view the visualizations of this example workload at [1]_.
+Visualizations of other realistic workloads are available at [2]_ [3]_ [4]_.
+
+.. [1] https://damonitor.github.io/doc/html/v17/admin-guide/mm/damon/start.html#visualizing-recorded-patterns
+.. [2] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.1.png.html
+.. [3] https://damonitor.github.io/test/result/visual/latest/rec.wss_sz.png.html
+.. [4] https://damonitor.github.io/test/result/visual/latest/rec.wss_time.png.html
diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin-guide/mm/damon/usage.rst
new file mode 100644
index 000000000000..a72cda374aba
--- /dev/null
+++ b/Documentation/admin-guide/mm/damon/usage.rst
@@ -0,0 +1,112 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+===============
+Detailed Usages
+===============
+
+DAMON provides below three interfaces for different users.
+
+- *DAMON user space tool.*
+ This is for privileged people such as system administrators who want a
+ just-working human-friendly interface. Using this, users can use the DAMON’s
+ major features in a human-friendly way. It may not be highly tuned for
+ special cases, though. It supports only virtual address spaces monitoring.
+- *debugfs interface.*
+ This is for privileged user space programmers who want more optimized use of
+ DAMON. Using this, users can use DAMON’s major features by reading
+ from and writing to special debugfs files. Therefore, you can write and use
+ your personalized DAMON debugfs wrapper programs that reads/writes the
+ debugfs files instead of you. The DAMON user space tool is also a reference
+ implementation of such programs. It supports only virtual address spaces
+ monitoring.
+- *Kernel Space Programming Interface.*
+ This is for kernel space programmers. Using this, users can utilize every
+ feature of DAMON most flexibly and efficiently by writing kernel space
+ DAMON application programs for you. You can even extend DAMON for various
+ address spaces.
+
+Nevertheless, you could write your own user space tool using the debugfs
+interface. A reference implementation is available at
+https://github.com/awslabs/damo. If you are a kernel programmer, you could
+refer to :doc:`/vm/damon/api` for the kernel space programming interface. For
+the reason, this document describes only the debugfs interface
+
+debugfs Interface
+=================
+
+DAMON exports three files, ``attrs``, ``target_ids``, and ``monitor_on`` under
+its debugfs directory, ``<debugfs>/damon/``.
+
+
+Attributes
+----------
+
+Users can get and set the ``sampling interval``, ``aggregation interval``,
+``regions update interval``, and min/max number of monitoring target regions by
+reading from and writing to the ``attrs`` file. To know about the monitoring
+attributes in detail, please refer to the :doc:`/vm/damon/design`. For
+example, below commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and
+1000, and then check it again::
+
+ # cd <debugfs>/damon
+ # echo 5000 100000 1000000 10 1000 > attrs
+ # cat attrs
+ 5000 100000 1000000 10 1000
+
+
+Target IDs
+----------
+
+Some types of address spaces supports multiple monitoring target. For example,
+the virtual memory address spaces monitoring can have multiple processes as the
+monitoring targets. Users can set the targets by writing relevant id values of
+the targets to, and get the ids of the current targets by reading from the
+``target_ids`` file. In case of the virtual address spaces monitoring, the
+values should be pids of the monitoring target processes. For example, below
+commands set processes having pids 42 and 4242 as the monitoring targets and
+check it again::
+
+ # cd <debugfs>/damon
+ # echo 42 4242 > target_ids
+ # cat target_ids
+ 42 4242
+
+Note that setting the target ids doesn't start the monitoring.
+
+
+Turning On/Off
+--------------
+
+Setting the files as described above doesn't incur effect unless you explicitly
+start the monitoring. You can start, stop, and check the current status of the
+monitoring by writing to and reading from the ``monitor_on`` file. Writing
+``on`` to the file starts the monitoring of the targets with the attributes.
+Writing ``off`` to the file stops those. DAMON also stops if every target
+process is terminated. Below example commands turn on, off, and check the
+status of DAMON::
+
+ # cd <debugfs>/damon
+ # echo on > monitor_on
+ # echo off > monitor_on
+ # cat monitor_on
+ off
+
+Please note that you cannot write to the above-mentioned debugfs files while
+the monitoring is turned on. If you write to the files while DAMON is running,
+an error code such as ``-EBUSY`` will be returned.
+
+
+Tracepoint for Monitoring Results
+=================================
+
+DAMON provides the monitoring results via a tracepoint,
+``damon:damon_aggregated``. While the monitoring is turned on, you could
+record the tracepoint events and show results using tracepoint supporting tools
+like ``perf``. For example::
+
+ # echo on > monitor_on
+ # perf record -e damon:damon_aggregated &
+ # sleep 5
+ # kill 9 $(pidof perf)
+ # echo off > monitor_on
+ # perf script
diff --git a/Documentation/admin-guide/mm/index.rst b/Documentation/admin-guide/mm/index.rst
index 4b14d8b50e9e..cbd19d5e625f 100644
--- a/Documentation/admin-guide/mm/index.rst
+++ b/Documentation/admin-guide/mm/index.rst
@@ -27,6 +27,7 @@ the Linux memory management.
concepts
cma_debugfs
+ damon/index
hugetlbpage
idle_page_tracking
ksm
diff --git a/Documentation/vm/damon/api.rst b/Documentation/vm/damon/api.rst
new file mode 100644
index 000000000000..08f34df45523
--- /dev/null
+++ b/Documentation/vm/damon/api.rst
@@ -0,0 +1,20 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=============
+API Reference
+=============
+
+Kernel space programs can use every feature of DAMON using below APIs. All you
+need to do is including ``damon.h``, which is located in ``include/linux/`` of
+the source tree.
+
+Structures
+==========
+
+.. kernel-doc:: include/linux/damon.h
+
+
+Functions
+=========
+
+.. kernel-doc:: mm/damon/core.c
diff --git a/Documentation/vm/damon/design.rst b/Documentation/vm/damon/design.rst
new file mode 100644
index 000000000000..b05159c295f4
--- /dev/null
+++ b/Documentation/vm/damon/design.rst
@@ -0,0 +1,166 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+======
+Design
+======
+
+Configurable Layers
+===================
+
+DAMON provides data access monitoring functionality while making the accuracy
+and the overhead controllable. The fundamental access monitorings require
+primitives that dependent on and optimized for the target address space. On
+the other hand, the accuracy and overhead tradeoff mechanism, which is the core
+of DAMON, is in the pure logic space. DAMON separates the two parts in
+different layers and defines its interface to allow various low level
+primitives implementations configurable with the core logic.
+
+Due to this separated design and the configurable interface, users can extend
+DAMON for any address space by configuring the core logics with appropriate low
+level primitive implementations. If appropriate one is not provided, users can
+implement the primitives on their own.
+
+For example, physical memory, virtual memory, swap space, those for specific
+processes, NUMA nodes, files, and backing memory devices would be supportable.
+Also, if some architectures or devices support special optimized access check
+primitives, those will be easily configurable.
+
+
+Reference Implementations of Address Space Specific Primitives
+==============================================================
+
+The low level primitives for the fundamental access monitoring are defined in
+two parts:
+
+1. Identification of the monitoring target address range for the address space.
+2. Access check of specific address range in the target space.
+
+DAMON currently provides the implementation of the primitives for only the
+virtual address spaces. Below two subsections describe how it works.
+
+
+VMA-based Target Address Range Construction
+-------------------------------------------
+
+Only small parts in the super-huge virtual address space of the processes are
+mapped to the physical memory and accessed. Thus, tracking the unmapped
+address regions is just wasteful. However, because DAMON can deal with some
+level of noise using the adaptive regions adjustment mechanism, tracking every
+mapping is not strictly required but could even incur a high overhead in some
+cases. That said, too huge unmapped areas inside the monitoring target should
+be removed to not take the time for the adaptive mechanism.
+
+For the reason, this implementation converts the complex mappings to three
+distinct regions that cover every mapped area of the address space. The two
+gaps between the three regions are the two biggest unmapped areas in the given
+address space. The two biggest unmapped areas would be the gap between the
+heap and the uppermost mmap()-ed region, and the gap between the lowermost
+mmap()-ed region and the stack in most of the cases. Because these gaps are
+exceptionally huge in usual address spaces, excluding these will be sufficient
+to make a reasonable trade-off. Below shows this in detail::
+
+ <heap>
+ <BIG UNMAPPED REGION 1>
+ <uppermost mmap()-ed region>
+ (small mmap()-ed regions and munmap()-ed regions)
+ <lowermost mmap()-ed region>
+ <BIG UNMAPPED REGION 2>
+ <stack>
+
+
+PTE Accessed-bit Based Access Check
+-----------------------------------
+
+The implementation for the virtual address space uses PTE Accessed-bit for
+basic access checks. It finds the relevant PTE Accessed bit from the address
+by walking the page table for the target task of the address. In this way, the
+implementation finds and clears the bit for next sampling target address and
+checks whether the bit set again after one sampling period. This could disturb
+other kernel subsystems using the Accessed bits, namely Idle page tracking and
+the reclaim logic. To avoid such disturbances, DAMON makes it mutually
+exclusive with Idle page tracking and uses ``PG_idle`` and ``PG_young`` page
+flags to solve the conflict with the reclaim logic, as Idle page tracking does.
+
+
+Address Space Independent Core Mechanisms
+=========================================
+
+Below four sections describe each of the DAMON core mechanisms and the five
+monitoring attributes, ``sampling interval``, ``aggregation interval``,
+``regions update interval``, ``minimum number of regions``, and ``maximum
+number of regions``.
+
+
+Access Frequency Monitoring
+---------------------------
+
+The output of DAMON says what pages are how frequently accessed for a given
+duration. The resolution of the access frequency is controlled by setting
+``sampling interval`` and ``aggregation interval``. In detail, DAMON checks
+access to each page per ``sampling interval`` and aggregates the results. In
+other words, counts the number of the accesses to each page. After each
+``aggregation interval`` passes, DAMON calls callback functions that previously
+registered by users so that users can read the aggregated results and then
+clears the results. This can be described in below simple pseudo-code::
+
+ while monitoring_on:
+ for page in monitoring_target:
+ if accessed(page):
+ nr_accesses[page] += 1
+ if time() % aggregation_interval == 0:
+ for callback in user_registered_callbacks:
+ callback(monitoring_target, nr_accesses)
+ for page in monitoring_target:
+ nr_accesses[page] = 0
+ sleep(sampling interval)
+
+The monitoring overhead of this mechanism will arbitrarily increase as the
+size of the target workload grows.
+
+
+Region Based Sampling
+---------------------
+
+To avoid the unbounded increase of the overhead, DAMON groups adjacent pages
+that assumed to have the same access frequencies into a region. As long as the
+assumption (pages in a region have the same access frequencies) is kept, only
+one page in the region is required to be checked. Thus, for each ``sampling
+interval``, DAMON randomly picks one page in each region, waits for one
+``sampling interval``, checks whether the page is accessed meanwhile, and
+increases the access frequency of the region if so. Therefore, the monitoring
+overhead is controllable by setting the number of regions. DAMON allows users
+to set the minimum and the maximum number of regions for the trade-off.
+
+This scheme, however, cannot preserve the quality of the output if the
+assumption is not guaranteed.
+
+
+Adaptive Regions Adjustment
+---------------------------
+
+Even somehow the initial monitoring target regions are well constructed to
+fulfill the assumption (pages in same region have similar access frequencies),
+the data access pattern can be dynamically changed. This will result in low
+monitoring quality. To keep the assumption as much as possible, DAMON
+adaptively merges and splits each region based on their access frequency.
+
+For each ``aggregation interval``, it compares the access frequencies of
+adjacent regions and merges those if the frequency difference is small. Then,
+after it reports and clears the aggregated access frequency of each region, it
+splits each region into two or three regions if the total number of regions
+will not exceed the user-specified maximum number of regions after the split.
+
+In this way, DAMON provides its best-effort quality and minimal overhead while
+keeping the bounds users set for their trade-off.
+
+
+Dynamic Target Space Updates Handling
+-------------------------------------
+
+The monitoring target address range could dynamically changed. For example,
+virtual memory could be dynamically mapped and unmapped. Physical memory could
+be hot-plugged.
+
+As the changes could be quite frequent in some cases, DAMON checks the dynamic
+memory mapping changes and applies it to the abstracted target area only for
+each of a user-specified time interval (``regions update interval``).
diff --git a/Documentation/vm/damon/faq.rst b/Documentation/vm/damon/faq.rst
new file mode 100644
index 000000000000..cb3d8b585a8b
--- /dev/null
+++ b/Documentation/vm/damon/faq.rst
@@ -0,0 +1,51 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==========================
+Frequently Asked Questions
+==========================
+
+Why a new subsystem, instead of extending perf or other user space tools?
+=========================================================================
+
+First, because it needs to be lightweight as much as possible so that it can be
+used online, any unnecessary overhead such as kernel - user space context
+switching cost should be avoided. Second, DAMON aims to be used by other
+programs including the kernel. Therefore, having a dependency on specific
+tools like perf is not desirable. These are the two biggest reasons why DAMON
+is implemented in the kernel space.
+
+
+Can 'idle pages tracking' or 'perf mem' substitute DAMON?
+=========================================================
+
+Idle page tracking is a low level primitive for access check of the physical
+address space. 'perf mem' is similar, though it can use sampling to minimize
+the overhead. On the other hand, DAMON is a higher-level framework for the
+monitoring of various address spaces. It is focused on memory management
+optimization and provides sophisticated accuracy/overhead handling mechanisms.
+Therefore, 'idle pages tracking' and 'perf mem' could provide a subset of
+DAMON's output, but cannot substitute DAMON.
+
+
+Does DAMON support virtual memory only?
+=======================================
+
+No. The core of the DAMON is address space independent. The address space
+specific low level primitive parts including monitoring target regions
+constructions and actual access checks can be implemented and configured on the
+DAMON core by the users. In this way, DAMON users can monitor any address
+space with any access check technique.
+
+Nonetheless, DAMON provides vma tracking and PTE Accessed bit check based
+implementations of the address space dependent functions for the virtual memory
+by default, for a reference and convenient use. In near future, we will
+provide those for physical memory address space.
+
+
+Can I simply monitor page granularity?
+======================================
+
+Yes. You can do so by setting the ``min_nr_regions`` attribute higher than the
+working set size divided by the page size. Because the monitoring target
+regions size is forced to be ``>=page size``, the region split will make no
+effect.
diff --git a/Documentation/vm/damon/index.rst b/Documentation/vm/damon/index.rst
new file mode 100644
index 000000000000..a2858baf3bf1
--- /dev/null
+++ b/Documentation/vm/damon/index.rst
@@ -0,0 +1,30 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==========================
+DAMON: Data Access MONitor
+==========================
+
+DAMON is a data access monitoring framework subsystem for the Linux kernel.
+The core mechanisms of DAMON (refer to :doc:`design` for the detail) make it
+
+ - *accurate* (the monitoring output is useful enough for DRAM level memory
+ management; It might not appropriate for CPU Cache levels, though),
+ - *light-weight* (the monitoring overhead is low enough to be applied online),
+ and
+ - *scalable* (the upper-bound of the overhead is in constant range regardless
+ of the size of target workloads).
+
+Using this framework, therefore, the kernel's memory management mechanisms can
+make advanced decisions. Experimental memory management optimization works
+that incurring high data accesses monitoring overhead could implemented again.
+In user space, meanwhile, users who have some special workloads can write
+personalized applications for better understanding and optimizations of their
+workloads and systems.
+
+.. toctree::
+ :maxdepth: 2
+
+ faq
+ design
+ api
+ plans
diff --git a/Documentation/vm/index.rst b/Documentation/vm/index.rst
index eff5fbd492d0..b51f0d8992f8 100644
--- a/Documentation/vm/index.rst
+++ b/Documentation/vm/index.rst
@@ -32,6 +32,7 @@ descriptions of data structures and algorithms.
arch_pgtable_helpers
balance
cleancache
+ damon/index
free_page_reporting
frontswap
highmem
--
2.17.1
From: SeongJae Park <[email protected]>
This commit updates MAINTAINERS file for DAMON related files.
Signed-off-by: SeongJae Park <[email protected]>
Reviewed-by: Markus Boehme <[email protected]>
---
MAINTAINERS | 11 +++++++++++
1 file changed, 11 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 13ec37ac23c7..1e56c3fe1bd3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -5128,6 +5128,17 @@ F: net/ax25/ax25_out.c
F: net/ax25/ax25_timer.c
F: net/ax25/sysctl_net_ax25.c
+DATA ACCESS MONITOR
+M: SeongJae Park <[email protected]>
+L: [email protected]
+S: Maintained
+F: Documentation/admin-guide/mm/damon/
+F: Documentation/vm/damon/
+F: include/linux/damon.h
+F: include/trace/events/damon.h
+F: mm/damon/
+F: tools/testing/selftests/damon/
+
DAVICOM FAST ETHERNET (DMFE) NETWORK DRIVER
L: [email protected]
S: Orphan
--
2.17.1
From: SeongJae Park <[email protected]>
DAMON is a data access monitoring framework for the Linux kernel. The
core mechanisms of DAMON make it
- accurate (the monitoring output is useful enough for DRAM level
performance-centric memory management; It might be inappropriate for
CPU cache levels, though),
- light-weight (the monitoring overhead is normally low enough to be
applied online), and
- scalable (the upper-bound of the overhead is in constant range
regardless of the size of target workloads).
Using this framework, hence, we can easily write efficient kernel space
data access monitoring applications. For example, the kernel's memory
management mechanisms can make advanced decisions using this.
Experimental data access aware optimization works that incurring high
access monitoring overhead could again be implemented on top of this.
Due to its simple and flexible interface, providing user space interface
would be also easy. Then, user space users who have some special
workloads can write personalized applications for better understanding
and optimizations of their workloads and systems.
===
Nevertheless, this commit is defining and implementing only basic access
check part without the overhead-accuracy handling core logic. The basic
access check is as below.
The output of DAMON says what memory regions are how frequently accessed
for a given duration. The resolution of the access frequency is
controlled by setting ``sampling interval`` and ``aggregation
interval``. In detail, DAMON checks access to each page per ``sampling
interval`` and aggregates the results. In other words, counts the
number of the accesses to each region. After each ``aggregation
interval`` passes, DAMON calls callback functions that previously
registered by users so that users can read the aggregated results and
then clears the results. This can be described in below simple
pseudo-code::
init()
while monitoring_on:
for page in monitoring_target:
if accessed(page):
nr_accesses[page] += 1
if time() % aggregation_interval == 0:
for callback in user_registered_callbacks:
callback(monitoring_target, nr_accesses)
for page in monitoring_target:
nr_accesses[page] = 0
if time() % update_interval == 0:
update()
sleep(sampling interval)
The target regions constructed at the beginning of the monitoring and
updated after each ``regions_update_interval``, because the target
regions could be dynamically changed (e.g., mmap() or memory hotplug).
The monitoring overhead of this mechanism will arbitrarily increase as
the size of the target workload grows.
The basic monitoring primitives for actual access check and dynamic
target regions construction aren't in the core part of DAMON. Instead,
it allows users to implement their own primitives that are optimized for
their use case and configure DAMON to use those. In other words, users
cannot use current version of DAMON without some additional works.
Following commits will implement the core mechanisms for the
overhead-accuracy control and default primitives implementations.
Signed-off-by: SeongJae Park <[email protected]>
Reviewed-by: Leonard Foerster <[email protected]>
Reviewed-by: Fernand Sieber <[email protected]>
Acked-by: Shakeel Butt <[email protected]>
---
include/linux/damon.h | 167 ++++++++++++++++++++++
mm/Kconfig | 2 +
mm/Makefile | 1 +
mm/damon/Kconfig | 15 ++
mm/damon/Makefile | 3 +
mm/damon/core.c | 320 ++++++++++++++++++++++++++++++++++++++++++
6 files changed, 508 insertions(+)
create mode 100644 include/linux/damon.h
create mode 100644 mm/damon/Kconfig
create mode 100644 mm/damon/Makefile
create mode 100644 mm/damon/core.c
diff --git a/include/linux/damon.h b/include/linux/damon.h
new file mode 100644
index 000000000000..2f652602b1ea
--- /dev/null
+++ b/include/linux/damon.h
@@ -0,0 +1,167 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ * DAMON api
+ *
+ * Author: SeongJae Park <[email protected]>
+ */
+
+#ifndef _DAMON_H_
+#define _DAMON_H_
+
+#include <linux/mutex.h>
+#include <linux/time64.h>
+#include <linux/types.h>
+
+struct damon_ctx;
+
+/**
+ * struct damon_primitive Monitoring primitives for given use cases.
+ *
+ * @init: Initialize primitive-internal data structures.
+ * @update: Update primitive-internal data structures.
+ * @prepare_access_checks: Prepare next access check of target regions.
+ * @check_accesses: Check the accesses to target regions.
+ * @reset_aggregated: Reset aggregated accesses monitoring results.
+ * @target_valid: Determine if the target is valid.
+ * @cleanup: Clean up the context.
+ *
+ * DAMON can be extended for various address spaces and usages. For this,
+ * users should register the low level primitives for their target address
+ * space and usecase via the &damon_ctx.primitive. Then, the monitoring thread
+ * (&damon_ctx.kdamond) calls @init and @prepare_access_checks before starting
+ * the monitoring, @update after each &damon_ctx.primitive_update_interval, and
+ * @check_accesses, @target_valid and @prepare_access_checks after each
+ * &damon_ctx.sample_interval. Finally, @reset_aggregated is called after each
+ * &damon_ctx.aggr_interval.
+ *
+ * @init should initialize primitive-internal data structures. For example,
+ * this could be used to construct proper monitoring target regions and link
+ * those to @damon_ctx.target.
+ * @update should update the primitive-internal data structures. For example,
+ * this could be used to update monitoring target regions for current status.
+ * @prepare_access_checks should manipulate the monitoring regions to be
+ * prepared for the next access check.
+ * @check_accesses should check the accesses to each region that made after the
+ * last preparation and update the number of observed accesses of each region.
+ * @reset_aggregated should reset the access monitoring results that aggregated
+ * by @check_accesses.
+ * @target_valid should check whether the target is still valid for the
+ * monitoring.
+ * @cleanup is called from @kdamond just before its termination.
+ */
+struct damon_primitive {
+ void (*init)(struct damon_ctx *context);
+ void (*update)(struct damon_ctx *context);
+ void (*prepare_access_checks)(struct damon_ctx *context);
+ void (*check_accesses)(struct damon_ctx *context);
+ void (*reset_aggregated)(struct damon_ctx *context);
+ bool (*target_valid)(void *target);
+ void (*cleanup)(struct damon_ctx *context);
+};
+
+/*
+ * struct damon_callback Monitoring events notification callbacks.
+ *
+ * @before_start: Called before starting the monitoring.
+ * @after_sampling: Called after each sampling.
+ * @after_aggregation: Called after each aggregation.
+ * @before_terminate: Called before terminating the monitoring.
+ * @private: User private data.
+ *
+ * The monitoring thread (&damon_ctx.kdamond) calls @before_start and
+ * @before_terminate just before starting and finishing the monitoring,
+ * respectively. Therefore, those are good places for installing and cleaning
+ * @private.
+ *
+ * The monitoring thread calls @after_sampling and @after_aggregation for each
+ * of the sampling intervals and aggregation intervals, respectively.
+ * Therefore, users can safely access the monitoring results without additional
+ * protection. For the reason, users are recommended to use these callback for
+ * the accesses to the results.
+ *
+ * If any callback returns non-zero, monitoring stops.
+ */
+struct damon_callback {
+ void *private;
+
+ int (*before_start)(struct damon_ctx *context);
+ int (*after_sampling)(struct damon_ctx *context);
+ int (*after_aggregation)(struct damon_ctx *context);
+ int (*before_terminate)(struct damon_ctx *context);
+};
+
+/**
+ * struct damon_ctx - Represents a context for each monitoring. This is the
+ * main interface that allows users to set the attributes and get the results
+ * of the monitoring.
+ *
+ * @sample_interval: The time between access samplings.
+ * @aggr_interval: The time between monitor results aggregations.
+ * @primitive_update_interval: The time between monitoring primitive updates.
+ *
+ * For each @sample_interval, DAMON checks whether each region is accessed or
+ * not. It aggregates and keeps the access information (number of accesses to
+ * each region) for @aggr_interval time. DAMON also checks whether the target
+ * memory regions need update (e.g., by ``mmap()`` calls from the application,
+ * in case of virtual memory monitoring) and applies the changes for each
+ * @primitive_update_interval. All time intervals are in micro-seconds.
+ * Please refer to &struct damon_primitive and &struct damon_callback for more
+ * detail.
+ *
+ * @kdamond: Kernel thread who does the monitoring.
+ * @kdamond_stop: Notifies whether kdamond should stop.
+ * @kdamond_lock: Mutex for the synchronizations with @kdamond.
+ *
+ * For each monitoring context, one kernel thread for the monitoring is
+ * created. The pointer to the thread is stored in @kdamond.
+ *
+ * Once started, the monitoring thread runs until explicitly required to be
+ * terminated or every monitoring target is invalid. The validity of the
+ * targets is checked via the &damon_primitive.target_valid of @primitive. The
+ * termination can also be explicitly requested by writing non-zero to
+ * @kdamond_stop. The thread sets @kdamond to NULL when it terminates.
+ * Therefore, users can know whether the monitoring is ongoing or terminated by
+ * reading @kdamond. Reads and writes to @kdamond and @kdamond_stop from
+ * outside of the monitoring thread must be protected by @kdamond_lock.
+ *
+ * Note that the monitoring thread protects only @kdamond and @kdamond_stop via
+ * @kdamond_lock. Accesses to other fields must be protected by themselves.
+ *
+ * @primitive: Set of monitoring primitives for given use cases.
+ * @callback: Set of callbacks for monitoring events notifications.
+ *
+ * @target: Pointer to the user-defined monitoring target.
+ */
+struct damon_ctx {
+ unsigned long sample_interval;
+ unsigned long aggr_interval;
+ unsigned long primitive_update_interval;
+
+/* private: internal use only */
+ struct timespec64 last_aggregation;
+ struct timespec64 last_primitive_update;
+
+/* public: */
+ struct task_struct *kdamond;
+ bool kdamond_stop;
+ struct mutex kdamond_lock;
+
+ struct damon_primitive primitive;
+ struct damon_callback callback;
+
+ void *target;
+};
+
+#ifdef CONFIG_DAMON
+
+struct damon_ctx *damon_new_ctx(void);
+void damon_destroy_ctx(struct damon_ctx *ctx);
+int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int,
+ unsigned long aggr_int, unsigned long primitive_upd_int);
+
+int damon_start(struct damon_ctx **ctxs, int nr_ctxs);
+int damon_stop(struct damon_ctx **ctxs, int nr_ctxs);
+
+#endif /* CONFIG_DAMON */
+
+#endif /* _DAMON_H */
diff --git a/mm/Kconfig b/mm/Kconfig
index 1f9bd3371765..504336de9a1e 100644
--- a/mm/Kconfig
+++ b/mm/Kconfig
@@ -886,4 +886,6 @@ config IO_MAPPING
config SECRETMEM
def_bool ARCH_HAS_SET_DIRECT_MAP && !EMBEDDED
+source "mm/damon/Kconfig"
+
endmenu
diff --git a/mm/Makefile b/mm/Makefile
index e3436741d539..709674b13497 100644
--- a/mm/Makefile
+++ b/mm/Makefile
@@ -128,3 +128,4 @@ obj-$(CONFIG_PTDUMP_CORE) += ptdump.o
obj-$(CONFIG_PAGE_REPORTING) += page_reporting.o
obj-$(CONFIG_IO_MAPPING) += io-mapping.o
obj-$(CONFIG_HAVE_BOOTMEM_INFO_NODE) += bootmem_info.o
+obj-$(CONFIG_DAMON) += damon/
diff --git a/mm/damon/Kconfig b/mm/damon/Kconfig
new file mode 100644
index 000000000000..d00e99ac1a15
--- /dev/null
+++ b/mm/damon/Kconfig
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+
+menu "Data Access Monitoring"
+
+config DAMON
+ bool "DAMON: Data Access Monitoring Framework"
+ help
+ This builds a framework that allows kernel subsystems to monitor
+ access frequency of each memory region. The information can be useful
+ for performance-centric DRAM level memory management.
+
+ See https://damonitor.github.io/doc/html/latest-damon/index.html for
+ more information.
+
+endmenu
diff --git a/mm/damon/Makefile b/mm/damon/Makefile
new file mode 100644
index 000000000000..4fd2edb4becf
--- /dev/null
+++ b/mm/damon/Makefile
@@ -0,0 +1,3 @@
+# SPDX-License-Identifier: GPL-2.0
+
+obj-$(CONFIG_DAMON) := core.o
diff --git a/mm/damon/core.c b/mm/damon/core.c
new file mode 100644
index 000000000000..651590bf49b1
--- /dev/null
+++ b/mm/damon/core.c
@@ -0,0 +1,320 @@
+// SPDX-License-Identifier: GPL-2.0
+/*
+ * Data Access Monitor
+ *
+ * Author: SeongJae Park <[email protected]>
+ */
+
+#define pr_fmt(fmt) "damon: " fmt
+
+#include <linux/damon.h>
+#include <linux/delay.h>
+#include <linux/kthread.h>
+#include <linux/slab.h>
+
+static DEFINE_MUTEX(damon_lock);
+static int nr_running_ctxs;
+
+struct damon_ctx *damon_new_ctx(void)
+{
+ struct damon_ctx *ctx;
+
+ ctx = kzalloc(sizeof(*ctx), GFP_KERNEL);
+ if (!ctx)
+ return NULL;
+
+ ctx->sample_interval = 5 * 1000;
+ ctx->aggr_interval = 100 * 1000;
+ ctx->primitive_update_interval = 60 * 1000 * 1000;
+
+ ktime_get_coarse_ts64(&ctx->last_aggregation);
+ ctx->last_primitive_update = ctx->last_aggregation;
+
+ mutex_init(&ctx->kdamond_lock);
+
+ ctx->target = NULL;
+
+ return ctx;
+}
+
+void damon_destroy_ctx(struct damon_ctx *ctx)
+{
+ if (ctx->primitive.cleanup)
+ ctx->primitive.cleanup(ctx);
+ kfree(ctx);
+}
+
+/**
+ * damon_set_attrs() - Set attributes for the monitoring.
+ * @ctx: monitoring context
+ * @sample_int: time interval between samplings
+ * @aggr_int: time interval between aggregations
+ * @primitive_upd_int: time interval between monitoring primitive updates
+ *
+ * This function should not be called while the kdamond is running.
+ * Every time interval is in micro-seconds.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int damon_set_attrs(struct damon_ctx *ctx, unsigned long sample_int,
+ unsigned long aggr_int, unsigned long primitive_upd_int)
+{
+ ctx->sample_interval = sample_int;
+ ctx->aggr_interval = aggr_int;
+ ctx->primitive_update_interval = primitive_upd_int;
+
+ return 0;
+}
+
+static bool damon_kdamond_running(struct damon_ctx *ctx)
+{
+ bool running;
+
+ mutex_lock(&ctx->kdamond_lock);
+ running = ctx->kdamond != NULL;
+ mutex_unlock(&ctx->kdamond_lock);
+
+ return running;
+}
+
+static int kdamond_fn(void *data);
+
+/*
+ * __damon_start() - Starts monitoring with given context.
+ * @ctx: monitoring context
+ *
+ * This function should be called while damon_lock is hold.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int __damon_start(struct damon_ctx *ctx)
+{
+ int err = -EBUSY;
+
+ mutex_lock(&ctx->kdamond_lock);
+ if (!ctx->kdamond) {
+ err = 0;
+ ctx->kdamond_stop = false;
+ ctx->kdamond = kthread_run(kdamond_fn, ctx, "kdamond.%d",
+ nr_running_ctxs);
+ if (IS_ERR(ctx->kdamond)) {
+ err = PTR_ERR(ctx->kdamond);
+ ctx->kdamond = 0;
+ }
+ }
+ mutex_unlock(&ctx->kdamond_lock);
+
+ return err;
+}
+
+/**
+ * damon_start() - Starts the monitorings for a given group of contexts.
+ * @ctxs: an array of the pointers for contexts to start monitoring
+ * @nr_ctxs: size of @ctxs
+ *
+ * This function starts a group of monitoring threads for a group of monitoring
+ * contexts. One thread per each context is created and run in parallel. The
+ * caller should handle synchronization between the threads by itself. If a
+ * group of threads that created by other 'damon_start()' call is currently
+ * running, this function does nothing but returns -EBUSY.
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int damon_start(struct damon_ctx **ctxs, int nr_ctxs)
+{
+ int i;
+ int err = 0;
+
+ mutex_lock(&damon_lock);
+ if (nr_running_ctxs) {
+ mutex_unlock(&damon_lock);
+ return -EBUSY;
+ }
+
+ for (i = 0; i < nr_ctxs; i++) {
+ err = __damon_start(ctxs[i]);
+ if (err)
+ break;
+ nr_running_ctxs++;
+ }
+ mutex_unlock(&damon_lock);
+
+ return err;
+}
+
+/*
+ * __damon_stop() - Stops monitoring of given context.
+ * @ctx: monitoring context
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+static int __damon_stop(struct damon_ctx *ctx)
+{
+ mutex_lock(&ctx->kdamond_lock);
+ if (ctx->kdamond) {
+ ctx->kdamond_stop = true;
+ mutex_unlock(&ctx->kdamond_lock);
+ while (damon_kdamond_running(ctx))
+ usleep_range(ctx->sample_interval,
+ ctx->sample_interval * 2);
+ return 0;
+ }
+ mutex_unlock(&ctx->kdamond_lock);
+
+ return -EPERM;
+}
+
+/**
+ * damon_stop() - Stops the monitorings for a given group of contexts.
+ * @ctxs: an array of the pointers for contexts to stop monitoring
+ * @nr_ctxs: size of @ctxs
+ *
+ * Return: 0 on success, negative error code otherwise.
+ */
+int damon_stop(struct damon_ctx **ctxs, int nr_ctxs)
+{
+ int i, err = 0;
+
+ for (i = 0; i < nr_ctxs; i++) {
+ /* nr_running_ctxs is decremented in kdamond_fn */
+ err = __damon_stop(ctxs[i]);
+ if (err)
+ return err;
+ }
+
+ return err;
+}
+
+/*
+ * damon_check_reset_time_interval() - Check if a time interval is elapsed.
+ * @baseline: the time to check whether the interval has elapsed since
+ * @interval: the time interval (microseconds)
+ *
+ * See whether the given time interval has passed since the given baseline
+ * time. If so, it also updates the baseline to current time for next check.
+ *
+ * Return: true if the time interval has passed, or false otherwise.
+ */
+static bool damon_check_reset_time_interval(struct timespec64 *baseline,
+ unsigned long interval)
+{
+ struct timespec64 now;
+
+ ktime_get_coarse_ts64(&now);
+ if ((timespec64_to_ns(&now) - timespec64_to_ns(baseline)) <
+ interval * 1000)
+ return false;
+ *baseline = now;
+ return true;
+}
+
+/*
+ * Check whether it is time to flush the aggregated information
+ */
+static bool kdamond_aggregate_interval_passed(struct damon_ctx *ctx)
+{
+ return damon_check_reset_time_interval(&ctx->last_aggregation,
+ ctx->aggr_interval);
+}
+
+/*
+ * Check whether it is time to check and apply the target monitoring regions
+ *
+ * Returns true if it is.
+ */
+static bool kdamond_need_update_primitive(struct damon_ctx *ctx)
+{
+ return damon_check_reset_time_interval(&ctx->last_primitive_update,
+ ctx->primitive_update_interval);
+}
+
+/*
+ * Check whether current monitoring should be stopped
+ *
+ * The monitoring is stopped when either the user requested to stop, or all
+ * monitoring targets are invalid.
+ *
+ * Returns true if need to stop current monitoring.
+ */
+static bool kdamond_need_stop(struct damon_ctx *ctx)
+{
+ bool stop;
+
+ mutex_lock(&ctx->kdamond_lock);
+ stop = ctx->kdamond_stop;
+ mutex_unlock(&ctx->kdamond_lock);
+ if (stop)
+ return true;
+
+ if (!ctx->primitive.target_valid)
+ return false;
+
+ return !ctx->primitive.target_valid(ctx->target);
+}
+
+static void set_kdamond_stop(struct damon_ctx *ctx)
+{
+ mutex_lock(&ctx->kdamond_lock);
+ ctx->kdamond_stop = true;
+ mutex_unlock(&ctx->kdamond_lock);
+}
+
+/*
+ * The monitoring daemon that runs as a kernel thread
+ */
+static int kdamond_fn(void *data)
+{
+ struct damon_ctx *ctx = (struct damon_ctx *)data;
+
+ mutex_lock(&ctx->kdamond_lock);
+ pr_info("kdamond (%d) starts\n", ctx->kdamond->pid);
+ mutex_unlock(&ctx->kdamond_lock);
+
+ if (ctx->primitive.init)
+ ctx->primitive.init(ctx);
+ if (ctx->callback.before_start && ctx->callback.before_start(ctx))
+ set_kdamond_stop(ctx);
+
+ while (!kdamond_need_stop(ctx)) {
+ if (ctx->primitive.prepare_access_checks)
+ ctx->primitive.prepare_access_checks(ctx);
+ if (ctx->callback.after_sampling &&
+ ctx->callback.after_sampling(ctx))
+ set_kdamond_stop(ctx);
+
+ usleep_range(ctx->sample_interval, ctx->sample_interval + 1);
+
+ if (ctx->primitive.check_accesses)
+ ctx->primitive.check_accesses(ctx);
+
+ if (kdamond_aggregate_interval_passed(ctx)) {
+ if (ctx->callback.after_aggregation &&
+ ctx->callback.after_aggregation(ctx))
+ set_kdamond_stop(ctx);
+ if (ctx->primitive.reset_aggregated)
+ ctx->primitive.reset_aggregated(ctx);
+ }
+
+ if (kdamond_need_update_primitive(ctx)) {
+ if (ctx->primitive.update)
+ ctx->primitive.update(ctx);
+ }
+ }
+
+ if (ctx->callback.before_terminate &&
+ ctx->callback.before_terminate(ctx))
+ set_kdamond_stop(ctx);
+ if (ctx->primitive.cleanup)
+ ctx->primitive.cleanup(ctx);
+
+ pr_debug("kdamond (%d) finishes\n", ctx->kdamond->pid);
+ mutex_lock(&ctx->kdamond_lock);
+ ctx->kdamond = NULL;
+ mutex_unlock(&ctx->kdamond_lock);
+
+ mutex_lock(&damon_lock);
+ nr_running_ctxs--;
+ mutex_unlock(&damon_lock);
+
+ do_exit(0);
+}
--
2.17.1
Hello, I am new user of this amazing tool.
I want to use this tool for Nvidia tx2 board with kernel version 4.9.140.
Do you guys have any timeline or update schedule for different kernel version compatibility?
Best
Dongjoo Seo
On Fri, Jul 23, 2021 at 04:02:44PM +0900, Dongjoo Seo wrote:
> Hello, I am new user of this amazing tool.
> I want to use this tool for Nvidia tx2 board with kernel version 4.9.140.
>
> Do you guys have any timeline or update schedule for different kernel version compatibility?
Use a newer kernel on that board, there's no need for anyone to port new
features to older kernel versions.
Why are you stuck with 4.9.140? That's very old and known insecure and
obsolete. Please contact the vendor who is providing it to you to give
you the latest version as you are paying them for that support.
good luck!
greg k-h
Thank you for your comment!.
It has been a long time since Nvidia ended supporting the new kernel version on tx2 boards, but for researchers, this set of boards (tx2, px2, agx) is a very attractive candidate to use.
That's why I asked. If it's impossible because the maintainer is busy with other work, I'll do it myself!
Best,
Dongjoo Seo
> On Jul 23, 2021, at 4:08 PM, Greg KH <[email protected]> wrote:
>
> On Fri, Jul 23, 2021 at 04:02:44PM +0900, Dongjoo Seo wrote:
>> Hello, I am new user of this amazing tool.
>> I want to use this tool for Nvidia tx2 board with kernel version 4.9.140.
>>
>> Do you guys have any timeline or update schedule for different kernel version compatibility?
>
> Use a newer kernel on that board, there's no need for anyone to port new
> features to older kernel versions.
>
> Why are you stuck with 4.9.140? That's very old and known insecure and
> obsolete. Please contact the vendor who is providing it to you to give
> you the latest version as you are paying them for that support.
>
> good luck!
>
> greg k-h
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?
A: Top-posting.
Q: What is the most annoying thing in e-mail?
A: No.
Q: Should I include quotations after my reply?
http://daringfireball.net/2007/07/on_top
On Fri, Jul 23, 2021 at 04:16:27PM +0900, Dongjoo Seo wrote:
> Thank you for your comment!.
>
> It has been a long time since Nvidia ended supporting the new kernel
> version on tx2 boards, but for researchers, this set of boards (tx2,
> px2, agx) is a very attractive candidate to use.
But why are you stuck at a known-buggy and insecure kernel version? I
would work on fixing that first, before worrying about new features.
good luck!
greg k-h
> On Jul 23, 2021, at 4:30 PM, Greg KH <[email protected]> wrote:
>
> A: Because it messes up the order in which people normally read text.
> Q: Why is top-posting such a bad thing?
> A: Top-posting.
> Q: What is the most annoying thing in e-mail?
>
> A: No.
> Q: Should I include quotations after my reply?
> http://daringfireball.net/2007/07/on_top
>
Sorry for the bad format email.
Thank you for your information. It really helps me with further mailing.
> On Fri, Jul 23, 2021 at 04:16:27PM +0900, Dongjoo Seo wrote:
>> Thank you for your comment!.
>>
>> It has been a long time since Nvidia ended supporting the new kernel
>> version on tx2 boards, but for researchers, this set of boards (tx2,
>> px2, agx) is a very attractive candidate to use.
>
> But why are you stuck at a known-buggy and insecure kernel version? I
> would work on fixing that first, before worrying about new features.
I need a board that has a GPU in my work.
However, Nvidia’s driver source code is not open-source.
I thought it would be easier to use the old compatible kernel with this tool than to use the Nvidia driver with one major version different kernel.
Thanks again for your advice on emailing!
Best
Dongjoo Seo
From: SeongJae Park <[email protected]>
Hello Dongjoo,
On Fri, 23 Jul 2021 16:02:44 +0900 Dongjoo Seo <[email protected]> wrote:
> Hello, I am new user of this amazing tool.
Thank you! It's always great to meet a new user!
> I want to use this tool for Nvidia tx2 board with kernel version 4.9.140.
>
> Do you guys have any timeline or update schedule for different kernel version compatibility?
I didn't have such a plan until now, but I think I can find a time slot to do
one-time back porting of current DAMON on 4.9.140. I think continuous support
would be not so easy, though. Nevertheless, if you need that, please feel free
to let me know. Once it's done, I will notify you.
Thanks,
SeongJae Park
>
> Best
> Dongjoo Seo
From: SeongJae Park <[email protected]>
On Fri, 23 Jul 2021 08:30:41 +0000 SeongJae Park <[email protected]> wrote:
> From: SeongJae Park <[email protected]>
>
> Hello Dongjoo,
>
>
> On Fri, 23 Jul 2021 16:02:44 +0900 Dongjoo Seo <[email protected]> wrote:
>
> > Hello, I am new user of this amazing tool.
>
> Thank you! It's always great to meet a new user!
>
> > I want to use this tool for Nvidia tx2 board with kernel version 4.9.140.
> >
> > Do you guys have any timeline or update schedule for different kernel version compatibility?
>
> I didn't have such a plan until now,
BTW, note that we are supporting latest two LTS kernels:
For people who want to test DAMON but using LTS kernels, there are another
couple of trees based on two latest LTS kernels respectively and containing the
'damon/master' backports.
- For v5.4.y: https://github.com/sjp38/linux/tree/damon/for-v5.4.y
- For v5.10.y: https://github.com/sjp38/linux/tree/damon/for-v5.10.y
Thanks,
SeongJae Park
[...]
From: SeongJae Park <[email protected]>
On Fri, 23 Jul 2021 08:30:41 +0000 SeongJae Park <[email protected]> wrote:
> From: SeongJae Park <[email protected]>
>
> Hello Dongjoo,
>
>
> On Fri, 23 Jul 2021 16:02:44 +0900 Dongjoo Seo <[email protected]> wrote:
>
> > Hello, I am new user of this amazing tool.
>
> Thank you! It's always great to meet a new user!
>
> > I want to use this tool for Nvidia tx2 board with kernel version 4.9.140.
> >
> > Do you guys have any timeline or update schedule for different kernel version compatibility?
>
> I didn't have such a plan until now, but I think I can find a time slot to do
> one-time back porting of current DAMON on 4.9.140. I think continuous support
> would be not so easy, though. Nevertheless, if you need that, please feel free
> to let me know. Once it's done, I will notify you.
Thank you for waiting. I back-ported this patchset and another one for the
physical address space monitoring[1] on v4.9.140. The source code is available
at DAMON development tree[2].
BTW, I can also agree to Greg's opinion. For the reason, I also back-ported
the patches on latest v4.9.y[3]. I'd like to suggest you to try that first if
possible. I also hope this patchset to be merged in mainline in near future,
so that NVIDIA be attracted by DAMON and thus forward-port their drivers on the
latest mainline version.
Nevertheless, DAMOS[4] and DAMON_RECLAIM[5] patchsets are not back-ported
because those need more works, and you are apparently interested in only the
monitoring part.
Please note that this backport is for only one-time. In other words, I will
not backport the patches for every future 4.9.y release. Nevertheless, if you
need another backport, please remember I'm here for you.
Finally, a disclaimer. I did only minimal test for this, as below:
$ git clone https://github.com/sjp38/masim; make -C masim/
$ git clone https://github.com/awslabs/damo
$ # monitor a virtual address space of a process
$ sudo ./damo/damo record "./masim/masim ./masim/configs/zigzag.cfg"
$ ./damo/damo report heats --heatmap stdout
[...]
88888888888888888888888888888888888888600000000000000000000000000000000000000000
88888888888888888888888888888888888888600000000000000000000000000000000000000000
44444444444444444444444444444444444444444444444444444444444444444444444444444200
00000000000000000000000000000000000000288888888888888888888888888888888888888400
00000000000000000000000000000000000000288888888888888888888888888888888888888400
55555555555555555555555555555555555555422222222222222222222222222222222222222100
88888888888888888888888888888888888888600000000000000000000000000000000000000000
88888888888888888888888888888888888888600000000000000000000000000000000000000000
[...]
# x-axis: space (140021718388736-140021924798416: 196.848 MiB)
# y-axis: time (247055083700-317341924740: 1 m 10.287 s)
# resolution: 80x40 (2.461 MiB and 1.757 s for each character)
$ # monitor the physcial address space of the system
$ ./masim/masim ./masim/configs/zigzag.cfg &
$ sudo ./damo/damo record paddr
00000000000000000000000000000000000000000000000000000000000000000000000000000000
00000000000000000000000000000000000000000000000000000000000000000000000000000020
00000000000000000000000000000000000000000000000000000000000000000000000000000040
[...]
# x-axis: space (4294967296-136303345616: 122.942 GiB)
# y-axis: time (356358198969-430518964169: 1 m 14.161 s)
# resolution: 80x40 (1.537 GiB and 1.854 s for each character)
Hope you get some fun with these backports.
[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://github.com/sjp38/linux/tree/damon/for-v4.9.140
[3] https://github.com/sjp38/linux/tree/damon/for-v4.9.276
[4] https://lore.kernel.org/linux-mm/[email protected]/
[5] https://lore.kernel.org/linux-mm/[email protected]/
Thanks,
SeongJae Park
[...]
(reduced CC list)
Hi all,
I have been asked to comment if Google is interested in using this
feature, its general usefulness and if it is sufficiently general and
non-duplicative. I will try to answer these but first I will explain
the use-cases we are particularly interested in and for which we want
a general access monitoring mechanism.
At the moment Google is particularly interested in four use-cases:
1) Working set estimation: This is used for cluster level scheduling
and controlling the knobs of memory overcommit.
2) Proactive reclaim
3) Balancing between memory tiers: Moving hot pages to fast tiers and
cold pages to slow tiers
4) Hugepage optimization: Hot memory backed by hugepages
In addition, these uses are not happening in isolation. We want a
combination of these running concurrently on a system. So, it is clear
that the first version or step of DAMON which only targets virtual
address space monitoring is not sufficient for these use-cases.
I think the more important question is if DAMON can be extended to
system level monitoring to fulfill these use-cases. Address space
monitoring is a core concept in DAMON and it has implemented address
space based optimizations (i.e. dividing address space into regions,
assuming locality within regions, random sampling within regions
instead of looking at each page and dynamically adjusting regions).
There is a followup proposal on monitoring physical address space in
DAMON. However for systems running multiple workloads, the address
space optimizations core to DAMON would be ineffective.
There are discussions/brainstorming on supporting abstract address
space based on LRUs which is somewhat similar to Multigen LRU [1]
proposal but not well articulated yet. BTW Multigen LRU [1] is another
similar proposal but targets one specific use-case i.e. memory reclaim
(proactive reclaim). Anyways I think we need more brainstorming for a
generalized solution of system level access monitoring.
Regarding merging DAMON, I personally think there are users who might
be interested in only their virtual address space and DAMON is
providing a solution for such users. SeongJae can provide more details
or knowledge if any big user other than Amazon is interested in the
feature. DAMON does not expose stable APIs at the moment, so these can
be changed later if needed. I think it is ok to merge DAMON for some
exposure. However I do want to make this clear that the solution space
is not complete. The solution of system level monitoring is still
needed which can be a future extension to DAMON or more generalized
Multigen LRU.
thanks,
Shakeel
[1] https://lore.kernel.org/lkml/[email protected]/
From: SeongJae Park <[email protected]>
Hello,
On Tue, 27 Jul 2021 14:30:38 -0700 Shakeel Butt <[email protected]> wrote:
> (reduced CC list)
>
> Hi all,
>
> I have been asked to comment if Google is interested in using this
> feature, its general usefulness and if it is sufficiently general and
> non-duplicative. I will try to answer these but first I will explain
> the use-cases we are particularly interested in and for which we want
> a general access monitoring mechanism.
Thank you for your great opinion below, Shakeel.
>
> At the moment Google is particularly interested in four use-cases:
>
> 1) Working set estimation: This is used for cluster level scheduling
> and controlling the knobs of memory overcommit.
>
> 2) Proactive reclaim
>
> 3) Balancing between memory tiers: Moving hot pages to fast tiers and
> cold pages to slow tiers
>
> 4) Hugepage optimization: Hot memory backed by hugepages
>
> In addition, these uses are not happening in isolation. We want a
> combination of these running concurrently on a system. So, it is clear
> that the first version or step of DAMON which only targets virtual
> address space monitoring is not sufficient for these use-cases.
>
> I think the more important question is if DAMON can be extended to
> system level monitoring to fulfill these use-cases.
I also think this is the important point. The main purpose of DAMON patchset
is providing a flexible monitoring framework which can easily extended for many
use cases. Once we have the framework, I believe people will be able to extend
it for their usages, and others will be able to reuse those (a snowball can
start rolling).
> Address space monitoring is a core concept in DAMON and it has implemented
> address space based optimizations (i.e. dividing address space into regions,
> assuming locality within regions, random sampling within regions instead of
> looking at each page and dynamically adjusting regions). There is a followup
> proposal on monitoring physical address space in DAMON. However for systems
> running multiple workloads, the address space optimizations core to DAMON
> would be ineffective.
Right. If the system is running a huge number of different workloads (e.g.,
systems running a huge number of virtual machines or kubernetes-managed
containers), DAMON's region-based monitoring's accuracy could be lowered.
However, I'd like to note that there are many people running only a small
number of major workloads on their systems. Even in the above case of systems
running a huge number of virtual machines, each virtual machine would have only
a small number of major workloads. People can use DAMON inside the guests. We
also confirmed the region-based physical address space monitoring on such
production systems achieves high accuracy (we found 4KB hottest memory region
in 70GB memory).
Also, the region-based monitoring is not mandatory. The followup proposal
which extends DAMON for physical address space monitoring[1] allows people opt
out it if they want. In addition to that, it implements a page-granularity
monitoring. I unsure if the implementation fits for Google's usage, but I sure
you can at least implement your own on the framework without the limitation of
the regions abstraction.
>
> There are discussions/brainstorming on supporting abstract address
> space based on LRUs which is somewhat similar to Multigen LRU [1]
> proposal but not well articulated yet. BTW Multigen LRU [1] is another
> similar proposal but targets one specific use-case i.e. memory reclaim
> (proactive reclaim). Anyways I think we need more brainstorming for a
> generalized solution of system level access monitoring.
The idea is using the positional index of each page in its LRU list as its
address. For example, a page at the head of a LRU list will have address 0.
On the address space, we can safely assume the pages adjacent in the address
scheme will have similar access frequency, and therefore DAMON's region-based
monitoring would work. Further, we can proactively move the pages in the LRU
list so that pages near the head of the list have higher frequencies, based on
the monitoring results.
For example, if we see below monitoring results from the address space:
<HEAD of a LRU list> HHHHHHHMMMMMMMCCCCHHCCCCC <TAIL of a LRU list>
(H: Hot page, M: Mid-temperature page, C: Cold page)
We can move the hot pages near the tail to the head, as below:
<HEAD of a LRU list> HHHHHHHHHMMMMMMMCCCCCCCCC <TAIL of a LRU list>
(H: Hot page, M: Mid-temperature page, C: Cold page)
This will improve not only monitoring accuracy but also other mechanisms such
as reclamation, which are based on the assumption of LRU list.
As Shakeel also told, this is only in a brainstorming stage, though.
>
> Regarding merging DAMON, I personally think there are users who might
> be interested in only their virtual address space and DAMON is
> providing a solution for such users. SeongJae can provide more details
> or knowledge if any big user other than Amazon is interested in the
> feature.
AFAIR, Huawei, Intel, and Alibaba shown some level of their interests publicly
and/or personally, so far. They did code review and/or tests and bug reports.
Also a number of researchers and individuals have reached out to me.
> DAMON does not expose stable APIs at the moment, so these can
> be changed later if needed. I think it is ok to merge DAMON for some
> exposure. However I do want to make this clear that the solution space
> is not complete. The solution of system level monitoring is still
> needed which can be a future extension to DAMON or more generalized
> Multigen LRU.
Agreed. We have lots more works to do. Some of those are already posted as
RFC patchsets[1,2,3,4]. I promise I will happily do the works. But, how dare
could only I get all the fun? I'd like to do that together with others in this
great community. One major purpose of this patchset is thus providing a
flexible framework for such collaboration. The virtual address space
monitoring, which this patchset provides in addition to the framework, is also
for real-world usages, though.
Now all the patches have at least one 'Reviewed-by:' or 'Acked-by:' tags. We
didn't find serious problems since v26[5], which was posted about four months
ago. so I'm thinking this patchset has passed the minimum qualification. If
you think there are more things to be done before this patchset is merged in
the -mm tree or mainline, please let me know. If not, Andrew, I'd like you to
consider merging this patchset into '-mm' tree.
Thanks,
SeongJae Park
>
> thanks,
> Shakeel
>
> [1] https://lore.kernel.org/lkml/[email protected]/
[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/
[3] https://lore.kernel.org/linux-mm/[email protected]/
[4] https://lore.kernel.org/linux-mm/[email protected]/
[5] https://lore.kernel.org/linux-mm/[email protected]/
From: SeongJae Park <[email protected]>
Hello Andrew,
On Wed, 28 Jul 2021 08:36:43 +0000 SeongJae Park <[email protected]> wrote:
[...]
> Now all the patches have at least one 'Reviewed-by:' or 'Acked-by:' tags. We
> didn't find serious problems since v26[5], which was posted about four months
> ago. so I'm thinking this patchset has passed the minimum qualification. If
> you think there are more things to be done before this patchset is merged in
> the -mm tree or mainline, please let me know. If not, Andrew, I'd like you to
> consider merging this patchset into '-mm' tree.
I'm wondering if you had a chance to consider that. If you had the chance but
this patchset didn't convince you, could you please let me know your concerns
so that I can make some progress?
Thanks,
SeongJae Park
[...]
From: SeongJae Park <[email protected]>
Hello Andrew,
On Mon, 2 Aug 2021 08:24:24 +0000 SeongJae Park <[email protected]> wrote:
> From: SeongJae Park <[email protected]>
>
> Hello Andrew,
>
> On Wed, 28 Jul 2021 08:36:43 +0000 SeongJae Park <[email protected]> wrote:
>
> [...]
> > Now all the patches have at least one 'Reviewed-by:' or 'Acked-by:' tags. We
> > didn't find serious problems since v26[5], which was posted about four months
> > ago. so I'm thinking this patchset has passed the minimum qualification. If
> > you think there are more things to be done before this patchset is merged in
> > the -mm tree or mainline, please let me know. If not, Andrew, I'd like you to
> > consider merging this patchset into '-mm' tree.
>
> I'm wondering if you had a chance to consider that. If you had the chance but
> this patchset didn't convince you, could you please let me know your concerns
> so that I can make some progress?
Because nearly three weeks passed since this patchset is posted, I considered
rebasing it on the latest -mm tree and posting it as v35. But, apparently it
makes no much sense because we found nothing to fix or improve. And, this
version can still cleanly be applied on top of the latest -mm tree. So,
instead of merely increasing the version number, I'd like to describe why I
believe this need to be merged into the -mm tree and eventually the mainline.
1. Merging this patchset will not bother other developers
Most changes in this patchset are for DAMON-dedicated new source files. There
is a change[1] for existing files, which makes PG_Idle independent of Idle Page
Tracking, but it is only small. Therefore, merging this patchset will not
increase the complexity of the other parts or introduce a regression.
2. Merging this patchset will not bother other users
DAMON utilizes a mechanism that designed to minimize and limit the monitoring
overhead. That said, DAMON can be opt out in the compile time for users who
don't want it. Even though it is compiled, it does nothing at all unless a
user explicitly asks it to do some works. Therefore, merging this patchset
will not silently introduce any additional overhead to users.
3. This patchset is deployed to real users
We are currently using DAMON patchset for profiling production workloads, as
described in 'Real-world User Story' section of the cover letter. It is also
deployed to real users other than us via Amazon Linux[2,3]. A few companies
and several researchers outside Amazon have publicly and/or privately shown
their interests in DAMON.
4. The downstream-only maintenance overhead is significant
Following development works based on DAMON[4,5,6] are also ongoing. Because
all the works are currently in downstream only, the maintenance overhead is not
small for us. Once DAMON is upstreamed, the overhead will significantly be
reduced.
5. This patchset is reviewed and apparently is stabilized
Since the first version of DAMON patchset is posted (2020-01-20), it has
evolved a lot. All patches of this patchset got at least one 'Reviewed-by:' or
'Acked-by:' tag by v31[7], which have posted about seven weeks ago
(2021-06-21). After that, we found and fixed only minor issues. We also got a
few more 'Acked-by:' tags. Since v34, which has posted about three weeks ago,
we found no more issues. We are also continuously running extensive
DAMON-dedicated tests. The tests include unit tests, self tests, functional
tests, performance tests, and static code analysis. Some of those are also
publicly available[8].
[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://github.com/amazonlinux/linux/tree/amazon-5.4.y/master/mm/damon
[3] https://github.com/amazonlinux/linux/tree/amazon-5.10.y/master/mm/damon
[4] https://lore.kernel.org/linux-mm/[email protected]/
[5] https://lore.kernel.org/linux-mm/[email protected]/
[6] https://lore.kernel.org/linux-mm/[email protected]/
[7] https://lore.kernel.org/linux-mm/[email protected]/
[8] https://github.com/awslabs/damon-tests
If you think above explanation makes sense, please consider merging this into
the -mm tree. Else, if this doesn't convince you, please let me know your
concerns or what I'm missing, so that I can make some progress.
Thanks,
SeongJae Park
[...]
On Wed, 28 Jul 2021 08:36:43 +0000 SeongJae Park <[email protected]> wrote:
> > DAMON does not expose stable APIs at the moment, so these can
> > be changed later if needed. I think it is ok to merge DAMON for some
> > exposure. However I do want to make this clear that the solution space
> > is not complete. The solution of system level monitoring is still
> > needed which can be a future extension to DAMON or more generalized
> > Multigen LRU.
>
> Agreed. We have lots more works to do. Some of those are already posted as
> RFC patchsets[1,2,3,4]. I promise I will happily do the works. But, how dare
> could only I get all the fun? I'd like to do that together with others in this
> great community. One major purpose of this patchset is thus providing a
> flexible framework for such collaboration. The virtual address space
> monitoring, which this patchset provides in addition to the framework, is also
> for real-world usages, though.
>
> Now all the patches have at least one 'Reviewed-by:' or 'Acked-by:' tags. We
> didn't find serious problems since v26[5], which was posted about four months
> ago. so I'm thinking this patchset has passed the minimum qualification. If
> you think there are more things to be done before this patchset is merged in
> the -mm tree or mainline, please let me know. If not, Andrew, I'd like you to
> consider merging this patchset into '-mm' tree.
Shall take a look. With some trepidation.
1-2 years from now someone will pop up with a massive patchset
implementing some monitoring scheme and we'll say "why didn't you use
DAMON" and they'll say "it's unsuitable for <reasons>".
I would like to see more thought/design go into how DAMON could be
modified to address Shakeel's other three requirements. At least to
the point where we can confidently say "yes, we will be able to do
this". Are you able to drive this discussion along please?
On Fri, 16 Jul 2021 08:14:36 +0000 SeongJae Park <[email protected]> wrote:
> DAMON is a data access monitoring framework for the Linux kernel.
Merged, thanks.
Presumably there are companion userspace tools for DAMON. Are they
available? Is there a plan to release and maintain these?
From: SeongJae Park <[email protected]>
On Thu, 5 Aug 2021 17:03:44 -0700 Andrew Morton <[email protected]> wrote:
> On Wed, 28 Jul 2021 08:36:43 +0000 SeongJae Park <[email protected]> wrote:
>
> > > DAMON does not expose stable APIs at the moment, so these can
> > > be changed later if needed. I think it is ok to merge DAMON for some
> > > exposure. However I do want to make this clear that the solution space
> > > is not complete. The solution of system level monitoring is still
> > > needed which can be a future extension to DAMON or more generalized
> > > Multigen LRU.
> >
> > Agreed. We have lots more works to do. Some of those are already posted as
> > RFC patchsets[1,2,3,4]. I promise I will happily do the works. But, how dare
> > could only I get all the fun? I'd like to do that together with others in this
> > great community. One major purpose of this patchset is thus providing a
> > flexible framework for such collaboration. The virtual address space
> > monitoring, which this patchset provides in addition to the framework, is also
> > for real-world usages, though.
> >
> > Now all the patches have at least one 'Reviewed-by:' or 'Acked-by:' tags. We
> > didn't find serious problems since v26[5], which was posted about four months
> > ago. so I'm thinking this patchset has passed the minimum qualification. If
> > you think there are more things to be done before this patchset is merged in
> > the -mm tree or mainline, please let me know. If not, Andrew, I'd like you to
> > consider merging this patchset into '-mm' tree.
>
> Shall take a look. With some trepidation.
>
> 1-2 years from now someone will pop up with a massive patchset
> implementing some monitoring scheme and we'll say "why didn't you use
> DAMON" and they'll say "it's unsuitable for <reasons>".
Agreed. And I personally believe merging this in will help avoiding such
situation, because the someone will be able to easily find the developer who is
responsible to convince the person. I will happily and definitely do my best
for that.
>
> I would like to see more thought/design go into how DAMON could be
> modified to address Shakeel's other three requirements. At least to
> the point where we can confidently say "yes, we will be able to do
> this". Are you able to drive this discussion along please?
Sure. I will describe my plan for convincing Shakeel's usages in detail as a
reply to this mail.
Thanks,
SeongJae Park
On Thu, 5 Aug 2021 17:43:24 -0700 Andrew Morton <[email protected]> wrote:
> On Fri, 16 Jul 2021 08:14:36 +0000 SeongJae Park <[email protected]> wrote:
>
> > DAMON is a data access monitoring framework for the Linux kernel.
>
> Merged, thanks.
Thank you!
>
> Presumably there are companion userspace tools for DAMON. Are they
> available? Is there a plan to release and maintain these?
Yes, the userspace tool[1] is available, released under GPLv2, and actively
being maintained. I am also planning to implement another basic user interface
in perf[2]. Also, the basic test suite for DAMON is available under GPLv2[3].
[1] https://github.com/awslabs/damo
[2] https://lore.kernel.org/linux-mm/[email protected]/
[3] https://github.com/awslabs/damon-tests
Thanks,
SeongJae Park
On Fri, 6 Aug 2021 11:48:30 +0000 SeongJae Park <[email protected]> wrote:
> >
> > Presumably there are companion userspace tools for DAMON. Are they
> > available? Is there a plan to release and maintain these?
>
> Yes, the userspace tool[1] is available, released under GPLv2, and actively
> being maintained. I am also planning to implement another basic user interface
> in perf[2]. Also, the basic test suite for DAMON is available under GPLv2[3].
>
> [1] https://github.com/awslabs/damo
> [2] https://lore.kernel.org/linux-mm/[email protected]/
> [3] https://github.com/awslabs/damon-tests
Ah. Useful info to have in the changelogs! I added the above words to the [0/n] introduction in mm-introduce-data-access-monitor-damon.patch
From: SeongJae Park <[email protected]>
On Fri, 6 Aug 2021 11:48:01 +0000 SeongJae Park <[email protected]> wrote:
> From: SeongJae Park <[email protected]>
>
> On Thu, 5 Aug 2021 17:03:44 -0700 Andrew Morton <[email protected]> wrote:
>
[...]
> >
> > I would like to see more thought/design go into how DAMON could be
> > modified to address Shakeel's other three requirements. At least to
> > the point where we can confidently say "yes, we will be able to do
> > this". Are you able to drive this discussion along please?
>
> Sure. I will describe my plan for convincing Shakeel's usages in detail as a
> reply to this mail.
Shakeel, I am explaining how DAMON will be extended and how it can be used for
your usages below. If there is any doubt or question, please feel free to let
me know.
What information DAMON (will) provides: contiguity, frequency, and recency
--------------------------------------------------------------------------
DAMON of this patchset informs users which memory region is how frequently
accessed. The memory region is a set of contiguous pages which having similar
access frequency. In addition to this, a following patch[1] will make DAMON to
track how long time the region maintained its size and access frequency. We
call this as 'age' of each region. That is, DAMON will be extended to provide
three attributes of data access patterns: contiguity (size of each region),
frequency, and recency.
Physical Address Space support
------------------------------
This version of DAMON is supporting only virtual address spaces of processes,
but will be extended to the physical address space[2]. The extension will be
quite simple because DAMON's monitoring primitives layer is separated from its
core logic.
How DAMON can be used for Shakeel's usages
------------------------------------------
The usages described in Shakeel's prior mail[1] are:
1) Working set estimation: This is used for cluster level scheduling
and controlling the knobs of memory overcommit.
2) Proactive reclaim
3) Balancing between memory tiers: Moving hot pages to fast tiers and
cold pages to slow tiers
4) Hugepage optimization: Hot memory backed by hugepages
In addition, these uses are not happening in isolation. We want a
combination of these running concurrently on a system. So, it is clear
that the first version or step of DAMON which only targets virtual
address space monitoring is not sufficient for these use-cases.
DAMON can satisfy all the usages as below.
- working set estimation: This can be done by iterating each region and
checking if the access frequency of it is higher than a threshold. Our user
space tool provides an implementation[3] for this. Below is a pseudo-code
for this:
workingsets = []
working_set_size = 0
for region in regions:
if region.access_frequncy > threshold:
workingsets.append(region)
working_set_size += region.end_address - region.start_address
return workingsets, working_set_size
- proactive reclaim: This can be done by iterating each region while checking
if it has zero access frequency and if its age is higher than a time
threshold, and reclaim those. We implemented this as a kernel module with
only 354 lines of code[4]. Below is a pseudo-code for this:
for region in regions:
if region.access_frquency == 0 and region.age > threshold:
reclaim(region)
- Balancing between memory tiers: Because DAMON provides access frequency, we
can know not only idle memory region but cold/cool/warm/hot memory region.
Once the functions for migrating pages from a tier to different tier is
matured, applying DAMON for this usage will be quite straightforward. That
is, for each region, if its access frequency and age is higher than
thresholds, migrate pages in the region to faster tier. If its access
frequency is lower than a threshold and its age is higher than a threshold,
migrate pages in the region to slower tier. Below is a pseudo-code for this:
for region in regions:
if region.age > age_threshod:
if region.access_frequency > hot_threshold:
migrate_to_fast_tier(region)
if region.access_frequency < cold_threshold:
migrate_to_slow_tier(region)
- Hugepage optimization: This will be quite similar to tiers balancing, but we
can use the size of regions. That is, we do monitoring of virtual address
spaces first. Then, for each region, if its access frequency, age, and size
are higher than thresholds (size threshold would be 2MB), makes the region to
be backed by huge pages. If the age and size are higher than thresholds but
the access frequency is lower than a threshold, makes the huge pages of the
region to be backed by regular pages. We evaluated this idea with a
prototype[5]. It removed 76.15% of THP memory overheads while preserving
51.25% of THP speedup. Below is a pseudo-code for this:
for region in regions:
if region.age > age_threshod and region.size >= 2 * MB:
if region.access_frequency > hot_threshold:
use_thps_for(region)
if region.access_frequency < cold_threshold:
use_regular_pages_for(region)
- Combination of these running concurrently: DAMON will be extended to be able
to monitor both the physical address space and virtual address spaces
simultaneously, like below.
struct damon_ctx *ctx_for_virt = damon_new_ctx();
struct damon_ctx *ctx_for_phys = damon_new_ctx();
struct damon_context *ctxs[] = {ctx_for_virt, ctx_for_phys};
[...]
/* first context for physical address space monitoring */
damon_pa_set_primitives(ctx_for_virt);
/* second context for virtual address spaces monitoring */
damon_va_set_primitives(ctx_for_phys);
damon_start(ctxs, 2);
Extending for page-granularity monitoring
-----------------------------------------
To my understanding, Shakeel wants to do above with page-granularity
monitoring. It will incur inevitable high overhead, but for someone who can
afford the cost, I will make DAMON to support it, as below.
Even with DAMON of this patchset, users can do the page-granularity monitoring
by simply setting the 'min_nr_regions' and 'max_nr_regions' of DAMON to the
number of pages in the target address space (nr_pages). Nevertheless, it will
result in creation of 'nr_pages' region structs. Assuming 4K pages, this will
result in about 1% memory waste, as each region struct consumes about 44 bytes
of memory. Our plan for removal of such overhead is as below.
In a future, the regions abstraction will be able to be entirely opted out[6].
In the case, no region structs will be allocated, so the memory overhead will
be zero. Nonetheless, the user will be required to configure DAMON to use a
special monitoring primitive which saves the monitoring results such as access
frequency and age in somewhere such as their own data structure or page flags,
like multi-gen LRU patchset does. If such data structure is commonly usable,
we can extend DAMON core to support it. To show how this will work, we
implemented a page-granularity idleness monitoring primitive with only 69 lines
of code[6].
Also, if someone has ideas for reducing the page granularity monitoring
overhead, we can put the optimization in the monitoring primitives layer, which
is separated from the core logic.
[1] https://lore.kernel.org/linux-mm/[email protected]/
[2] https://lore.kernel.org/linux-mm/[email protected]/
[3] https://github.com/awslabs/damo/blob/master/wss.py
[4] https://lore.kernel.org/linux-mm/[email protected]/
[5] https://damonitor.github.io/doc/html/latest/vm/damon/eval.html#efficient-thp
[6] https://github.com/sjp38/linux/commit/9e0cb168d30e
[7] https://lore.kernel.org/linux-mm/[email protected]/
Thanks,
SeongJae Park
