Received: by 2002:a05:6902:102b:0:0:0:0 with SMTP id x11csp886004ybt; Tue, 7 Jul 2020 02:41:45 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxwNUUAQ92tb70o2/jbD9Ih+/Hk/VqL1Jae37XvLWtL8V+XliFBvvJUpiW33gAs9CQzrZHS X-Received: by 2002:a50:cd53:: with SMTP id d19mr61626119edj.300.1594114904963; Tue, 07 Jul 2020 02:41:44 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594114904; cv=none; d=google.com; s=arc-20160816; b=Xj+Fi3HR5Yox5zMsa7L7pbZ1KdzqFE05O39/SJC23JakevgVXb2+DnYfr6qFLx2yqq xZGRi8tRIAL61sCaZ64oS5ys8GMXH/AG+NDiewgZrOWbFylgZjmJMgfdnggKOzDRzPBN 47jCbRLeUGj4agijTbxckyGn9+GJqSsuNnYYk6I3JUvKgnp2Hg0qYWlIsiQBEtZDB7tj 4JqcWTkSSFyBk/fkGtP3e/3P11ki5Deqx5Jr2kV9SSc1GYhyPaw2aPXsp3rYdQnqJO7p 1+M5KxzzNH2wluU4P1RtPZYsfuPlWOUnR8rttOzAkbD3MuOmbg3bX7V3EJUNVAB+9wzV IP1w== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:ironport-sdr:dkim-signature; bh=7QoswoiahhrG5EUF3hQrEfiZSC2O10x9Pf/apPsyVeg=; b=FtKtPfz+UsnekTO9zAWF2WoF4cfJpLYtPY+K/boFoEz3j85ygA2Ptrw6tyMJCu7K74 8uxuBnx0aad88OgzX/tAs9nBj46X/I5aSUM1WK3DjIQnucvsR/qD0Fem1oK4QgWCftSY ZO/c5iyXeKyUUf5+HA95DEnTaE6Q84awopNjz/uM8f1a0yLm3EFxa5WqDsgZE/F2Bj+Y v2t98Lc/RtvIBAy0aICEnKAXT8wH+llUiW2Nh9VXBPHxn6A+FucVnBu0yPToHIvGgjLT 0MUohcKs3WLAxX4cnqpLB2806NqR9Alu4WVGNJugtsTdyOt2To9dzUT1FR5VgKyE2R/A ZoGA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@amazon.com header.s=amazon201209 header.b=sUuVkjsz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amazon.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id g4si14506467edq.459.2020.07.07.02.41.20; Tue, 07 Jul 2020 02:41:44 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@amazon.com header.s=amazon201209 header.b=sUuVkjsz; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amazon.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1728162AbgGGJjF (ORCPT + 99 others); Tue, 7 Jul 2020 05:39:05 -0400 Received: from smtp-fw-9102.amazon.com ([207.171.184.29]:12398 "EHLO smtp-fw-9102.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1725825AbgGGJjE (ORCPT ); Tue, 7 Jul 2020 05:39:04 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1594114743; x=1625650743; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=7QoswoiahhrG5EUF3hQrEfiZSC2O10x9Pf/apPsyVeg=; b=sUuVkjsz39X8YBkP0ZZUpRv4UboukMlv6SKmTcNpE2YC1VMXiYcPPibv 7rgVMh+9bynoOEEGYsTYLaE5u679sWbw/+iwKZalc0ZTKpC/+UXOVS5+a KDAOD8XibRAsd/Um1TNpkFHmbtICuh0SLdDnpO7+ayj59dvYOdHP7tyrX 0=; IronPort-SDR: bOwZKzLi7B1NEq6k2ZvFULN0IhlnVpjZDjWC8tV+eJLExrwJMRFlGB1HxE++0/mtaetKNcYQvp WzrQTSHWeqwQ== X-IronPort-AV: E=Sophos;i="5.75,323,1589241600"; d="scan'208";a="57903384" Received: from sea32-co-svc-lb4-vlan3.sea.corp.amazon.com (HELO email-inbound-relay-1a-16acd5e0.us-east-1.amazon.com) ([10.47.23.38]) by smtp-border-fw-out-9102.sea19.amazon.com with ESMTP; 07 Jul 2020 09:38:55 +0000 Received: from EX13MTAUEA002.ant.amazon.com (iad55-ws-svc-p15-lb9-vlan2.iad.amazon.com [10.40.159.162]) by email-inbound-relay-1a-16acd5e0.us-east-1.amazon.com (Postfix) with ESMTPS id 5C143A1F80; Tue, 7 Jul 2020 09:38:43 +0000 (UTC) Received: from EX13D31EUA004.ant.amazon.com (10.43.165.161) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 7 Jul 2020 09:38:42 +0000 Received: from u886c93fd17d25d.ant.amazon.com (10.43.161.203) by EX13D31EUA004.ant.amazon.com (10.43.165.161) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Tue, 7 Jul 2020 09:38:25 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [RFC v13 0/8] Implement Data Access Monitoring-based Memory Operation Schemes Date: Tue, 7 Jul 2020 11:37:57 +0200 Message-ID: <20200707093805.4775-1-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-Originating-IP: [10.43.161.203] X-ClientProxiedBy: EX13D36UWA002.ant.amazon.com (10.43.160.24) To EX13D31EUA004.ant.amazon.com (10.43.165.161) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: SeongJae Park Changes from Previous Version ============================= - Wordsmith the document, comment, commit messages - Support a scheme of max access count 0 - Use 'unsigned long' for (min|max)_sz_region Introduction ============ DAMON[1] can be used as a primitive for data access awared memory management optimizations. For that, users who want such optimizations should run DAMON, read the monitoring results, analyze it, plan a new memory management scheme, and apply the new scheme by themselves. Such efforts will be inevitable for some complicated optimizations. However, in many other cases, the users would simply want the system to apply a memory management action to a memory region of a specific size having a specific access frequency for a specific time. For example, "page out a memory region larger than 100 MiB keeping only rare accesses more than 2 minutes", or "Do not use THP for a memory region larger than 2 MiB rarely accessed for more than 1 seconds". This RFC patchset makes DAMON to handle such data access monitoring-based operation schemes. With this change, users can do the data access aware optimizations by simply specifying their schemes to DAMON. [1] https://lore.kernel.org/linux-mm/20200706115322.29598-1-sjpark@amazon.com/ Evaluations =========== We evaluated DAMON's overhead, monitoring quality and usefulness using 25 realistic workloads on my QEMU/KVM based virtual machine running a kernel that v12 of this patchset is applied. An experimental DAMON-based operation scheme for THP, ‘ethp’, removes 31.29% of THP memory overheads while preserving 60.64% of THP speedup. Another experimental DAMON-based ‘proactive reclamation’ implementation, ‘prcl’, reduces 87.95% of residential sets and 29.52% of system memory footprint while incurring only 2.15% runtime overhead in the best case (parsec3/freqmine). NOTE that the experimentail THP optimization and proactive reclamation are not for production, just only for proof of concepts. Please refer to the official document[1] or "Documentation/admin-guide/mm: Add a document for DAMON" patch in the latest DAMON patchset for detailed evaluation setup and results. [1] https://damonitor.github.io/doc/html/latest-damos More Information ================ We prepared a showcase web site[1] that you can get more information. There are - the official documentations[2], - the heatmap format dynamic access pattern of various realistic workloads for heap area[3], mmap()-ed area[4], and stack[5] area, - the dynamic working set size distribution[6] and chronological working set size changes[7], and - the latest performance test results[8]. [1] https://damonitor.github.io/_index [2] https://damonitor.github.io/doc/html/latest-damos [3] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.0.html [4] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.1.html [5] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.2.html [6] https://damonitor.github.io/test/result/visual/latest/rec.wss_sz.html [7] https://damonitor.github.io/test/result/visual/latest/rec.wss_time.html [8] https://damonitor.github.io/test/result/perf/latest/html/index.html Baseline and Complete Git Tree ============================== The patches are based on the v5.7 plus v17 DAMON patchset[1] and Minchan's ``do_madvise()`` patch[2], which retrieved from the -next tree and slightly modified for backporting on v5.7. You can also clone the complete git tree: $ git clone git://github.com/sjp38/linux -b damos/rfc/v13 The web is also available: https://github.com/sjp38/linux/releases/tag/damos/rfc/v13 There are a couple of trees for entire DAMON patchset series that future features are included. The first one[3] contains the changes for latest release, while the other one[4] contains the changes for next release. [1] https://lore.kernel.org/linux-mm/20200706115322.29598-1-sjpark@amazon.com/ [2] https://lore.kernel.org/linux-mm/20200302193630.68771-2-minchan@kernel.org/ [3] https://github.com/sjp38/linux/tree/damon/master [4] https://github.com/sjp38/linux/tree/damon/next Sequence Of Patches =================== The 1st patch allows DAMON to reuse ``madvise()`` code for the actions. The 2nd patch accounts age of each region. The 3rd patch implements the handling of the schemes in DAMON and exports a kernel space programming interface for it. The 4th patch implements a debugfs interface for the privileged people and user programs. The 5th patch implements schemes statistics feature for easier tuning of the schemes and runtime access pattern analysis. The 6th patche adds selftests for these changes, and the 7th patch adds human friendly schemes support to the user space tool for DAMON. Finally, the 8th patch documents this new feature in the document. Patch History ============= Changes from RFC v12 (https://lore.kernel.org/linux-mm/20200616073828.16509-1-sjpark@amazon.com/) - Wordsmith the document, comment, commit messages - Support a scheme of max access count 0 - Use 'unsigned long' for (min|max)_sz_region Changes from RFC v11 (https://lore.kernel.org/linux-mm/20200609065320.12941-1-sjpark@amazon.com/) - Refine the commit messages (David Hildenbrand) - Clean up debugfs code Changes from RFC v10 (https://lore.kernel.org/linux-mm/20200603071138.8152-1-sjpark@amazon.com/) - Fix the wrong error handling for schemes debugfs file - Handle the schemes stats from the user space tool - Remove the schemes implementation plan from the document Changes from RFC v9 (https://lore.kernel.org/linux-mm/20200526075702.27339-1-sjpark@amazon.com/) - Rebase on v5.7 - Fix wrong comments and documents for schemes apply conditions Changes from RFC v8 (https://lore.kernel.org/linux-mm/20200512115343.27699-1-sjpark@amazon.com/) - Rewrite the document (Stefan Nuernberger) - Make 'damon_for_each_*' argument order consistent (Leonard Foerster) - Implement statistics for schemes - Avoid races between debugfs readers and writers - Reset age for only significant access frequency changes - Add kernel-doc comments in damon.h Please refer to RFC v8 for previous history SeongJae Park (8): mm/madvise: Export do_madvise() to external GPL modules mm/damon: Account age of target regions mm/damon: Implement data access monitoring-based operation schemes mm/damon/schemes: Implement a debugfs interface mm/damon/schemes: Implement statistics feature mm/damon/selftests: Add 'schemes' debugfs tests damon/tools: Support more human friendly 'schemes' control Documentation/admin-guide/mm: Document DAMON-based operation schemes Documentation/admin-guide/mm/damon/guide.rst | 41 ++- Documentation/admin-guide/mm/damon/plans.rst | 24 +- Documentation/admin-guide/mm/damon/start.rst | 11 + Documentation/admin-guide/mm/damon/usage.rst | 124 ++++++- include/linux/damon.h | 66 ++++ mm/damon.c | 344 +++++++++++++++++- mm/madvise.c | 1 + tools/damon/_convert_damos.py | 141 +++++++ tools/damon/_damon.py | 27 +- tools/damon/damo | 7 + tools/damon/schemes.py | 110 ++++++ .../testing/selftests/damon/debugfs_attrs.sh | 29 ++ 12 files changed, 880 insertions(+), 45 deletions(-) create mode 100755 tools/damon/_convert_damos.py create mode 100644 tools/damon/schemes.py -- 2.17.1