Received: by 2002:a05:6902:102b:0:0:0:0 with SMTP id x11csp94107ybt; Mon, 6 Jul 2020 04:59:42 -0700 (PDT) X-Google-Smtp-Source: ABdhPJyqNtDEU5T+alog6O53I2s3FoCdAdYw1XYUIHY1WKE9zW7446Bwu/iXnqb7o1+aOJFA4WZn X-Received: by 2002:a50:f109:: with SMTP id w9mr53267119edl.277.1594036781752; Mon, 06 Jul 2020 04:59:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1594036781; cv=none; d=google.com; s=arc-20160816; b=QDsNC882eC3NgA5bOVWrni5CCNn8hFmttX+IAENy/FFKCdH4TmoT+w94knM1c1dTZn ssZcEL/npY8+VBmuzSFH3ZqRYF2UVACJFfmRZM5AkLLKgMUHEwDAy0QhsuX8n0SMDndj GTyvv4K3bSGR7vRbrYVMCAEZ0EVVv5OUdb7aaBOynqnP5q2NOA3lLdiu2yfvXVlhJhYc N3Qj9l9Z14Ly27y2ceMrdWJkgzpKTvmRAgAwNzDOiZ9itJXPf7Lu94QlkMZQXJHvT2U0 DPjUkZNauIg8bW5bA/znaYo5Qk8JPw8WFzbD3Zbqc3kWYso+OpSS44nGW70yQQ167js5 A+LA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :ironport-sdr:dkim-signature; bh=ko0vpgy02JEFekxbxwTtiofkc4jbkaq/7gzJ34iGgAE=; b=zcHnoiw3SGsaA2gE5HhUUox+9oFjqAOGUQBvPkstEbF6OQ+d0+0CmIugpOB6/X/ihj 7eJ5FklBLNT6gjVb76X/4fNS6nVvCBUTsYaA4wVGtuTLh82ovxIeBDwaYvCH7hGNivUw QHLYZsptZkQeUVNMzdXwYmpaHED2JT6TjKT/2iqUcHL73Xalx8SLiCgRWZyoFaxaPvVN CVi/MjEYOoU+X6XT7ngyizM8dHXDQhPSC4IxpZmNs+n1g9z1+oMRtlH2ARYvfdC8A5y3 hJHlHBdtP+JwZmxyEoUMMfPLRnyiDdlxJ/eZgeeBRA++X5SC3gm9o8IlNpB/uw3sdyQt Gdcw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@amazon.com header.s=amazon201209 header.b="fMjT3u/R"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amazon.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id i23si13307261edx.261.2020.07.06.04.59.18; Mon, 06 Jul 2020 04:59:41 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@amazon.com header.s=amazon201209 header.b="fMjT3u/R"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=QUARANTINE sp=QUARANTINE dis=NONE) header.from=amazon.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729139AbgGFL6n (ORCPT + 99 others); Mon, 6 Jul 2020 07:58:43 -0400 Received: from smtp-fw-6002.amazon.com ([52.95.49.90]:19146 "EHLO smtp-fw-6002.amazon.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729062AbgGFL6h (ORCPT ); Mon, 6 Jul 2020 07:58:37 -0400 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=amazon.com; i=@amazon.com; q=dns/txt; s=amazon201209; t=1594036680; x=1625572680; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=ko0vpgy02JEFekxbxwTtiofkc4jbkaq/7gzJ34iGgAE=; b=fMjT3u/RsMc/ePKTtw9Wut3I8GZ0mC3gx6J6jjaOnW/dFISKUaLM3Q1e drBRbcXLWBniWpiILiqMm32dixEXbp/LabYn3Qdsf89fTumjaV6Yhr3lC eCDEZUx2/hJnS6hXT0DgOZbrv/9r24oT8+Eo6FC7bt6Ui8Ce6RLXfSFTo k=; IronPort-SDR: 9Hk813sbLvigrNVVqsCBajQQydwaAaU1y3AKa8p/0BpMkjwkikXr/6Ququq25kKF08lLUX8rFv gQpIV80b9trw== X-IronPort-AV: E=Sophos;i="5.75,318,1589241600"; d="scan'208";a="40227611" Received: from iad12-co-svc-p1-lb1-vlan3.amazon.com (HELO email-inbound-relay-2a-d0be17ee.us-west-2.amazon.com) ([10.43.8.6]) by smtp-border-fw-out-6002.iad6.amazon.com with ESMTP; 06 Jul 2020 11:57:53 +0000 Received: from EX13MTAUEA002.ant.amazon.com (pdx4-ws-svc-p6-lb7-vlan3.pdx.amazon.com [10.170.41.166]) by email-inbound-relay-2a-d0be17ee.us-west-2.amazon.com (Postfix) with ESMTPS id AA633A1E01; Mon, 6 Jul 2020 11:57:50 +0000 (UTC) Received: from EX13D31EUA004.ant.amazon.com (10.43.165.161) by EX13MTAUEA002.ant.amazon.com (10.43.61.77) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 6 Jul 2020 11:57:49 +0000 Received: from u886c93fd17d25d.ant.amazon.com (10.43.160.100) by EX13D31EUA004.ant.amazon.com (10.43.165.161) with Microsoft SMTP Server (TLS) id 15.0.1497.2; Mon, 6 Jul 2020 11:57:31 +0000 From: SeongJae Park To: CC: SeongJae Park , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , Subject: [PATCH v17 12/15] Documentation/admin-guide/mm: Add a document for DAMON Date: Mon, 6 Jul 2020 13:53:19 +0200 Message-ID: <20200706115322.29598-13-sjpark@amazon.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20200706115322.29598-1-sjpark@amazon.com> References: <20200706115322.29598-1-sjpark@amazon.com> MIME-Version: 1.0 Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: 8bit X-Originating-IP: [10.43.160.100] X-ClientProxiedBy: EX13D21UWA003.ant.amazon.com (10.43.160.184) To EX13D31EUA004.ant.amazon.com (10.43.165.161) Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: SeongJae Park This commit adds a document for DAMON under `Documentation/admin-guide/mm/damon/`. Signed-off-by: SeongJae Park --- Documentation/admin-guide/mm/damon/api.rst | 20 ++ .../admin-guide/mm/damon/damon_heatmap.png | Bin 0 -> 8366 bytes .../admin-guide/mm/damon/damon_wss_change.png | Bin 0 -> 7211 bytes .../admin-guide/mm/damon/damon_wss_dist.png | Bin 0 -> 6173 bytes Documentation/admin-guide/mm/damon/eval.rst | 222 +++++++++++++ Documentation/admin-guide/mm/damon/faq.rst | 59 ++++ .../admin-guide/mm/damon/freqmine_heatmap.png | Bin 0 -> 9999 bytes .../admin-guide/mm/damon/freqmine_wss_sz.png | Bin 0 -> 5589 bytes .../mm/damon/freqmine_wss_time.png | Bin 0 -> 6550 bytes Documentation/admin-guide/mm/damon/guide.rst | 194 ++++++++++++ Documentation/admin-guide/mm/damon/index.rst | 35 +++ .../admin-guide/mm/damon/mechanisms.rst | 159 ++++++++++ Documentation/admin-guide/mm/damon/plans.rst | 29 ++ Documentation/admin-guide/mm/damon/start.rst | 117 +++++++ .../mm/damon/streamcluster_heatmap.png | Bin 0 -> 42210 bytes .../mm/damon/streamcluster_wss_sz.png | Bin 0 -> 6327 bytes .../mm/damon/streamcluster_wss_time.png | Bin 0 -> 8830 bytes Documentation/admin-guide/mm/damon/usage.rst | 296 ++++++++++++++++++ Documentation/admin-guide/mm/index.rst | 1 + 19 files changed, 1132 insertions(+) create mode 100644 Documentation/admin-guide/mm/damon/api.rst create mode 100644 Documentation/admin-guide/mm/damon/damon_heatmap.png create mode 100644 Documentation/admin-guide/mm/damon/damon_wss_change.png create mode 100644 Documentation/admin-guide/mm/damon/damon_wss_dist.png create mode 100644 Documentation/admin-guide/mm/damon/eval.rst create mode 100644 Documentation/admin-guide/mm/damon/faq.rst create mode 100644 Documentation/admin-guide/mm/damon/freqmine_heatmap.png create mode 100644 Documentation/admin-guide/mm/damon/freqmine_wss_sz.png create mode 100644 Documentation/admin-guide/mm/damon/freqmine_wss_time.png create mode 100644 Documentation/admin-guide/mm/damon/guide.rst create mode 100644 Documentation/admin-guide/mm/damon/index.rst create mode 100644 Documentation/admin-guide/mm/damon/mechanisms.rst create mode 100644 Documentation/admin-guide/mm/damon/plans.rst create mode 100644 Documentation/admin-guide/mm/damon/start.rst create mode 100644 Documentation/admin-guide/mm/damon/streamcluster_heatmap.png create mode 100644 Documentation/admin-guide/mm/damon/streamcluster_wss_sz.png create mode 100644 Documentation/admin-guide/mm/damon/streamcluster_wss_time.png create mode 100644 Documentation/admin-guide/mm/damon/usage.rst diff --git a/Documentation/admin-guide/mm/damon/api.rst b/Documentation/admin-guide/mm/damon/api.rst new file mode 100644 index 000000000000..649409828eab --- /dev/null +++ b/Documentation/admin-guide/mm/damon/api.rst @@ -0,0 +1,20 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============= +API Reference +============= + +Kernel space programs can use every feature of DAMON using below APIs. All you +need to do is including ``damon.h``, which is located in ``include/linux/`` of +the source tree. + +Structures +========== + +.. kernel-doc:: include/linux/damon.h + + +Functions +========= + +.. kernel-doc:: mm/damon.c diff --git a/Documentation/admin-guide/mm/damon/damon_heatmap.png b/Documentation/admin-guide/mm/damon/damon_heatmap.png new file mode 100644 index 0000000000000000000000000000000000000000..ff485f9f59ca2717f5221b2824eaaed616b22c0e GIT binary patch literal 8366 zcmZu$2|QHa+rRfV!ysEip^VCs>|`m+$Wmj^E^B2;mZWTDxsghevSl~P8nW-(s3>cp zvW01}Wt+&p%zH<_|Nr;CZ_Q`SxaXYbe4p)n&+`ywpm&;)o{Js;07m>7bwdDvU;qF~ zm|e&S#4Pd&0CoWeI>s6l3I!P<5D3&SA^^Y?B0vG==7>1p?OQ0}wR89{CJ96n;o7+-&DLeE)tAL=^z&yxTPm zP*tP=UEP$u(Uc$qG7M8RfOXh&D((h({6T%COVu5QrKQgV7)!|n=7Q&9fZk!k@FN49 ziHkVA7l8Nfg>lFmhV6*39R;T9kE0Nfw;hGxK_L<;h&>cyXJsmxLMBvnHc^^7iADYt z9D#yuXlOvJ+Y8Vi0QMdR4hjK6qJWeXpePGyDggRQfCUb?pa!^T0@v|CfDUj+4|sSM zcwz*knF3knz>9NWxiwg43%)uJwq69^Ie;IXz(Hql)CK(R22EapX0AeHPiXNvwCoM7 z`tI8B+qHRf*H*x;?LZU-Mp15|C?V+WQ1sRv^u|4!^)Q;1aN4B@vyV^gNCE*;AIJbk^u+toJka1ZT1NW$*RK zVRy>quzt>IoWD=2V87}MF1bQ((ITG1#RvA69%L@#MVDiNibFu=P%Qw|2m*CN zK)o=~a13Y^1&CrmlLYWm5_lyIygm*z%K|O(z?&1mTScH%8E8`h+EsxLHK0=i=sFF& z!vpVifNnjY#{lRx1U?u8AI*SI7C@gB&~F0_*a3qVfgwlW^Ce)|1sHJ$Mm&JgYXHd` z_~HkA^#{HM0pEjxu~1AvFaRImvp45&Ywvi7vXOo`F^83y<0oM?h>G;A(blNnD<8NV~&G^ zQ&EYIMsod|{bu-v(QI`s73`TDg{Z6B_I?Apfk$8Y#n74CL?qEp)j87~#J?^H)P+V+a(jvkjyXJAUa zeRPZ|Xa#!XadpBFjwYy=HM7-Xccsk3=-q9j!>cupmYX93F@2>TR zOb4Gt{&pV*?i!N7zI{@?l}2us0%M&W&h52FaxFD11k(Gr%4qp?JGu=b#imxZtoD#8 zu9tZ`82Dl=&h%MJh;PWX8diE~TzjJbDxMY3QyKVRo+u`btW=DKg9So5`?l7@HdN0% zP8nD_+1axy;3b$}b~ntgQjvA|_J3A}!_Fh($D}{mchM$g4kOb$|ZUTr`iMs_`LnEoi$?mt{!oo)(G$g|fK?6OZwOls5wR+&Ib$I~Yb{ooRRj^DBE+CJ<&v-!)-16Y4n*i*grM?2 z5E}dcKuB}@13~{!gwy{I0Wmz}Ps2z4Fg(cV{~G>}4V8Z)aQ}%={s)5oAMWsV{Rd!A zBJ3=JDr}Hx(8}NQl(_Hwmj zl%J~b*O@*;=a@57*&V`ktIdLGn@cOpS|{kiTrleDRO_DNh3|@OD3SVCorY~%FHQE4 z=eoN;GA~hAgsFf3rZab*`K(iy?zr=(cTwgK!RY*NV{C6+L2d~X>mv9H&)mq#l_uw=<4}XY|Mh$o`=I$rk_TmS5?30O z&U@V|`w^Y*)N-ocO@9wx58pZ`j=Sq?BVT?dHL8fZ36NaG111*6$Gf@^4=p9FmQ@1=W(uodD;9XE3*T&xez|O+9G`-c06gFBF!nL=$2}` zYJBshGNd)wky>MCVgFWr`~F9?oNBCf9V{N`IB5D)m=tLfP!;6i>FMw5;qT#}!Fgga z`dePF;W*S#7gXfq>*?v~>yd7hnrzSOA>;MrFef>7xW0bt*;akM(w6;7(?}{)(zxf% zm#1^83*58ojm*b}A6zJT)sD57nvcGBLHRw`Z5dT|i#qtq$-U%jN#cz|w#4bvr|&AE zKPxH!I7V998bCh~7A2+Xx;ItKQ^+8x07(imG{CQWe%-dCgj?RJ5DSX46S^O+wQUjU?~G2wui zY>78_WQSOam{5wlp3kJC6cbg6L*Mq04`3!~p3l5QD=JtLD_i(UN~6$7`7M}*1?plz zL{H&t4IKDLWVC-D4f24I7e}ys<6t>&J_2`62@6>SoS)y|AiHs6Fd!ePZpZQ@M0$O% z2QyWm>N+^zkHaCe6QRNkARj>-k@8rD8XW$kXg)0%vU?9PHO!+&u??FnQ*CYuV%xb3IYN>?)}b)FsTr})jmM`cw|Y_GG^Z5^ ziD?sKB+uVC-?wrgYe(=|N95wh3!|MNZ01Hx4@+nAIN1?)=9)2QxexuJ+!BUW zimwXWy^M;En{KRi4t$!W==DpKe{;($_Q*|x zZ$q?-cVK=^VM2_~9tbjp;da(X+8j;~3K{Oe4jcj`+kq-2p7yc3Pj*x_7xWHN-b2l0 zmZFFDBHjTagj+~D+!ZH9y72^Q$7wZvzu*J*n%_(OxZ~lZ(8sP%Z9|J>49nS+c6=uL zG%9{dR$FvtQ76phz)6#eT3BxM4jhQu)aHI;rb#Mx-;NpKw)za4uQWP$Iz1Ob?N-Aj z1bA`x(suYA$=!bBPg}HfPsDFs_o`Z4bFEcTYoKkL`j&{G76w#fkzl_nM$kEaS@YZF zm>X^5P2?lBEh2*Ro-HO(^D9Rm_vr7ZZm5dudghuWe)%&YOG~l!g=G8X^9we~dQx#8 zvl4z=(=O}XSA_#(GcM@NzI6kuou?O~!={E^-?x{h_jDJqNce&Y4<^!-9v|vXxqK*V zzppUqi1xMP?VIWXH~Oq>TSWdT9&K@~i!uo-20i|jqA9W;#L8ifxcgUABd<*j=g%G! zpp2*8y8}0%HW`v+!7%$SeB$Ty+&8;E9m)P(P9`b80>D>n64M*&hf5>1YP^~ zEvJIHRgjsxLIexbN+XMgwpvC_kHG~cyv(V$>Mceq{4S7T+zD&L8jt&UiSLhc4raHVBC?>2Z}bX|D;^T)U8Ci5 zGPHN{-qI_TX~3VoL9H!O2$$T!+AVwSGjhyt{SngHEZy4mK~s10EirrlkoB>9j^8Frz!b;a7VGKRX6cOGMnSCXW@gnfu~e;57?f&Rj< z8o*s?rupawO5}BLWNYP9>fxAVKq7VfCHFMaJ$L@4}MJ9 z+Rp719ueOMy?6i%zf>eJqO9WJcv@F`FZ5MJgSKT& zidfWcGaNDjflv6c;~+OOTsT-YqGZ%|m4;?VZw{;YV=g*+20myi3lmQy_omLiCAyn4f@T`HtV!BM*aS<8`hC8C(61ig&V~d;@D^>HH%dDM z5ava==ZNv{zHL<;re7=ZMB1G@@H1BOkgPi`UsvJJq=}0#^?)wY5qWmB@bVSOsJ6z? z@KK4%jk~f}0xc#ThQ*?O?f&oOfJNmS)R=JAkm&P{k>Vmwf|;6_bGuzw7D$YkDtbtB z{7%9z>|K6RlPIR@5aeBbe(BKfm{d7`TTXg3<5G3q{O(0nVl1l!hoZ?*&Q~Wy*>)US z`{i=JmE#^&VmGGh{I~V84&%yO?xpwsi%z5uNkLA*2+FF`qABbWriu&FjKAl;!=c4G z>b5r-AwBJC39!#qf|U_`#qJ@(*4<}Q&kSZuoCDuTu?Eb>-emEhXJ8|vxpmalh>y@m zKDW*hj(C71_lX_ z!1cakd)Ot%MDEfd^o872X_Z$6vEjM17uzhDvMu%_I_F~(9vwxw0+5&n@yuDljt{{q zvJs&<)?VS%q$Z#l)i@L}pPq4@8&b zNmF9lmPayVXjb<&vUlrBBJYF=4q?{w7@{~LYiJT&#o%1ILWZjF>7T#LjQ-@qI>;#dX{eaizetFV_cdjxiM>th$5s{8aL2C zARjHhm7&{~oYb*cyW2AJY3&FO#|&++MNV*#Es{7uqpV8`r+JfOCXhFs-m(B`_AFh0cq^gOBj3n5r{o&l>Q+SO zPJvy}8Bq{XzEm)~RIrq?l-=mPz1cUJX0flUJ2-Hog<;y#tfI0)?)s?|cbAJd!(UXC zmb%5+est-%Y?~P~(dEEB2&`*_rRW z>rh-oou>ewAN| z!8Cli)r!FnYJ+sd?4+s_Pi6NQXkZB!L2>29OBZ``Ax$Mq;+c3!oZ^JcZl^+ZuPTK=j^Bs6xf?3KGm4|sOFjbd8O26ZNYcipbb!%<;?I|J- z#q;ZUIv7}s8N_H#|SUUSFCPEF&v~N}?)aqdF??NshlPdc)IljqYKCD~z82vtt_L?H7 z8Kl8%`7f5vJ`RP$ zImmgkOjk^Ek^`!>{m&rYSViyPC=g!p)P2W*nzx87-`x8PZ-zKs9dQL7ezUtIjhVw?{Y=y9%R zKWbAci3MF}!bTnrgKKZH6z#{zGw;1?1kRhF{TEx*i6?9S8KY)kz(3j7i*#BCkmp+J zGs}BU#Iw}N`vV0LJet+4AjSJEcf0$3N>sABUXgpHk)tO`<=r+ zZY@U%2C=q?$0ShT#r3!R!%lH{m9V5f+ju(> zKSwP}ujfiw3}t(Q4)y+swfll$^{-y;cRVN3_bnm~GSv@Fn0PPWhPR$hhbZMw87io$ z%SV67G{-$#Q15)KkN%gk>HyVh!SsGMw|%$G5gsD3>@?tPvgK~iJ5H_t2o|jVr}^F| zJ$G;V*(h{h%?FI+BQ)bs#yVNsG<@0B+@l$Dw_}$j@V)EmNRIF1aN|WImzkcb&Dg$@ zqn`R57;|T~()scD=K<_ycTIy~K8(EH|4B5Y$Nc|^h7>#K-v@RS={5h7SV{jzT!^8{ zlATOL=er?*az#U3yM7Dm_y4ggwa{h!Yew=`{uM>??~9RM{Vyt2F``^)k-Efv=S1%4 z|L>_e{`Z|K{<;(2Ul%uC-USX(8*lC%;`mSQhy3e_|CRf^e?O1lUw5AW(|T$x|4%gS z{|X!Mci6wX#^$fEjhBo#A2A558qOv>&Vmeiuyz;-Y5rasPH-({6~n*N`cOB+{*x;? zauMi^9qwJdr)z@KaAV+|dwBjDNIj~+P8rm%Tvfdz0>c))>35FV?Mxzsz8$<17cpz1 zq)(Gwj?SoFtf)edDh4UAmku3Vkm-2OfEmA#%+`(N(Ofg(0d`(n_=JH*V@WpY$ZMNN z4%miG7JjlzNu|YQ4ggKUMN8gNE-$X2g`X>42bfOpB6s{=&Ss z$!OcdSohJ0j6*Nz<^7%}Ht?N@`YCf%FqJQtbaej7het&v@NjPn^3N1yeC(t9;WqrT zep!aU@FyNoGG;=HJ^2VO-yw^J8Cq1PFO&(~=Dtx^(F7NPsY4^`F zQ-Lf#0tBY3XBX@6BAM@eMrJK~Zs!K#mwe%s_K$UVs4La<^_tCe2R`wZZFhpMsX#O`lYsHq^K{VS{(>6wzpeB4aFS z?WlF$b-Z}EkCQ%E!fM_@d@f@vYUYXCprEA&{gM_}toPy#cHfK+d5k1?d4IS2pql#y zsSt^4h7KHiDGAYS{^vxejJ^uB)u$;;lcjVZo_;Q?jfD*LC(tH#!8L&*~Ep zPfoCTU*=wzA*rk0u8gPq5$vRw=kvYCl||}iMXp}^27k&7w46V4Y3i4WRdSHNy4(F8 zRxulS>43Hyf)Ozt4fm%X$4EfSXOdb5=mSbR8}Cm?%M9I`v9|b~qks8)ANtjpxuu|; z^TQeSS1wNveqxFX)#^2VJ>Dj2rE6Wh_QUTgi&-fU$Z9Qe*@lkO3$++somp$w#C8q3 z)1e=KM$1XhK5L2{70#ZL+jOC32AznL5YLA0tMiMDYe{K5u{BUsDHdtWh_kr$BA=LH zc-`+&mXaFa_|=#w-?)z^MN@m?JqM?Kd!J+o)BcHU?>tO^ggKB6agFIDIia){r)hJ9 zf-21uHTC5}2b{~F+MGv=JeCF9pHy|$d<}!2_p2E1p6Cl!aCQTyJbBhmeB_Xj60A;Y zVHU;4Cpm)8w1#9Y?xKtbGMD6+!6xVus|2f6X0wd8_tV@{L}TVhlJqnT=^D61+8AL= z!8BTt8oP&nxFRdEE{06sz(5n&)GhH}ugQl{Xi>Q3^R3}_KGZ?HhMxM1Q+C1s1NaT3 A^#A|> literal 0 HcmV?d00001 diff --git a/Documentation/admin-guide/mm/damon/damon_wss_change.png b/Documentation/admin-guide/mm/damon/damon_wss_change.png new file mode 100644 index 0000000000000000000000000000000000000000..36699547c342cd05c9f416c78bc3fc86b9dc415e GIT binary patch literal 7211 zcmeHMcQ~8xyH6sBhKf-!T7uG|_G&3&SJkKzYDR}mDX|*$+FFzj+N#;sru192N7VRI zwMSKJS8Za|mh(n`<6PG{=fCsUxlXR+eV=hZ&;5-1bFVxwHq>Q0#(fL|fiUS^(lUWS zU;qRH9Y@fBEimSDgJ4C*_=>4Eg+c*aGBPr#FA@X-P)HC8WOG9GQb;5UxE~6sqdbjFA!n3z zG*B8kNF_uHHiLqxtE=PVP-PAv2DHWMikk6s5sXS{m+)z%jIjic@??lV^-OZ?#*JcKJ;CPsYY7|J%jk#O>|jo zTpG*e3@F^7`xDoNOJziq^yiraf!p;-ToV) z`ho*%Au;Z$Z6j{1Qp|y=mn1z`2T-We#)Sczm7FtA0iB@@+0E)69iN7bQzkfoNL#Ihc<=xQlieMwTZurVu~j(>4zhFk-0z6QLGhJ`8tTJ6tIE(7sCH$EVXHU zYJSv0k`v}jx($yN%Pt;_3jbhWXAdB;U?V?hXMg&kHtx!M-8hKW`;9B>MEQHW2|^%cg-|TgY)l zzEM_`czHNHIcZchrVob(^o0Nnc^dZ|Zy3GW{dOA9yIIA)*nd-Ge0_(#>rDdhXO`{6 z+!M8k*)XfonX!0vEStFjv)Y@;t1aHudM# zi~|Cl0{~hqRtbZL9yNn^b1q0%sVMt(sq6>LI5)!Ig#&I)12yUh1za)XdDl^%Oz?~H zNZ7j&Ajd(SB#$EZgY(kC(voSRU$LZX;GpEc{C=9ALE)$@?5UIgFBeJiv_GsQD|it@ zjzjUZpoGQ5o#-_1Vpz@r)R$@kJ69Bf3zhO0wd?X(*qbWGE>00o> zH9IOjZ!AvXHL{MtnF!do@<(!XRobV#;+-kGld|8awK39|3%?SUu;JB3)pSGev`{5o zQUXx--Tvpd*<9)fJ+yb01x@&vpnFJVivh}#7rPWP>DE)16lNk{xZCoSg&cfQHnxVA zQS{M2)n2&-C_l1=nSjCzVFOQt$kg2pE@Q{BN#b}ZL6v2sgvz2MdA_WLtGfPMj)cla z)m74J!FeuSl0s=v-(a)3mtB$*9d(L|sv=w)CX3=#LW?5f?6U@nBA&Ncy1qs4uD2VV z?KuWsfQ3Eb9%Z-N;gCyR4gwn$Ei9>;R*UDR0goJab;KF=jU^a%QB^`RP(QFp*w?EXBzk-{~|ps@TKtP={b55J$z9RtVYsFDTF5Nin1bxG>7iMpUp#| z?amekLiRzt*A37qhQ)VhL-j)d+I~ij@V}Vbs0>zlK%5O$;ZDmuX?WH9!CR z6C;?a#Od*MFEgkW?{#g0@(Ck{OjR@##%ll)d@lITuoMcU{WgUo{LdG435pdj`yaPU zL7K)r*#ZFizTJy|ID8GwPl9 zwcKW`8rQ(!8BXNfl)tUy0CJVKu^Lwcv~ucWc7nD=dO<`qm0?zp_v$=y^5rtV#I zIX0saVUc&>y+(z3QLnET-%IBOVlAHMR@t$1=$(^;o~X$muFV_7-h62utEY*L7mDVR z_5v?_WTNT}U-o@gPq%$*w}nWng`A>*?uEM^RbP4FHQt<$0o-}<-UqT??q4U$>YQDz z-9--Nc5csx-r>~|&bT|+9Hp+U80+0Pc;wG8oq4T4`oyDPuFC!vtVPGDrM6n{JrmR_ z3Dh*4gTAQ=0qA|sYOBm&R%jdYFI55gEbb7XHo^=?Xi(xWR&Ui;$ZLVN?O|?t^Gyp` z!l3sUU8@aS^-1S{Un+2oEc0+%j&y$E<+Mh4fWwk_8@b=H-3W7*F+6CFzpvx;l#ahg zvB<${hOK8T(TNtmC|x+A#IH)2^SVB<{bb?~@?V^Hw;`U`GW#>ZyAOs4_@d0B(gFxB ziN=3a3-hbu`0G`K;ESgV%`Afm-nIIw@R|zXQ;vvJS;1`$QUUTsmrW#)WvvXt*z%Ml zW5`h%4O5LZp@SveT`EaOa27}Rp-K{x229xprymb6aMrg)01;RctLSoq(*A2(aRw@W z*IMrrT^})lXeyD2FWl%vD2!x^_Xzu=5&m+8?Sf*pim~@bGK&nz5-75h#dFR6@dXbk?1lLTbIs>$@Hgl-yzWWY(l`T6Z(W{m&aT+xS@ag&0sC<=lJpbsFLVpiJ_(i^^;ZOG8jCId>ATo(LYCCgNkx+ zUD;JrjIl!XqVPYQJmgt|^4Sn#Fqlb5i2w$3;-5uPorWFs&ea*)y%g;-A?e&PEImE7_F{gC5!`u^w*3kiA8tvf%lx z7f2^<>$3+zuE}kK*QaiMuuU62V*SvgS-I?ARI)XhJ7|zpKXGT7rNm}fb6T9AE)u9!I0gI7 z;#&nM1V3CK4D7A;#)2e=mi}Q*yonGZCfgvok4(2TK z^_EppWKWBD-N*o}C@B1>gkxZjJwbmjLD1mtRtON)7X{sHzUWlTw9^Ie;Vv})$gmzzxDt|k5WSK*9`B%=WqiKg^Bfsqw`4Br)`SsykeE+U<{d+=PbF*FGw>+vQ z7%=mDLawhBSGG9{rgRswsn&1Jwl@#-UplF$W+J#fHkt%$fyQk0(Gr{s^c)9X{Giw6 z>7gA{0^J{!q^hB|G~2}Y;hn=Yk%96$Yw%TY1P2KRJK(VDdQhE z7Yd{i#|PWH8%d@jb9Mg?PrhU0)9`99`eaNL5EKz~b>QB;ln$Cl5B20b>FbU_dyyCR z8?7Uh!E=E8Pf(4L8GANM+4^()notm@+-yAcxi25w8Y@k9Pc_mya*-uK7a6pI6N0v= z?&ygSQ|Hs??5xms&GJrrH{aH zj1>d|0BB)JG8haCs)P}72@XF;bJSds3{YytPMKe`w}{fEgFXO&+gPjs2G4~`Kp-sW z;HP1*Bq-F`-(w#gWErLFD+mql6;>xPTN{!N%BQJO7XtnkY9E2ca-(pq&G7vdg*?zV z0!@JMIF0zYo_+dCv^CukiM27Qb}MTJM99!4{6~;o{%GZ$G3zozqD{(mix|x?fS{Zw z*+igqyX%B1@i?;?0cOMQszbL%pRc*NNPU_T!N+8Z!S`Fg!jfv=l2o4_Sq4$@ z_g9#zzOAWAIBx8Ps^i`dro9!R4`?~|l8_KJj*^MM2H9h3vD6`IHxm-$NJdYb{iT``8-uJL(){M6LY>e2D)lF!< zNY8q@E~7FkL9%(wNvGN;O>!)|>Vb8EJ(KKa$60V#^PDCKMMnJd=vD*LhAI$TC` zp;LPYKSrwVuWtVpOnYv7_1L-nn57vsjH7fZLKhc&5qx|+69_Hx^x&L5ZgAtig0rqH zOD!RHozjKI^(n3_!|zpDD98k0*;=QqSl%* zk-(^Oo%n2KPucc zR`P&qS}7{{HOgH%`PjdH&)l>*1Sn_irdxU|{SEYFjlKxKE?6Z3oO!y%|9yO>ckrhc=Co1h$nKI=ksf%a}7^;5BHUfuh9sI$s#& zz>IJ^ZPWK|l)zBc|%=5){xZZ#MBc_!}h+z(_GqM(%U zaN}%tyNHv(sAMO8eR^opEvTyYT8qWwcbXY7mz3k|oO`veJipehRb!W;ZnZL<;`KV# zv{c$Q68<@QwO`;<^i^zC^mM>gH^YQo&wM;vjc^>pZ*AzDK%u8)6Wc{Z{EeEFP>CA> z4zpaFwoOq_v#c`V|5s;{$ zIF~jy=h|iU0QRYA_Hwp_8j-=`^-{eXy#wYyZ9IoRWIaOQe!N3wA4)03Zp_`-@XQQ* z%gY}w`X=UuL{7yBCVbm1Nb%4v_FRijuSkWD?h{{btLs&7z)nnzg-uGsFHxpp2E8nU za3$f->YVoB3Xgo1f~rzNg(nXt85P)i-rBWsdUa|;^{~VfW!Cu0LKDBB zle9IT022%%H6N5_g|%yHTb%H9se{|}ePyMMKF>q_Rpu7vCL*razDeh8n7*`?MP!(b zqP&1_G`h=A-X3e8{&|f#$di__)hB`%Ag>hP7+J z>c(k|DotnzF?~wPVL%^R`|%F!@^s-#2jkw3$98YVy{rxowV2a;{{i;^r5cP>$TZ+5%b?DJOu7 szI*`uV@MpKGS3h7Qegb&4SI4NLMj77KkXD{OHkYNv<!8&_r>`Anc3kx7f(}GdK+my%t3qNA(E1jz%m{(UvvOydz4>lh* z{}9$k$|faVz?(QrgX}PnogKtO7YN!CL0cNQCqJH+4PCahY=Ho?U*HN zGjdDx_@fe896GEW3BBs~hb*h^Bei+}XyC$1c~F6Ab-BKC`Ap%~`g3mXFMZh;+-x&E z)+lMw^2=Lt8CS&XZPRmraH9obWUa1Gpb<){%Pr|_kJLa=&pj$~;q>d~<;}J*-Gv6Hl!f!xNQ(wy-c~7TgJR?nia~y`hS+Ev=*~YlTHkg@$qdON+&?fh&&lf) zHl|#B-`6eotP^GGGJLAWjc@xJj;HhOd~HR0T7~GECLY8t1`yuJzn9eEPa``V+7L5b z>6#}DeKV_!XHfA`&t&qus>!}+p!KXiWH_y`#G@66$I2) z@+@pLaZwFkGchPcPqJ$eCKN~QB}faRHEn8EwGd&`=PDbb@aP~)KNkj-D^2>3gV+aV zzUbT3w&x))e_miJL74TeTBls z0rm4}bj^v;(7t`G5$$}o`%3C&YY;`YkV0hBGQ&_x`uwYFR8oTnYJ+1ME%SJM`6ZJk zu@oN(CKv9o-cV}cLv66SBwd*7y_Ipr_l5PsazxYC+=;s$5zfT1YJTgIgh+nt@5mKl zq_Yn_eezUgE$iK<%&X39KzyV&+<9)wm-lHwg==cCQqsqw^+*9t;^M|Hz*SW&&2=W9 zD)sR3n>Xbq#Oi^xn@w$2Xm_uYkn6ingI-f@g!Cy@=@>9)_K8Y8Cu>ZsT$6@Tkuu?A zwq%J`Qg3e$0qFD|Jwa{7uS{FU1nVIyJX_LEwelka0pP&9)r+Sk0qU3I6aCHe&~Zx% zvov-Wj4Z0A#k;(idy(Qj<44vI`R-(V6%z`}l5)m9p0A+FfAMzqu4HL>$bYu8G#Cw! zR40I)kt_o*K7p_Kf@&(V)6U)UU+>!+t!`;0bQ0t%z> zJSKp()Omz{UJ0|OtVKdc#y7d-v=Jm+N4pgCxoKA9d*{XXSB)90H@a3RuiwQ#<9^A5F#p&trB|q}`l2Q-S`0pM z{RwN5vk$e~vJ}2EYr)Mn(mBtNdrt^(8e%%6!L+zau}oVj{2~dIRF1n);H`!8!=3!D z?}83tyz+Cvp4yv<%c=@~^w9RrAy$Nk(brf`@zQpOc{LGCj>5K`l`(G?G}y)w(0)E) zbPVzHJps)$P&(obJI!bV7bIV(U@wamD=6ez@`p_xhGv;FrQG8CT%OAXav=^!KL!jr zYzq-jD{v4j4p}iD29`a&g|#d&J^IA8KfF9)G<)HD)6jt7{>iCp00@d z<9GM*`GBygaJg=m7M}PIM^W1oDh7c4U?B>nz%GfD@Z9)lmc3Q=DQgd|e8A5R$8*5z zIeg|S{R12sU%}qZ!>g}r=q1$;{a53jwB2(xw=y0`&j|nrG>zcxf*MCYBxnke734VF zH#?_Qq*vr8i7+|#?tlsL&LHdQ1RONPE^{Zz!7Uz!3R;&9{-Ks(LA$8hB;dLMawvXB}x zEG(c7y}M!#nYIdC@X_W3;x#7hQszo?GwdKp{{XmHfV|WNK!`%HXMkeXn-IRI4_}ne zsc>69wxdmY!xXdjALhzfK->UnsZGn>s_Pa|BYHRvBf5O?5(YBG71(Y=g(*Up@Pl@2`)5){L)-@szlBOo5vW%k}(|$yQ6WwLlt8Z=$ z=Bh4^tw7$=x4afomFHAbQ)(eLy*O((@U*CZsr`p_Bh!KMEfVB3*PV9-A#JR>o$X|KG@s#CfU(lK9gv(N%<}(krb`>h( zz=`a0#3iRR(0MXh&#F>_VAIvPXJZxv{u)HP9&uZ_wlqbL{Mor0i5ZKo$E3#}ol_d) zg#E6|F;8F8pPYMk)tp+rI%~O3xEsQvned5mw~f(E?2LuVnLlRstQvG%XB?DoN`75^ z(lunQ8=<-r+Z=MA$$zmBq#gg#_*LY4KVx;9J!$C3o!M~Zo!C(wM39pO-L8FBVj7tD z>!R;K`p=rzH5L(e{UvR!7&V6>rVXF0M1@rQ_HLW%73TMmLyV#gcPf=?1)_E(1gG^a z;G^@{2hK^>Z#bkzUyVeU=P?`>!B&<>T6hR(Au=eJ=w6*i6<2U12C0+d-?zxjVga2e z33L&-2IZsB`OH!}Qv868IWC~7`hZ%wg>bPPr+a2?ELhg-f4DoDK?9Ax{De)DVG016 z^|q9b$+n>k9`Rwu=PW!;yFH+U{T{3gi=k5n6totEw9!qjEwfT#L7hQvf3Qn4rn z7FMJe<&F9f01AsN@GF0vzs$-#F#8~!S>;eSAKW$FmL!g+D^g~ zFFKw%u;KdqaoULPm7a=(9A~1*(wOdywM^0XmbjTutywK%n+vAq1K>e%p)$KwVnt^B z(U&KhSDe-ZOogDe8n`6NPN?(swY7?&p;dMi$eZFqd%0-qWj?4~5B|EWz0j(%MEcyc zIvZTuUpA|5HX$)6YHuBO$^J(8L#Y7xQ&HYL z)l7oWU~qrc&}>20kSUm1{A~%pW{!}vj|ED2ijc3{U=|(} z=((mgn{-9gjOwNgldlRHEIfbCS@zrn87rKZmj$7`M-?}qG@>efJSzpv>n#EdJ7!;e z#lqhS0mqG*&BW`k5Zn8?gI)f_oN%nLV0z|H1H9WOMc?-|-t+_4M7~oT(s?jZJvH=} z>xFH*-a39*8BUi%`hptQKQXA(ZTbiqSHbE2qU~ohN~Q4-Q*4sc8~v}Z?04?BL%|5W zABB!IIS}1;0!rCcrO;w57Otu!MoXu zh99`6V^EpBb&PANe0^iuTZU1UkBmMBu>uFw3+xVgpxx1mteP=*!Moktm(w?1$jG7; zPz0pDlqV}n-ru10cEQ6f9>ih9dw7n>)JDNxY8vKH!O{Ib#_DtdIT=Sj1T~J9z{g;F z22%~TOHe;(pS<8GMc&r8$YzYt^Mq>zHFT7j9yT7bq0>$lxSYfegSTdDi2P;WXR8hX z@oEzuMD8SRhVaz)6QEM3=*Cf$0(~r&Q$@r#6BF0?z#&?w`T zEIq~FYs+yZ8lq-%uz(fih^J|8)*Zk{ytrEz*eGCKYHXb2_z(*!8p zKXrx;aX4%ZkiufF4@}kL$T%oCCLi@DojuwJz$3Hk!Tv$?U2jXe=DbL-eO_l={yW+o zrHJ})u-^Tt2KEybgl-~l6|y=@V1v?r#)@)H$2SGX3OVcskCV!-QNfie`Z{(-Qfk;| zwReHV0LC3v`2p9_)E zE0l0!#t>@7wWrP`g;{Rnxp@LK|1N%WP!DPQpWLQE)n$eH*rxpy$>ks1-B z(B3H9$0yR~g34PGe{ydx3$HET3UwTd!n4EyKZc&2Pxea!n0$d^at)$nS3Q0ydUiVe+#_7J zV?X#$;=Hm^x>%MV1>;~8s^e>HZRV`C)*gdfnRKyAC?^%|lg)X2zK(^G{v0rb>BQj_OY$BbN9*#K6xJ%U&mS2 zGijqGK@zo!2V5B}@2q;auNYg`yJ0ad+h&_5YqyJZ^J}S(^{lvl_`Kmy38*jROowuR z8j%gC4>;|=7Hj{=M$3|TF7syftQ&*+;K?9=Nght_tW!-PJHC$sHw&T_J`qCJdYSIP zpF)*{X(^NOz>1seH*w}3KO>~a7suj5EQo~}R(t{M3nrS1Mg>o!1kAf^a^KeL(ZQwP z&#}#>ks0zs_?9?>%W6R%F zDy5~MUtDD4+x7G-JGb%On!-JOOahz3 z^Xk(+DFo?t%7b^BMp+t(;<>?&)p??(Z~sjA@MCAaBIVe%4431BhDUWdUm8jtl(@=q zp(JXe?$}A}vv2ww%J*iqVp3ptQi?Zfgw3c|y(pA+j)`OjzG5#nJH1p(%W}NlaSRAZZQfcecU)uUGf($$NjY9nPoE9V_i!chW`>2eu;M@tAE z5_mA{PW_|wbM7e4`lDF=v`F9uI1(MukrXLgG4jt1|FiMXeIxmptopCDTBM_fSl^fs zpf0lg3Mv6Kb03EwWCO=NG>? zKKT9B`DFkxd9i1BY;x--z#Ib(p9LGoe?G0VubfR4kL(D>_V`x?W_2;Z1$;6YHcG4R z9KTDGWQM)nRGak{jdoYEY$#R!atFKMbC6sr0ql;h*}kfkjC&Pgy;%qjYgE8Ot`{!k z^OTj4UUFA|0k>M%PLgj!Q(Xo?XPf2@=jGzzV*&>RkagWOFk#TYN$OqMM-wiCQofOi zF)`bCk2z$i2z)nsIDw8j%op#?__$@PrY_qv(gS}U%OXHzW&OKQ_4menTcnlxbfjHh Rl5p7TIjd!;S)y(m_Fp#ycijL0 literal 0 HcmV?d00001 diff --git a/Documentation/admin-guide/mm/damon/eval.rst b/Documentation/admin-guide/mm/damon/eval.rst new file mode 100644 index 000000000000..b233890b4e45 --- /dev/null +++ b/Documentation/admin-guide/mm/damon/eval.rst @@ -0,0 +1,222 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========== +Evaluation +========== + +DAMON is lightweight. It increases system memory usage by only -0.25% and +consumes less than 1% CPU time in most case. It slows target workloads down by +only 0.94%. + +DAMON is accurate and useful for memory management optimizations. An +experimental DAMON-based operation scheme for THP, 'ethp', removes 31.29% of +THP memory overheads while preserving 60.64% of THP speedup. Another +experimental DAMON-based 'proactive reclamation' implementation, 'prcl', +reduces 87.95% of residential sets and 29.52% of system memory footprint while +incurring only 2.15% runtime overhead in the best case (parsec3/freqmine). + +Setup +===== + +On a QEMU/KVM based virtual machine utilizing 20GB of RAM and hosted by an +Intel i7 machine that running a kernel that v16 DAMON patchset is applied, I +measure runtime and consumed system memory while running various realistic +workloads with several configurations. I use 13 and 12 workloads in PARSEC3 +[3]_ and SPLASH-2X [4]_ benchmark suites, respectively. I use another wrapper +scripts [5]_ for convenient setup and run of the workloads. + +Measurement +----------- + +For the measurement of the amount of consumed memory in system global scope, I +drop caches before starting each of the workloads and monitor 'MemFree' in the +'/proc/meminfo' file. To make results more stable, I repeat the runs 5 times +and average results. + +Configurations +-------------- + +The configurations I use are as below. + +- orig: Linux v5.7 with 'madvise' THP policy +- rec: 'orig' plus DAMON running with virtual memory access recording +- prec: 'orig' plus DAMON running with physical memory access recording +- thp: same with 'orig', but use 'always' THP policy +- ethp: 'orig' plus a DAMON operation scheme, 'efficient THP' +- prcl: 'orig' plus a DAMON operation scheme, 'proactive reclaim [6]_' + +I use 'rec' for measurement of DAMON overheads to target workloads and system +memory. 'prec' is for physical memory monitroing and recording. It monitors +17GB sized 'System RAM' region. The remaining configs including 'thp', 'ethp', +and 'prcl' are for measurement of DAMON monitoring accuracy. + +'ethp' and 'prcl' are simple DAMON-based operation schemes developed for +proof of concepts of DAMON. 'ethp' reduces memory space waste of THP by using +DAMON for the decision of promotions and demotion for huge pages, while 'prcl' +is as similar as the original work. Those are implemented as below:: + + # format: + # ethp: Use huge pages if a region shows >=5% access rate, use regular + # pages if a region >=2MB shows <5% access rate for >=13 seconds + null null 5 null null null hugepage + 2M null null null 13s null nohugepage + + # prcl: If a region >=4KB shows <=5% access rate for >=7 seconds, page out. + 4K null null 5 7s null pageout + +Note that both 'ethp' and 'prcl' are designed with my only straightforward +intuition because those are for only proof of concepts and monitoring accuracy +of DAMON. In other words, those are not for production. For production use, +those should be more tuned. + +.. [1] "Redis latency problems troubleshooting", https://redis.io/topics/latency +.. [2] "Disable Transparent Huge Pages (THP)", + https://docs.mongodb.com/manual/tutorial/transparent-huge-pages/ +.. [3] "The PARSEC Becnhmark Suite", https://parsec.cs.princeton.edu/index.htm +.. [4] "SPLASH-2x", https://parsec.cs.princeton.edu/parsec3-doc.htm#splash2x +.. [5] "parsec3_on_ubuntu", https://github.com/sjp38/parsec3_on_ubuntu +.. [6] "Proactively reclaiming idle memory", https://lwn.net/Articles/787611/ + +Results +======= + +Below two tables show the measurement results. The runtimes are in seconds +while the memory usages are in KiB. Each configuration except 'orig' shows +its overhead relative to 'orig' in percent within parenthesizes.:: + + runtime orig rec (overhead) prec (overhead) thp (overhead) ethp (overhead) prcl (overhead) + parsec3/blackscholes 107.228 107.859 (0.59) 108.110 (0.82) 107.381 (0.14) 106.811 (-0.39) 114.766 (7.03) + parsec3/bodytrack 79.292 79.609 (0.40) 79.777 (0.61) 79.313 (0.03) 78.892 (-0.50) 80.398 (1.40) + parsec3/canneal 148.887 150.878 (1.34) 153.337 (2.99) 127.873 (-14.11) 132.272 (-11.16) 167.631 (12.59) + parsec3/dedup 11.970 11.975 (0.04) 12.024 (0.45) 11.752 (-1.82) 11.921 (-0.41) 13.244 (10.64) + parsec3/facesim 212.800 215.927 (1.47) 215.004 (1.04) 205.117 (-3.61) 207.401 (-2.54) 220.834 (3.78) + parsec3/ferret 190.646 192.560 (1.00) 192.414 (0.93) 190.662 (0.01) 192.309 (0.87) 193.497 (1.50) + parsec3/fluidanimate 213.951 216.459 (1.17) 217.578 (1.70) 209.500 (-2.08) 211.826 (-0.99) 218.299 (2.03) + parsec3/freqmine 291.050 292.117 (0.37) 293.279 (0.77) 289.553 (-0.51) 291.768 (0.25) 297.309 (2.15) + parsec3/raytrace 118.645 119.734 (0.92) 119.521 (0.74) 117.715 (-0.78) 118.844 (0.17) 134.045 (12.98) + parsec3/streamcluster 332.843 336.997 (1.25) 337.049 (1.26) 279.716 (-15.96) 290.985 (-12.58) 346.646 (4.15) + parsec3/swaptions 155.437 157.174 (1.12) 156.159 (0.46) 155.017 (-0.27) 154.955 (-0.31) 156.555 (0.72) + parsec3/vips 59.215 59.426 (0.36) 59.156 (-0.10) 59.243 (0.05) 58.858 (-0.60) 60.184 (1.64) + parsec3/x264 67.445 71.400 (5.86) 71.122 (5.45) 64.078 (-4.99) 66.027 (-2.10) 71.489 (6.00) + splash2x/barnes 81.826 81.800 (-0.03) 82.648 (1.00) 74.343 (-9.15) 79.063 (-3.38) 103.785 (26.84) + splash2x/fft 33.850 34.148 (0.88) 33.912 (0.18) 23.493 (-30.60) 32.684 (-3.44) 48.303 (42.70) + splash2x/lu_cb 86.404 86.333 (-0.08) 86.988 (0.68) 85.720 (-0.79) 85.944 (-0.53) 89.338 (3.40) + splash2x/lu_ncb 94.908 98.021 (3.28) 96.041 (1.19) 90.304 (-4.85) 93.279 (-1.72) 97.270 (2.49) + splash2x/ocean_cp 47.122 47.391 (0.57) 47.902 (1.65) 43.227 (-8.26) 44.609 (-5.33) 51.410 (9.10) + splash2x/ocean_ncp 93.147 92.911 (-0.25) 93.886 (0.79) 51.451 (-44.76) 71.107 (-23.66) 112.554 (20.83) + splash2x/radiosity 92.150 92.604 (0.49) 93.339 (1.29) 90.802 (-1.46) 91.824 (-0.35) 104.439 (13.34) + splash2x/radix 31.961 32.113 (0.48) 32.066 (0.33) 25.184 (-21.20) 30.412 (-4.84) 49.989 (56.41) + splash2x/raytrace 84.781 85.278 (0.59) 84.763 (-0.02) 83.192 (-1.87) 83.970 (-0.96) 85.382 (0.71) + splash2x/volrend 87.401 87.978 (0.66) 87.977 (0.66) 86.636 (-0.88) 87.169 (-0.26) 88.043 (0.73) + splash2x/water_nsquared 239.140 239.570 (0.18) 240.901 (0.74) 221.323 (-7.45) 224.670 (-6.05) 244.492 (2.24) + splash2x/water_spatial 89.538 89.978 (0.49) 90.171 (0.71) 89.729 (0.21) 89.238 (-0.34) 99.331 (10.94) + total 3051.620 3080.230 (0.94) 3085.130 (1.10) 2862.320 (-6.20) 2936.830 (-3.76) 3249.240 (6.48) + + + memused.avg orig rec (overhead) prec (overhead) thp (overhead) ethp (overhead) prcl (overhead) + parsec3/blackscholes 1676679.200 1683789.200 (0.42) 1680281.200 (0.21) 1613817.400 (-3.75) 1835229.200 (9.46) 1407952.800 (-16.03) + parsec3/bodytrack 1295736.000 1308412.600 (0.98) 1311988.000 (1.25) 1243417.400 (-4.04) 1435410.600 (10.78) 1255566.400 (-3.10) + parsec3/canneal 1004062.000 1008823.800 (0.47) 1000100.200 (-0.39) 983976.000 (-2.00) 1051719.600 (4.75) 993055.800 (-1.10) + parsec3/dedup 2389765.800 2393381.000 (0.15) 2366668.200 (-0.97) 2412948.600 (0.97) 2435885.600 (1.93) 2380172.800 (-0.40) + parsec3/facesim 488927.200 498228.000 (1.90) 496683.800 (1.59) 476327.800 (-2.58) 552890.000 (13.08) 449143.600 (-8.14) + parsec3/ferret 280324.600 282032.400 (0.61) 282284.400 (0.70) 258211.000 (-7.89) 331493.800 (18.25) 265850.400 (-5.16) + parsec3/fluidanimate 560636.200 569038.200 (1.50) 565067.400 (0.79) 556923.600 (-0.66) 588021.200 (4.88) 512901.600 (-8.51) + parsec3/freqmine 883286.000 904960.200 (2.45) 886105.200 (0.32) 849347.400 (-3.84) 998358.000 (13.03) 622542.800 (-29.52) + parsec3/raytrace 1639370.200 1642318.200 (0.18) 1626673.200 (-0.77) 1591284.200 (-2.93) 1755088.400 (7.06) 1410261.600 (-13.98) + parsec3/streamcluster 116955.600 127251.400 (8.80) 121441.000 (3.84) 113853.800 (-2.65) 139659.400 (19.41) 120335.200 (2.89) + parsec3/swaptions 8342.400 18555.600 (122.43) 16581.200 (98.76) 6745.800 (-19.14) 27487.200 (229.49) 14275.600 (71.12) + parsec3/vips 2776417.600 2784989.400 (0.31) 2820564.600 (1.59) 2694060.800 (-2.97) 2968650.000 (6.92) 2713590.000 (-2.26) + parsec3/x264 2912885.000 2936474.600 (0.81) 2936775.800 (0.82) 2799599.200 (-3.89) 3168695.000 (8.78) 2829085.800 (-2.88) + splash2x/barnes 1206459.600 1204145.600 (-0.19) 1177390.000 (-2.41) 1210556.800 (0.34) 1214978.800 (0.71) 907737.000 (-24.76) + splash2x/fft 9384156.400 9258749.600 (-1.34) 8560377.800 (-8.78) 9337563.000 (-0.50) 9228873.600 (-1.65) 9823394.400 (4.68) + splash2x/lu_cb 510210.800 514052.800 (0.75) 502735.200 (-1.47) 514459.800 (0.83) 523884.200 (2.68) 367563.200 (-27.96) + splash2x/lu_ncb 510091.200 516046.800 (1.17) 505327.600 (-0.93) 512568.200 (0.49) 524178.400 (2.76) 427981.800 (-16.10) + splash2x/ocean_cp 3342260.200 3294531.200 (-1.43) 3171236.000 (-5.12) 3379693.600 (1.12) 3314896.600 (-0.82) 3252406.000 (-2.69) + splash2x/ocean_ncp 3900447.200 3881682.600 (-0.48) 3816493.200 (-2.15) 7065506.200 (81.15) 4449224.400 (14.07) 3829931.200 (-1.81) + splash2x/radiosity 1466372.000 1463840.200 (-0.17) 1438554.000 (-1.90) 1475151.600 (0.60) 1474828.800 (0.58) 496636.000 (-66.13) + splash2x/radix 1760056.600 1691719.000 (-3.88) 1613057.400 (-8.35) 1384416.400 (-21.34) 1632274.400 (-7.26) 2141640.200 (21.68) + splash2x/raytrace 38794.000 48187.400 (24.21) 46728.400 (20.45) 41323.400 (6.52) 61499.800 (58.53) 68455.200 (76.46) + splash2x/volrend 138107.400 148197.000 (7.31) 146223.400 (5.88) 128076.400 (-7.26) 164593.800 (19.18) 140885.200 (2.01) + splash2x/water_nsquared 39072.000 49889.200 (27.69) 47548.400 (21.69) 37546.400 (-3.90) 57195.400 (46.38) 42994.200 (10.04) + splash2x/water_spatial 662099.800 665964.800 (0.58) 651017.000 (-1.67) 659808.400 (-0.35) 674475.600 (1.87) 519677.600 (-21.51) + total 38991500.000 38895300.000 (-0.25) 37787817.000 (-3.09) 41347200.000 (6.04) 40609600.000 (4.15) 36994100.000 (-5.12) + + +DAMON Overheads +--------------- + +In total, DAMON virtual memory access recording feature ('rec') incurs 0.94% +runtime overhead and -0.25% memory space overhead. Even though the size of the +monitoring target region becomes much larger with the physical memory access +recording ('prec'), it still shows only modest amount of overhead (1.10% for +runtime and -3.09% for memory footprint). + +For a convenience test run of 'rec' and 'prec', I use a Python wrapper. The +wrapper constantly consumes about 10-15MB of memory. This becomes a high +memory overhead if the target workload has a small memory footprint. +Nonetheless, the overheads are not from DAMON, but from the wrapper, and thus +should be ignored. This fake memory overhead continues in 'ethp' and 'prcl', +as those configurations are also using the Python wrapper. + + +Efficient THP +------------- + +THP 'always' enabled policy achieves 6.20% speedup but incurs 6.04% memory +overhead. It achieves 44.76% speedup in the best case, but 81.15% memory +overhead in the worst case. Interestingly, both the best and worst-case are +with 'splash2x/ocean_ncp'). + +The 2-lines implementation of data access monitoring based THP version ('ethp') +shows 3.76% speedup and 4.15% memory overhead. In other words, 'ethp' removes +31.29% of THP memory waste while preserving 60.64% of THP speedup in total. In +the case of the 'splash2x/ocean_ncp', 'ethp' removes 82.66% of THP memory waste +while preserving 52.85% of THP speedup. + + +Proactive Reclamation +--------------------- + +As similar to the original work, I use 4G 'zram' swap device for this +configuration. + +In total, our 1 line implementation of Proactive Reclamation, 'prcl', incurred +6.48% runtime overhead in total while achieving 5.12% system memory usage +reduction. + +Nonetheless, as the memory usage is calculated with 'MemFree' in +'/proc/meminfo', it contains the SwapCached pages. As the swapcached pages can +be easily evicted, I also measured the residential set size of the workloads:: + + rss.avg orig rec (overhead) prec (overhead) thp (overhead) ethp (overhead) prcl (overhead) + parsec3/blackscholes 590412.200 589991.400 (-0.07) 591716.400 (0.22) 591131.000 (0.12) 591055.200 (0.11) 274623.600 (-53.49) + parsec3/bodytrack 32202.200 32297.400 (0.30) 32301.400 (0.31) 32328.000 (0.39) 32169.800 (-0.10) 25311.200 (-21.40) + parsec3/canneal 840063.600 839145.200 (-0.11) 839506.200 (-0.07) 835102.600 (-0.59) 839766.000 (-0.04) 833091.800 (-0.83) + parsec3/dedup 1185493.200 1202688.800 (1.45) 1204597.000 (1.61) 1238071.400 (4.44) 1201689.400 (1.37) 920688.600 (-22.34) + parsec3/facesim 311570.400 311542.000 (-0.01) 311665.000 (0.03) 316106.400 (1.46) 312003.400 (0.14) 252646.000 (-18.91) + parsec3/ferret 99783.200 99330.000 (-0.45) 99735.000 (-0.05) 102000.600 (2.22) 99927.400 (0.14) 90967.400 (-8.83) + parsec3/fluidanimate 531780.800 531800.800 (0.00) 531754.600 (-0.00) 532009.600 (0.04) 531822.400 (0.01) 479116.000 (-9.90) + parsec3/freqmine 551787.600 551550.600 (-0.04) 551950.000 (0.03) 556030.000 (0.77) 553720.400 (0.35) 66480.000 (-87.95) + parsec3/raytrace 895247.000 895240.200 (-0.00) 895770.400 (0.06) 895880.200 (0.07) 893516.600 (-0.19) 327339.600 (-63.44) + parsec3/streamcluster 110862.200 110840.400 (-0.02) 110878.600 (0.01) 112067.200 (1.09) 112010.800 (1.04) 109763.600 (-0.99) + parsec3/swaptions 5630.000 5580.800 (-0.87) 5599.600 (-0.54) 5624.200 (-0.10) 5697.400 (1.20) 3792.400 (-32.64) + parsec3/vips 31677.200 31881.800 (0.65) 31785.800 (0.34) 32177.000 (1.58) 32456.800 (2.46) 29692.000 (-6.27) + parsec3/x264 81796.400 81918.600 (0.15) 81827.600 (0.04) 82734.800 (1.15) 82854.000 (1.29) 81478.200 (-0.39) + splash2x/barnes 1216014.600 1215462.000 (-0.05) 1218535.200 (0.21) 1227689.400 (0.96) 1219022.000 (0.25) 650771.000 (-46.48) + splash2x/fft 9622775.200 9511973.400 (-1.15) 9688178.600 (0.68) 9733868.400 (1.15) 9651488.000 (0.30) 7567077.400 (-21.36) + splash2x/lu_cb 511102.400 509911.600 (-0.23) 511123.800 (0.00) 514466.800 (0.66) 510462.800 (-0.13) 361014.000 (-29.37) + splash2x/lu_ncb 510569.800 510724.600 (0.03) 510888.800 (0.06) 513951.600 (0.66) 509474.400 (-0.21) 424030.400 (-16.95) + splash2x/ocean_cp 3413563.600 3413721.800 (0.00) 3398399.600 (-0.44) 3446878.000 (0.98) 3404799.200 (-0.26) 3244787.400 (-4.94) + splash2x/ocean_ncp 3927797.400 3936294.400 (0.22) 3917698.800 (-0.26) 7181781.200 (82.85) 4525783.600 (15.22) 3693747.800 (-5.96) + splash2x/radiosity 1477264.800 1477569.200 (0.02) 1476954.200 (-0.02) 1485724.800 (0.57) 1474684.800 (-0.17) 230128.000 (-84.42) + splash2x/radix 1773025.000 1754424.200 (-1.05) 1743194.400 (-1.68) 1445575.200 (-18.47) 1694855.200 (-4.41) 1769750.000 (-0.18) + splash2x/raytrace 23292.000 23284.000 (-0.03) 23292.800 (0.00) 28704.800 (23.24) 26489.600 (13.73) 15753.000 (-32.37) + splash2x/volrend 44095.800 44068.200 (-0.06) 44107.600 (0.03) 44114.600 (0.04) 44054.000 (-0.09) 31616.000 (-28.30) + splash2x/water_nsquared 29416.800 29403.200 (-0.05) 29406.400 (-0.04) 30103.200 (2.33) 29433.600 (0.06) 24927.400 (-15.26) + splash2x/water_spatial 657791.000 657840.400 (0.01) 657826.600 (0.01) 657595.800 (-0.03) 656617.800 (-0.18) 481334.800 (-26.83) + total 28475091.000 28368400.000 (-0.37) 28508700.000 (0.12) 31641800.000 (11.12) 29036000.000 (1.97) 21989800.000 (-22.78) + +In total, 22.78% of residential sets were reduced. + +With parsec3/freqmine, 'prcl' reduced 87.95% of residential sets and 29.52% of +system memory usage while incurring only 2.15% runtime overhead. diff --git a/Documentation/admin-guide/mm/damon/faq.rst b/Documentation/admin-guide/mm/damon/faq.rst new file mode 100644 index 000000000000..f55d1d719999 --- /dev/null +++ b/Documentation/admin-guide/mm/damon/faq.rst @@ -0,0 +1,59 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================== +Frequently Asked Questions +========================== + +Why a new module, instead of extending perf or other user space tools? +====================================================================== + +First, because it needs to be lightweight as much as possible so that it can be +used online, any unnecessary overhead such as kernel - user space context +switching cost should be avoided. Second, DAMON aims to be used by other +programs including the kernel. Therefore, having a dependency on specific +tools like perf is not desirable. These are the two biggest reasons why DAMON +is implemented in the kernel space. + + +Can 'idle pages tracking' or 'perf mem' substitute DAMON? +========================================================= + +Idle page tracking is a low level primitive for access check of the physical +address space. 'perf mem' is similar, though it can use sampling to minimize +the overhead. On the other hand, DAMON is a higher-level framework for the +monitoring of various address spaces. It is focused on memory management +optimization and provides sophisticated accuracy/overhead handling mechanisms. +Therefore, 'idle pages tracking' and 'perf mem' could provide a subset of +DAMON's output, but cannot substitute DAMON. Rather than that, thouse could be +configured as DAMON's low-level primitives for specific address spaces. + + +How can I optimize my system's memory management using DAMON? +============================================================= + +Because there are several ways for the DAMON-based optimizations, we wrote a +separate document, :doc:`guide`. Please refer to that. + + +Does DAMON support virtual memory only? +======================================= + +No. The core of the DAMON is address space independent. The address space +specific low level primitive parts including monitoring target regions +constructions and actual access checks can be implemented and configured on the +DAMON core by the users. In this way, DAMON users can monitor any address +space with any access check technique. + +Nonetheless, DAMON provides a vma tracking and PTE Accessed bit check based +implementation of the address space dependent functions for the virtual memory +by default, for a reference and convenient use. In near future, we will also +provide that for physical memory address space. + + +Can I simply monitor page granularity? +====================================== + +Yes. You can do so by setting the ``min_nr_regions`` attribute higher than the +working set size divided by the page size. Because the monitoring target +regions size is forced to be ``>=page size``, the region split will make no +effect. diff --git a/Documentation/admin-guide/mm/damon/freqmine_heatmap.png b/Documentation/admin-guide/mm/damon/freqmine_heatmap.png new file mode 100644 index 0000000000000000000000000000000000000000..4bdde033017856246a47c338084c31761296b27c GIT binary patch literal 9999 zcmZ`<2|QHqyFcfcVT`h5tt?aaU6Ld*wovx$yRw!ng-AH|il~sS7$ii9tdT)QSyQqv zDf^l&TjriAe!qMF_gr$j4v84j*v+6gTbV-y2AN+Qa6BT1MHCZz@w2z#UhRW{2zJnpgUp;0JKK5%>ZN* zNkH2t5>qTGP#c;KQkaOEuEsR8(F0^!;~v>uRf z0Z20fo|ymzW?-o$SZxh9*n%G}f$jESmm}Ec3=Urbe_Vwo-Jn?yXz?1f>w4tPgxK0J@BUuckn^1<+#!^x6P@mwY=@O5Cs8~El6 zjNSsi2LeBWfw3@P{5CKV2~5NQlMjKZcwjmSn0XA$5`ei(U_Kj|e-11Z0E@-I&oW@C z3RtcIR_cJ2cfjgLV66@K)d{SB1vdJCjbUK(JFqngY|jF}e*(W(ft^jHvH-?qi2)?X z)b8r0*8za`E%_IW7p6T7060KHMbW_L@!XR;F+bG3(~hs}+{b}OccdKc?G3%U<1LMU zdH(X&Sy*YACBNEQ3S1QXN{xbfCWGxX9@!pV>njo)UPSlbr!xt*qj^Z6d1#{%eo#?z z`ONqwmgBK@sN;cL9fsm1O-)Tc*0FUX44DjwCI9|N2zatT7x1#ADJp;afb(7Yj2T?R zyRY~Kp)>jlSjB%YzSEb<-mix6T_@5Y@yo4%H~pfcZ(YDlS78b{+`V8G=fk-sF$cPG zgA|RbwI|g?;u4D@<`K zT>)HX(HWnF4{=T7vjPLS1vx{3YUDL4zM0o~fyFxry0mT1{cT!&}fP%%?%l&`|HKL%mO2 zyzf6KZl8dHf2XVaep5{dseNE}%Zl)E=vf8LBVA(P{P~3Y-Ab?|>E@-{d0s0_q8HG7 z?}aD*QT@VkSwdXuIX#k&O&C0u`ZE4z)6+;x_phMHJH`lG9=WSdaYqx13t+_|s{Hmq z8Wr2iRGG_}a~E%j)2Yz-^mPcRie;t48>zjt&8zN0Vr>z&`ZJYA$9``=i0;}pcpRF& z|DOI2{X=dbRRck|mR}bnnb~xlV)q>Tg~DJSLtxN_95|3yVv?P zmWj=imdbMD3D;H& zX$A*?;BUvP;0L>)=Ht{iDN!q2LL*$*d>{2nxmbb@9#863A~GL;>l6Ho8P$)eNYv;s z#t}$JDDX(k?f4XRg5aH!csMc%^&16kwzEK;f3N76I~KP>{rH5Vj^KI@35_h!AT|l z!IN2zASnhw#j%FMc}Bfzpcm49V++jIX7ojE!d*46zT_GD#i9asOiz?dXJ*AUv!eCR zU^w<%th!eB12}$mo=C%H^zDgT^aZt zXxM&sMT*xnKzI9cs8KF=45qTmS#FHot5d(htfo5w}b+x#cW8^3YZ>3^y=IM9rS}o70*y4QP2l z`m#3q_G78-yM)w4o+e0pxrr}Vo9JAs5g|14prXj6GxU}L1^D6cYd)KtdY8`*8Fo(_ zs$-0?iM1m0v?n6b@3_w06+ff**&d{ebSU4y5CePGB&)0)8DZEE_zZfh{=Prr_t<%;x^yv>`aL- zrk(Q=9TOzt0rYZM?e&!OgKT1jlJ3dsBM{R@Ugm|7;6+5OzGf`u}%!6OpV zesuEPdBlySudMD6XncR+GKC>+@$J_(cSg@224nal&L$rMV~l9l;p*B_m|tmr%>ci4 z)NjKa8v27B%KN5pDo>q=7R0GdNE-!aUk4kr;O}G7R>t^}3QPN&PFdfe=&lMh*l2+_ znvn%#T#kGfv@XpPC1gY6uSfDJOz!g2HmLvDNQdNkAkJao5#^Mc?~Fnt?<|PUg1GiJ zK8Q2PtS4D>jyA9!Ehl;Ys;JGoqh-^V!R!lGQ|}=$(M02@vD=|L)NE8^is~0-;di2h zCHyPmLpWlh66WViDfzBXjD5mVWbr-_47@;m0gqdxaD;2jd28n-4#sC+I-$VK|B`1! zLu_Qd6YJAyKs>(`pbWMqJv<9JG=IUEcoU@=+%-`mc~qcZ1Xkw6RST?7a?Obo5NH3T zfh|;Zn_ocLtH^DDhqYscmJZ2L>^3FD$(hica?3i$9?V|ulcPsJ=4^@SnBcO)%KS7T z#-3qXoE(|_mWc8Bngu)DHa{RCZo;xyC3W$p2*&5KP&xhLqRpFU6Ob6kjY5{iKJ1hu_5Il*Ldkm3K^;h3t~#=?Ow#^ZxlyITx2Vo9&QuVM?ZW? zeV@e13UNB5J>0s4k`qMBwKQ{bF?T0x$MmRXTM#oZR@4xf7CWP5wPF?+?%y`TIt8W~ zunw4mTeSs$&Es7`LMUldrYF6h&UF^?mhdn zBJT1vbF7Rgy2a2>=FCy~s05Uqd0wW6yj&cqJ;kbobcqIRu2C&>6jZ2X>R<`1GMT6_ zv8^(O`kuryxEG6z7><7)`_N!P%r?UJw?v3v6&}(9wKzGSq@~*RF@Wvj+U;B%mRuG^ z4Z$fBYNk7FQE($8_ee-~7))3ALjIB>(Q(3p*l>Yw(jPd{jFNjo z>>eFasW6MkWR`tU@Opo9x} zHBjWr!`P6JI}D4P88X-a55Hg2T{9=Sh62_`r}7G03asT65^Dk`ex6haT(w)<%*<=- z?p_Gk-hNkNOh;S0v(Y-}S2$FCb8Wk;xv<86d9+uJOCeyZtz^P_WAZoMu;2Ba>9L3# zIs-0K|F8MA&CTy_HNFcQvrEh33Vzr9etnzZ8jySQI;nMNV>5f9c5CH{c5HECI?4QBlFbK z;2SP*Nw+8=@mr<1;KAikQj$2-)HS-cPi67!r#MfjPDiVTQpD@n zoI3g~8tWAFR}x96NW{kK>_!}f!F&xz*0-YA>3^JVN2BHuw~wGuHAF-o=_mjy&LA$* zL%~*vfPd&B{|}vKm+qc`mcswhxyz!2!Q3R9MbZPH;IIFx_=m3e|In@eAG-D~{aXP| zg;X7#J4G`C^$kn>2sJsJD(#AC1nPmc9Hlm&)9Y)W8% z!!$H^R7=em%hoe2^in)ywifdDDXrTzM!FwrLl@oAnR7WZsHd&QdNz_y_)K%msim~O zHWt0ysF`p|$v;&>)Jt8r6Rw-LmFKWb_K&9JUfbAc<|)jW+J}02k#g6Mb#c@@qr{{) zO+{j~n4Yu2%Fjs)uVrJ|dYKVja&1sT)2>^vr)-szJb$bJA#_*z0G^oh`A$`jr)S+Q z*8?7`BHyaggZ4GkX>=>FnuyvA%+V%lt0f?35V}}eC6&je5+>Bmdc2mJZ>HShRx1zNkBQbXjS?0ZbT-Phyt3PoLL%dFDyL$ceDGg z5aTIoqra7b-b*(Pc>3LLMyd|z#^dQR_w~2_Gpxp@lVyC z0!~}>$N>P0EaRZgxl1$MPZA)j73xikAZa3e%GX6}DudmuwPRzazOn38Su{%1 z&~D!4d|&OGzP<8h43ekJh`l93-ypH4#nWp4K9vP>dw2@t%HThq)$(^Vipi99-R`e_ z{b83^_JD4lgr3Gz&l%p=u1>gyYbr`=Q1XQ%c?%1JOnWsJgT8s__SYwI)4OJIP_r2Y zlt=8=k|z)0`L^pzyAfXfHLdj+)Fz|EI~I3q`eHX_en{+jm~Emz86&h&ng`-dWPQF6 zcKNm=Ab$&nLDLU=jz540|CS*+x@=x+kFEC%phG=ETWGSVPhbav1y1iwj+<`nG8H|5 z?TU$8oSz1mPS4n~rYP>R^tjNdr!rg(C|xEvm6%=)JU+Q=JLwG(3obl)c8Q~9`Mr59 zZm%s11V?HZnNRTFHb|V5y1X&uxhtrm0RE-b^_9M4t2HLntGRJtSMlNk^$L!VtdMmh zRCL6O469yf7iZRiK~L+yD-1i|{m2uu=-5%bt?fmVCGxHKf#Ml*V_&D~>63^`%1w_A z!tXhF9uh{mT*1ledEBS;pgP`O`8OB!4$!!L;?A)Db+q);o;7p@&dd65W7za3|ETTb z3^}NXw`M4IYp50OXzY^M5R{QS;^l@cN#_Pj&rm;pU-fS((tye#FAQ(XkGvsUYt1xi zMhxw^`E4)yVepgD-5BlS>rODf8|wUUuuPV=>=%`Rn6uYzFN&d}44Ir5pK847dNJkk zG@S7rj?_aT3N4z#?6vyq$?mVS@_0YT4-HDUn{TYU38x=d2j9I2&HxxZibtLI-m~{qxV2?uwY__~g_^8P8 z-*7WQ7+h_NLVr~_$R`~zkT{HSdGGkH~mX{0pJD=kKN_J6?dxJ;l1&!K$NM- zvm0ZBe&(tD4wPO7#R_yt&m@C#n%z}@rk8B=rq8(mrbH{w`MdpxksWVRotPi%xarZQ z_v0VziAITZ%&|`GrakZ+F~Wm*?_Z>;zQOK(>0kZEQ53Fs_L^4%EdLJoKiXIQ^x<#c zNp*Y_#r$D1+VJ4sgTMBZH{DR+{ogSI!S>wBm5LB5*_pLS&wqAH&G?$%`L8RP&ED?e zSOs!8+(0+krf^7?XtiR$in1i_Aw36m;`1h^yAu@i`I{ljb%V14#WhNzxP9)sJm zC;s5yE)<&D)T{5T4_|Vy7)<6f3%heJgXO%4M{g5>lk83oevqZ8l{O{XK=^uq=pIlV zre^!$Ph@G-Y$-4(oEQ1|2aa>JNCIM6aOHt z0#8SSm`rJQ(20LbU{n%yPxKE0AUH2{HPG3Dq2^EU+-O!j_;|R2wB7ma$QXkF2n92* z;sG!Byq9|o%46fr^w98m)Ai^&uDA{E-55A|nqv&6{M zC*A)Yg0w_5>TS|}fAW`NP0|Q?xc2TpIw<7|UZITVL)I76?X@i)=_aEibuk~N_WEbOT z5;);K_!SQ$Ld8%m-Zi!Z?+ubXo~n7+3ot;N)MUtdhJfGmQN0B7Ky9|fSivg&^&T&C z3J@s;X(ow#20eWtp|d;2-PAdAl={mt?M8w_FJBKH)ox)ax+Z(@uZ$FGwj3N|W{ooC zb(DTLRaCua+|oo@mYzhQ(oMFhmb%d}!L!Rm zjtF|+?7~GZ&R*)rQs(r3Wge!k`2F*_%AxKUip!aaPdGAkU#^HKj5h^dKJ9CwYKWQS z|6AfrNdcx*A>N9EtFS{_T4h3%=P}ibf;Ok@HFl2nLzKw3VXw zUxzQ3#Gfg09D4sFK3alSh6zbVk1jrTOgekIDk3K_>GGIQ>&jU|MB96o;}hvLccw1nd`z@v2JmNo?b4n+cw z%+=J1YgIH>E!2;^fb&1!+OkpanNQ$gr<#^ER5(OS^;c9VOkhPkSg!<7s2AkvX7|KGPrpK* z$A+WcV^yGHvhNJy3shC69-ZbyT_0{O}YQK zCIto<{!`=sq2e%z!Tc%iU5oY_{GT%y75O-;jQFt2xOipHsQ)+y;s5V5)jf2S;lPVi zqUksSdC*?;2Q`SeP}qj%Z(=zp>FHfSBov3SY--#XOc}W+3ek_yQjB4Ed=Z*lO?-r; z^!-q3MJ$U-6JMrLS*f~G0z`uFkJ5uJDIq|l5vVZr7pnev& zw3jnjmxjw=8chYW1o9whs;|6W{feJ|#u0s?ACdFV;E)|9Wk!iiA)K#pQLW)cYn7K) zW=4wmcD4dIz4!IAwaYz~jvL3+b}sbOFfG@%>(O@{4;QvczB=j2_Nn}uUR>cN7EDK^iAoN&OubMqMSc_+mKdu8`YH z%c4-kqdK&mcyWRbx|%@(X`u+VPQ}}|r*-j;oU`{~X*He|Gh#kP#U$-`ZLdTWdHpd9 zknuAZrL<{ek2Av~;!H$GPD9b?mOPGLJ+|XLtl~qGh2;CI#wF6ImAiG+6&y&mC{Zqz zf*eH+kUT>V9oU#6okf8gao5#C58!g?-5GHs&%JvIe-;c;&sM-xv)xNv^Ohxnc zI6@DOzDV4N#zzf6NK)|@*b0&8=tKMB{ytrTB>fJp28KroLlReq`dlh96?EmQz zTt_T}D`9Clrb8%bP4zr9oxTd*fiI*h;%uy6@z_Y+_I-#W1m?3WK7e^_RZV@6Cu#*aGp?GSkAQ!s36)rsSaynr}nM(?hk zQwh^OLt;d{42q+@Rfv&{hz9_qghw8IVJ73cAS(=>7_T)2U}V^tP%HZ|iPSQdm;d@L z$H-O%M_i$ivAXG~dO;R|Xd`A{b8@wO`DO-bCk&Z>ilc?$2E{Rm6Pe|aOyv??F+Pu} zWm56@iP=opID%guTk&%f38&rg>iY6RzyvoJdccQoj)P?}oj8KtHXhz;Q@epYEiehe z6W_q_ezY76KPd`?0=BxbApaF=A5w_nkr6&56e>23sK-ipIv7?VQk1@cKX0*^fhZ9& zE+&c-a5l!XY4C6mERRIvQqVV=_4D^o?Ct#+$h&C=yFTJn3e#}mY$C{uHoL6X+a{9W z?8YY;nIRcw`l2QV>KK%_B2m1G(f!iBHDzK})qD+1hqFl&@z=m$z%4s1FdL#TqQvw7 z7!$0|^_zUiJ>&i?Dsbc2ak35dgXCu&e)d`*9ok!!ctv=`GZBi;e7p8tVo7FEJ$q*+-geK>^u4>hl8(&ol;*LT?kH63$&%x$8 zB4OL7C7$BE{i4^h;+Pk=E~Dfgm-l=EovSRFi|9PI=p}UdT2s#mR6W!Gh*@7A&S!9c zejDbmX1JkCY>f~}DanE_1@VI4-G6>Qd|4<9H{!rmA?S}`J3H<;+FhrimBoY_n58(G z#R`6x>#nkoD9d=JkA+;MgJ;6P=YMJQmQvGykmRIm}0(isUi^r0z+E5$9lEa;|i zs9Buyp>XhZ6HJ*=Czf$(?0c>ZoiKX$+4xoZgv?BhoQpMA1z$>h?YrUp+B|`Sw^S=O z)J*MDS(ZpZxn8!``ToJGSsvfdp>?HX@XF1vv9~;{ z__SVNut-eV{zVCQu9+DEksGTgs`Hib%aZ%1jhv0J8RZ{WJRcgJzaz&AaS<@^5^*`23^1PI75%n=z-T_p$tZL$x|Gk0~#~fY(80O*tR#R7G{s8-IuH4u9WL12%HYpD9>pIdPjzWE+)jDF#2T4lHzn?IZRr}ek$g? z;lo%fWrO_E-jjEk6kG=j#Kshxj9#4Pk*>Ufv*yEi>ir#$XAX7DHhM%QjX4K#A#W7*0NHq{OFMN1*4A5n^}(xW1ev_ zNU-}*dv*Ha?Ru>lNpup1D&*peph+14zur4j87dorjmhbqhD>=bCLg!iWOMQk^m@Dy6G?RRjIBc1p&YiV z61z(0^j&+&Y|SgJy|~kD_xm1oJ9J3Zib|Me=U#(BALd_?c(XCEus)7+&gM&X|v!Qa6PeLK)h1m;*|VRAo+mwP(YUp z(E4IYvUA@LsRH*4m=9q2-V%@L8%<$F;Y{@35j9J$TQkPo7Rv!m@r_`n}Unm7wix zk9KTTMcAzrHOi1P5m+bd!{xKrKP}DmRfn+VU8}3Fkl5$BlZ|x)qgA2U})Z5$_s}Dc-)r$}Ra- zpFI@5m&@A0j*jt^OvqykKbFp*tCs6axKwp9^bqWq%PJ9Ab}pltcg`Wk1^k@DM)eji nHA(UKE`@8|JIuc!w4vV)+)TUxL=J2>lBpW1+A0M~Ho^Y|9YC~` literal 0 HcmV?d00001 diff --git a/Documentation/admin-guide/mm/damon/freqmine_wss_sz.png b/Documentation/admin-guide/mm/damon/freqmine_wss_sz.png new file mode 100644 index 0000000000000000000000000000000000000000..83cb1bd67734b340be690dd3a5a058069d0286cd GIT binary patch literal 5589 zcmb_gc|4SF*S}|m>|0EhY*AT@s4S@&TVyW`Su*(*SqfQ3l(|#LQlteLON%9pB{8zj zRKm|vk|i1j$v&p+S?0Nip7-;-@AJOTU$4)Y`#$GfXSvRGUFUqSiM6pZ;p3Lz1^|HX zl&R4f0Dyx40HbjtAP!tWwFe?JY%J`InM@|c$f@K zMa`xr*`V!x)Ibpc6cGW@&<6yuWDv^)S@ENp+0X~e%=TxJ$xLWIOmaIZlfh(USGH4_ zly-9IbtXESDO+D(KQuJt=H@0NBZESrva+%o8X5=^m?CJ-T>hqK0|9{lJ?kI#NQHkl z0EnDAWu$KxN|>ecguGA|i8?PHGvA^u7_@0S0hh^nya?!9jWUVB53J6d{{6~1>Wozv zWN7Qp21XUX81tD(z8s9v%21+me|tPBW1^p9u7zNdp4v_e@+}3Nn;oiqpA|grKuSaf z9}P#GuAnTzRXF{(xUSa1cj zy@!}E)V64PJGpN(=`V@UOJgU=-ni-=21G>uT~ra@!SLBpm6~fUD`R4xn;a5TmLy>T zJwFcXisgKG-Y*{=QvW_3(MNPEIWP~JVo{u(U9#T~eGscu`?^EW)Llx^CsQW;B2MjV zW@MtGoth6Ye#Z-^b16mWL#OO_ZLq}|_mh8x$ED%cE&1$U_%E&2{CVe#y{L`HXlni& z{JBA^N+}VwJABgv6n)!&?5l~LFn8M4sJr#E_R6?U!z!_gA~XJasz=Q$C@<5m(Y3R> z6-RcHi`gW&_gQaSyt5zwc25O)5sdzvjQj$xS;= z)Yc&yO1B`Y3X#3Mn6xFTd3{fuJXm2aMRp}t6mulb5phdihl2Ru`R-+9I<8z=Yg#0%80t7=WK_TB$9=>Jw{N%pcQ zxkqX3RRTp+YWJLiy>J}n%MqZ}J3e*JuMJ<`8HXHn{RPnDkw-24LLlj$T{>2j0l>R! zE)mjruZjm`6zpZtamW$$5>Rch=LCy;a|RYPM*Cv7^PYQDU#k=v^)|H_ri8g8nugHc zWswCpEi;&@a*8^O(9Ul`ml;U5pt!I6ke6-b{;B*I5Hl7!j(|o;n#z!Gl)d%fh4=w9 z>oCz)C?Wq#4*qR=kqGVAJz@j=LvCq1U`5VYApH5m8#aG047p{%A=(ijdM&QxJw38C zYHqs+4Cdj%t}Vs;a$>1sx3lvZz|@BfQ2e%REDc!72!A-S3i@4PE4@O21*wSk6;GCk zM+%br**R=1<2hZCyl_Vn4Uiy3B!xHY+1^@$j6@>(z7Dnn{DjE+2L3?BVtguHwCwL_ zpyxn{w2i`*CTGZ)h0$}!5r21m ztOi@D_x)RfxRLeiKP;jF%mHPv&OW#MR^9inL2$X-^~f-2M|_gA5GU$g@1uue#H+>}`6i4!g-0 zo)d~jns}dbixaFda^TQ3dxMihPFex7L&sT^`>Esb?0buM*9%3jW0eLNMVJ-UF7=2b z5&A5B!kwO*C0SjS1(+%}e^*uLyyJv*_(oA2>m65Jmy0l=Vq|%?QjbH=X_E`P?MD1N zddr=Lhm}@B@ggcky>6k)1?R+37|xZ99BBq5ftp$w zLbT4H%oH+^dufz*bU!%l9n~b<*|cU@F#?~Ev0-kIe*AsA&^zj*dz4!tawqvo;|@l^ zj|)+D1krh`=7q_Ei@&(A?N#y;#!--9B%?QaSR|qA`sQ1V>1D?~$QCw^QUL02N>DE` zX1CG05?k5Y<0%g!VJ3FAA9tahhbEjc;H6Pyyk;5{$Xk;NlPh&R!YjWS8UG0X_CzZ&!wcPVIb}oP>#!S}$m_{PWgh$Eu1j%8>?CPF5F9NO;NkgrrGR=Zo z0{Mn2`=Xwn`Fw-Gg{|~72)Ss$iLJ|BzgY6{`^f_#y9>1{Mx=k=DCNRJi4}8E^0KK1 zdUbF5`7|CcM5}WCldDR>&TQ6j?E)}a~|%*jfRD4Ee1yJJ3dHK=dR<)2I)ZhlunGoQi2Va}%*$xO&`t#+ql zn#Z~+QCsf~5RLO=j2H#JKMd6wAsid;Il?7fctJE&V(t*`K$`h7B`|02#`7u>(xqHa zxCujX&oPomt=JhFubg9qIc3eB%q!HgK+Mno)s<-K6H?ja$fZRIEr-0|f;{96`S z5-lo`0@UIj!|(9ZUN;A(sE9#; zAZ7ws+Pt=K@cYXJN&)rwVTe_?;)5oA4z$_)-K1|xf|`AQtiGwIuj%H3&nhw6J&J_$ z5RnX8cSb&a3W0?H*kO6(Rjq`I*-=rKDMid3V;5lHksE&ee!c$k=3etMgSKbZ6pDcS z-PWr1#biYcr&@%xdXRO!X|KqkIHddQFPvQTL7K5nwI=`ph?XfJbuBFkLQ%Pnd-nf? z@#6Qte?qZ7Xvs9?jq|| z7p6ZHOs2?!_HhE)71aUu5{@#03hz1m?Q@5BvXz>#3GO4#i-vJa&z-!a!!AD<3-JHF zqPxf8z~_^IPmmn=g^O~{6W(W|z~-63OM7&I#2?V%g4ln#8OK(7QUShFwdqsBy9ZjJ zSYWDN(;+$u=$V+2eL+&Si52c(_rF9uV`0Ry`kUbziZq2xdJBG@dT6;?-JpVt_VN8C zTCJDvY;_lETiU{8sH+fyHNd~%wnu(IyXUDUY++c_wM4jb9NHZa74#Q2qx68=GH$ru zKgu%4B4#-GW^ij#8-iYsQ&S@6fN`EB8N!~BimlKWp8zr+;|4ViD= z4n9;dG6nSpi0*3@$;Ro6;=F6BPE?H0o7v}RK&Zaxvd3<;cO&(4R=is*3;NxI#@5K= z7gU7EO;v9>lAHpxLawilkI~&7@+d5l6hf%qj~A`+cJVG~Hii(3GIARPzC_~OVnzkj z{6rHM3-W+7D)ijJTs8bMzqbZdCF%z=BQjK~L-pX0p!fnCWNCTf{EiPb$}EQ^)*R?9 zxVzHQq*=Pv4cBi&n>=w_a zuPoHO!a5-(e};Xmr*e*N-!B{_VW&SYoB#0@4cVIe`C!X~8dOex|2e-5ahQ=liWe=p zZHpDi(68-EEe%*P_A zi4q7vSif-sgyDaMZyuZ+8Ej-V;h$?6>K*K^MNK{sG#@-k7GH@wSZRCL03xD_fROpL zN07qL`olju_E85qR-Zc#Te3-RK3RADcJP5JbjqW$1n>{qT)#o^%yIbmZ5O6aT1{WV za#au{<2eu0I~6irDteTCVmo7npAgww>ro3GIHe$7)jC$eRWle>*m?;c{*z=PW8P1G zi{(r~2%3bjAblmL?BB7ex0Wuj>e2b(NYP+GWGmT?S@AM3Bc;s=6X3Mb%vjcTLao;S z!yA2Rv%_`s+SXH>6QJd|~VOO{(Fd}(pZ8MWW-+bmvf`&|NZ zbpm&)qv7rX1hcAZ%nogklBm$O!l~g41;#lVc(&G=RMvhj`D;oP>!MqIY-IqST5bZ| z?@TV64N#kqSkRURc?~W|HJLHh8+QfFv>#6Rrfw=A z3zxDsYUM-3=C_dw0{;Aw!E64tFYYGpHvC!z>xDhz} zqWrD^Hg^`~=h9k!%M&Q~#dFgF2i~md{+&Ls@F2jm%;lK)7j>3nVz5KmjY_FYqt+C| z+-*xZH|qQ*oP8ous~ltKke0Q+YKp&z~@D4 z%Nbq@n^$%H0M}7sO9euUvok{Q>NwzWTopHZ@%rP81jE}!dDtUYFrNK4kIXmi&WbEQ zms;7SuF+~0y?QNXPp{iEXLzb7N1&|6WhPVX_IJrAqe- z1h+&}1$3FUCgai>>V@*OUpr+@eRN65C+9NHE+?k*jV|W~e~PA$ag1hF=F!H@LD|2& zzXkSoAxI&cnHgD)gG}-POc@VtejhsN&?)-l^v|2$d)kJarELb8*o2W^t@L->qFWLR z<)$wMk~0xi%$v?we^GKbWNb}7o0<9XE1h#?o#F#D5?yXGKTJvZzlrooVb8DK5a5S5 zcgx7F-kjCoz7jOhp)-&W*^8Q3|J@ehC8A&aG@jFa-3N7*2`kir;M$PiVd)l{{XB3Z3+MY literal 0 HcmV?d00001 diff --git a/Documentation/admin-guide/mm/damon/freqmine_wss_time.png b/Documentation/admin-guide/mm/damon/freqmine_wss_time.png new file mode 100644 index 0000000000000000000000000000000000000000..a61ded5c5cb9d018109fee3971ef3620b6126eef GIT binary patch literal 6550 zcmdUTc|25a`~RF{hMBPqV+%2|RkAPHVhEA7KDH1eg``kI6vrS-l(J`=WJ`#T7Q3+~ zyA~~$21T+oA1b@~p3(R9d!Fa@d;a|W^PAV1Irn|t*L~mDeO>SCJ=Zz$HdeNo!YIG8T03iedprKh{3CMkD1n#KVSlAjf7z|iKp-`ANDgps9s0apPX^Bcibax{R z&QCI^1O!B+LJTDmbPItXLI?)Lh+w=eI(QJCN~8_J^Mr+kRaI3jEiFAgJtHC_va+&j zYHFxdYF}U9*RNmUE$AvL$xF!XCG?9TA7-sOLFd}T|5X3M>Y(SUi(nIMpiA^sq)ZS$O^=pg* z5g0{?CGt`#;>|c^JlTe5cR>;2LqL3d5D~s0h(v`*42WqzkwJkk5`*H;pi&v|c^K6G z7kP9Bo$|cDmC@Qyt-8)2QW%8h=H}0zKfAfPiHnQt>+9#|=eM-9QwA0ter%x|w3|A~Q z|5^)GH(v(Hm8ZU{IDwVEGwt2!*_P2?6xmk$r9H^ph%Ihs>fJ}5+bZ5aw-nfpP4ay3 zwsIMOA|?fG%xf=QoqQfYciDF@_kb;TrdGc?;=u>-@`K0E?CK6TPrT}tnW?z@^Via| zIkGTrE5ls19@}M6@%y#SQb|YyRMT`MLx>HD@z;^KJ|EqS~schZ( zt)3ZTmrwO+(KItFtN1c3b0WVT`lG(YVbP$ls@$@Da;$7#Yrr!sC30|~kuT$6;754o z{=C+f(uP0+a8-P&-kgos<{yc?;`pZU_I|ksY!n{$s9#BVTLKHN9RJ7ROHW_oo`I>FH zCi=sXj!2(O$Mph{kEOesr6^mI#YvyG-DM>%kw4Z+9kd@a;<4bc1#gtsuJb7=*o*p5 zCm!(oAIYZGGUo()dvrEEb^b*P^kuH7(@{O;%2_UUGd_u*4G1e)`s+Hp4GzbL0?7qL zq75G!od5y`CkTZ7SR~Ro5~A$?e>wP_XGo+R2mZD?0)|HM8&W489V7Zx8NznGZNamycRRAopq&@pdL7^tdMCy@EccptzmsZ0RZij5RJ1K3e;@~{+C z>;TLB{e9Unv7{K<-YJ{Y$LkhFnL2DI0ZZynOCnMZ$@Qfb>+rmKpQ8l*=s3IBZ(-&* z_Y^5c`_?sLm7zb&)vC5ubQ~#$YCT@E2FK+;eNc26U=#YF^3%f+O%#py+?uvRAIGXrXIo98K@ky}6;^6aRTVcmeJ1Zg$;k<{BQgYObo z3SQST_rv?A-x3B~hLKT5(U|U{ypFfI9SIz8U{82m(G8He6bX$-MS9z$5&?$;u3-!B zqhG-ie29)fijQ`S@?aVXy~cv%GP>h=2-mJe;E7}z9$PR7nEAGg5!vI!d%{B^a$IoO zNO;}(BQr0CR5A1}7_UuuOJ@>*j@WO+9S!RK%F?@)3JcaWi0NBP3foU#@c>6W$Y;W^www zs~@P)Q`Wt!Qh;O;IV08F3h+zT3ZSYddQ_g)vi8GuS1#Y>&2o+6NbM58rt`@X%N zBiPF0GhsbnaHjJSDPV&ixsXhE46tUUJOOVvL=i!zH}a17{mcR0IiYb4mbA)cSlR@aq)56hR<_;WDxfbUb6it9eI!fuPsXDJKA zG}}hKVg`B`bAoerHeGcO3?DMeMG7zJb}rrP4IDD0UTR?|=LTQ0*R?!699-Hnnn(Hs zpiVF~plv}$MYSxj(BHhr!OA94aoB)m@x+>^X9Ik|1$U^{LGJ=(^_^}t=Y>)L>zf@9i1984oczpc>71=0VTJNnURmQ5r~EpPFP zAH?Cz53)M)4|%&4D?yt5KO(NHjkF7N!{EDnD~6al&wJXZjYSndU|hc`)LZZ_a@wN- z=NKbuW=LxuQEjuJS8aWV6oad>GevHRDX#6+IoK~y;lJ9)#djGGPuF<0(yb6bjSACj z)5sad4k~fZx{sb1j!2F(lDbpkXYP^h^<`*kZmbduUY)I8Rrb~lIpWvqkxz$NE zLvJC)q*P;N&}kUk)UI^6C<;n4l%jlRyQ2(#8pdw34DW0aNDpl=rQ$Xb&F{ssh7#;U zSmm~aE8tVJ=R!}?RX=k)f=}jM8SNW!*)oM27>)5IXv^ZNPQPYf^9fn4b00!R`4tNK zi^guL78+9R=gKg85s~%~Ow?-NvVrAr`o(Y!^alkgP@zT)b+(Qp8d@~GqBBzF*Xy=y ze4Gqb7zW8Z+C|jGx)VrY$>{xWPp=paqBbE|Z6EEB;8pzju3hJPSE9_fzg2F2V}-$d zdjCt{TA!6Nks5SO=rbS7FvR+w!It2$(TsQ$3H2|lRT}giHLK-gYc@PcOAy9gk2IuC z$0u7%mPwG>ZQ%Uu)8)B0|HuNiQO(l7^1dG%sKK5D)z!gJaVDD*+f2>$2(T;Brh5*& zVng~02m=QsrL%U_TNI*TNERIc7cY z4_8aTdvY4BBqvz|Eu9G>6Ac7hm3yfVRl@i3K7~Mhqh1{U>IA9|y{5ahw-gWt+GS;( zIxnnLeA*P1TWVrIbNn!8DIgMEDk!S0b@zKTy#miG2y^Ch%ty{g?kV`JZnc?UmvT*3 zAp1)^H2Efp&D!dzT-QU7&bbx+6)sQ%IlK)$NWouJCx{d<2~a~qIF2hf$6I+8^0>|< zBbma(h0C+qOBqvt#^+{kd&AQI`iF!a?njGUDVB|bPi2GYjut)@zI`_SEdt1n`PTX~ zi~BoyY~!SIqs62PDms$Wz?S9mc5H8?m)5mt*k`LQbS3Xi0$~lA1htK6sV7)W!rWSe zJA!ZfQk(d_>$O&|R2r`8RClr0U%@B8CfK2Y(V}~^WCjN5n?5%==`(j6jr2wD1Kxjh z!NwmriG+2%kuI6ZHT%jeCW%zf2Rt@k?q>(H{0aP^5YzQ&O)n7$#uq%8?40VaX#XA{%dzf6ub{A^( zO4K%eU+fsjG)e3ZlB4B`bjZoua5bmrD1r3lE)Z5)%S!&`OP-G6Z)Qj1FT|Qs>0X9~ zj=xAhBgp)D;>cj>r{GpLZm6b~zm1nIHO=>%ubyMHl>H#Ile*fW>H3qJ1-+}bd#IzGRDPp=k^ z6L5|;CCa;X)evDNBTz+SIqtO)3%s8x@g181C%QT63S6pc14)ssVV%IPTs}WjQ0zpZ zP$D!NhW>y(n~TeOCJ4eqWvqz@di9wH4&lng!|VMkVNzL9jkn-3*tyUE;BRIY8OFl= zLxOTk%gV)zhmqwu_mMdtuU_7!?hYN~_2+Q_jxuvo04`e<1WH&r68UHOajf=mIorI! z??nCP81Mts3#Xy@9ts0?Ijr)eKxXFddxtJ92&k6d=KvD`+txANHlag`CCeSreJnm~){X0CNV@1S{&A zV}KxmBxx&6(#2JskeOfgA`l3OKZSg1zJSt%rDHA@Erdc^ppc0mSUeSs3as4AhHH@x znGQy6HGXHWQsi$rMAJX0jLyWOzb!0t{Og{ugKGQ{Q+t6x0_p|mE3pk&p^<0XB@lfT z0lEfXLnAFZe_+?b!`EDHF|2A405^el^_A+5G@+~d;%?K1MaH9kYu(5T4At(&RUujq z(vZMB#Tj^a42xHyJ>>vproB+t-B5rzJp~lo9(oK7HbKyS*f$yq0u{zM zC4JnG6B1Zq`7a&)gOnd0N2$C|_tP#`wXs_x8_UBgV|2(uSfuXx6U3Oqj*}=SfKh~l zyLRxSMjJb>3RA|De1{yL<(3}9D%S{eB1IS|a*VZeX~c5bhe+5`A-oPc^R^H2tx3hD zUSWZ9dfU&rb_1$YsR@zOA)%8^u2E_IkVuZPF>=dKwqEW#um6cpOsUN?NXftCAW)~p zmI$u|fEosNV2`|syYHWUeR%yRoJUGYF_+XD&_KQq0KECfg1AA7iF$ouMN!*vGR7LT z@>3*`B)&faUV0=;!y**=1M(fZRy2sY!5>>=>Rz6+;%AbFfjYR8VhZFs3_i`=iq~TW zoruIuJNQ+M4SKi9kL=?@K6N8f7ht)u0Kf>teiXzFh2OO%3e^I@q99I52krtu%}NSr zm6w65;@HN9SZQJaIacr!Y)BA+vJHYWGdsH5NkORVDf9MJFDD{3kVu_pM`vc0fL3oh z%D5E2;^`Y@R)E~j@tUEloZe0KU0&I`&IZ(dTM1CRG`5?8lZk7Qhv1@YjV>P2ip%01QXB?0@Ijero=LG4mc~hcsHz6(f zv1-}(>iWuhujv6wi$^MCUvP#I2RS_P;iY$LHXZskdKih;R3B&GF$SDV8R3T18s$(K z+WW3sCe$zd0?59eRXGXu5$Fw3hQ@cAS{^7zD-yesx-LV@sF$pO!YngC38FJIWYuX; zFdxW~)~KBO&3vE1{@h`j!K zwa$FD-3gs}#~a8!hkpkBK5QjJ8QsKM9}#Skp_H7dRk+{WL$Ce(`^M_rV&}B29=E+A z(J-epKm-7_#nqjtwMXxpP$#cAj60YemZvy5--32tg^H&u94ES^Y&UO7j4T~f=l@iC z;hZ1psn5$-HHUcjMU3L0`Ob*0b`ij_v+3S*W}uF zE`+-Me7+nSNx#Qmv%HmAKHj!blRb8)reQWbLH6UvYbFcpXNzWg16RwCoxVV@$A^X>D;gN`;!H%(gYe}`?&GO2kr$8C;hD1GOCLDOt!(u+G@ z4ulD^_W2M-(gXwe%>I4zMS~~Zl=|Z0Jf2>RVIBXAs=@8{usz^0BpZ)K zZq@#+3bdFcnRn(Y3=}l@(J$SKJup$xI6Z9@Y!!+P+&r)p6!F_Hu%TpjzM;9on(B9A z`rlqRB5q8zSn8YgJS3f&Da`8gEw+eAvq%iwQ@)(`-7sZGyJoR6MfGC6$2Rx3%G;*Z zT#s?7S%=!trl~t^>F-;H4bxhCTp4+il4cQUz2T#8vVtn*+$FXc8+3W?e!Ckjwpsqw z-a!28*-uH4hghz--K#$qFSTt+RlV5ev3HR1BYB;}AwT0_$|)`!T~(ZTMZWWgND$ z(eqZ_itMIN@`E#Kc~bZ{#)OOHEVFd{IYQ^oW^6X;HU3S&yve<@e=iN{l-r7*(L)x# zA_N@+O~G`QStR{iaf?X@c^KZ9=hh`BIH{Ih@suVkmtUg~~U%say_Uhjw^B?WCwuiG9 zJ=cHu?&N~slJ4fV=(qoh5@Ymu;(9X{sZajdI@KOw2RUnaoaRoCv`8eIYluV@zWw2~ z($`g<7lG^IZc59`?A2w^LtlP5HJ>GG+;8GJ^Xx{G*+ifFb8R^bJ!g&4>C3eDLY<9E zc9}f%U#^1{XAZae%$XNlopG^5QTrcyH7IB6gi3BxN9!h}p3GV}&%WTFd1pU7^4nXo zn>^gjOV3$t7qF54T2>$|5Ttuu?vC0*VRTctt{BTmt(>i11H?O??A9R~?o8c&XGi>X z@sN&+7(>HqA!Mn2BShU9|5bXR0!<$Imw z)5PGI0XA?)?VldSuzHP+3slu>C9F@E)4a(MqLSovsHOzeu3X4XUJ$rxt{!Tcx^l{5iNkpx8 zB*pP9B43h{*V!pY5qpo9{ES!59-W3ZH{{nn)&@^K2!8vKL=f)q@Oi>G5N#en5W6z4 nCJqFsf@_N2M!^4u4$A+^o!(TdT%_+C%@mm#TNzavlF0u9cBXpq literal 0 HcmV?d00001 diff --git a/Documentation/admin-guide/mm/damon/guide.rst b/Documentation/admin-guide/mm/damon/guide.rst new file mode 100644 index 000000000000..15adbe2b4331 --- /dev/null +++ b/Documentation/admin-guide/mm/damon/guide.rst @@ -0,0 +1,194 @@ +.. SPDX-License-Identifier: GPL-2.0 + +================== +Optimization Guide +================== + +This document helps you estimating the amount of benefit that you could get +from DAMON-based optimizations, and describes how you could achieve it. You +are assumed to already read :doc:`start`. + + +Check The Signs +=============== + +No optimization can provide same extent of benefit to every case. Therefore +you should first guess how much improvements you could get using DAMON. If +some of below conditions match your situation, you could consider using DAMON. + +- *Low IPC and High Cache Miss Ratios.* Low IPC means most of the CPU time is + spent waiting for the completion of time-consuming operations such as memory + access, while high cache miss ratios mean the caches don't help it well. + DAMON is not for cache level optimization, but DRAM level. However, + improving DRAM management will also help this case by reducing the memory + operation latency. +- *Memory Over-commitment and Unknown Users.* If you are doing memory + overcommitment and you cannot control every user of your system, a memory + bank run could happen at any time. You can estimate when it will happen + based on DAMON's monitoring results and act earlier to avoid or deal better + with the crisis. +- *Frequent Memory Pressure.* Frequent memory pressure means your system has + wrong configurations or memory hogs. DAMON will help you find the right + configuration and/or the criminals. +- *Heterogeneous Memory System.* If your system is utilizing memory devices + that placed between DRAM and traditional hard disks, such as non-volatile + memory or fast SSDs, DAMON could help you utilizing the devices more + efficiently. + + +Profile +======= + +If you found some positive signals, you could start by profiling your workloads +using DAMON. Find major workloads on your systems and analyze their data +access pattern to find something wrong or can be improved. The DAMON user +space tool (``damo``) will be useful for this. + +We recommend you to start from working set size distribution check using ``damo +report wss``. If the distribution is ununiform or quite different from what +you estimated, you could consider `Memory Configuration`_ optimization. + +Then, review the overall access pattern in heatmap form using ``damo report +heats``. If it shows a simple pattern consists of a small number of memory +regions having high contrast of access temperature, you could consider manual +`Program Modification`_. + +If you still want to absorb more benefits, you should develop `Personalized +DAMON Application`_ for your special case. + +You don't need to take only one approach among the above plans, but you could +use multiple of the above approaches to maximize the benefit. + + +Optimize +======== + +If the profiling result also says it's worth trying some optimization, you +could consider below approaches. Note that some of the below approaches assume +that your systems are configured with swap devices or other types of auxiliary +memory so that you don't strictly required to accommodate the whole working set +in the main memory. Most of the detailed optimization should be made on your +concrete understanding of your memory devices. + + +Memory Configuration +-------------------- + +No more no less, DRAM should be large enough to accommodate only important +working sets, because DRAM is highly performance critical but expensive and +heavily consumes the power. However, knowing the size of the real important +working sets is difficult. As a consequence, people usually equips +unnecessarily large or too small DRAM. Many problems stem from such wrong +configurations. + +Using the working set size distribution report provided by ``damo report wss``, +you can know the appropriate DRAM size for you. For example, roughly speaking, +if you worry about only 95 percentile latency, you don't need to equip DRAM of +a size larger than 95 percentile working set size. + +Let's see a real example. Below are the heatmap and the working set size +distributions/changes of ``freqmine`` workload in PARSEC3 benchmark suite. The +working set size spikes up to 180 MiB, but keeps smaller than 50 MiB for more +than 95% of the time. Even though you give only 50 MiB of memory space to the +workload, it will work well for 95% of the time. Meanwhile, you can save the +130 MiB of memory space. + +.. list-table:: + + * - .. kernel-figure:: freqmine_heatmap.png + :alt: the access pattern in heatmap format + :align: center + + The access pattern in heatmap format. + + - .. kernel-figure:: freqmine_wss_sz.png + :alt: the distribution of working set size + :align: center + + The distribution of working set size. + + - .. kernel-figure:: freqmine_wss_time.png + :alt: the chronological changes of working set size + :align: center + + The chronological changes of working set size. + + +Program Modification +-------------------- + +If the data access pattern heatmap plotted by ``damo report heats`` is quite +simple so that you can understand how the things are going in the workload with +your human eye, you could manually optimize the memory management. + +For example, suppose that the workload has two big memory object but only one +object is frequently accessed while the other one is only occasionally +accessed. Then, you could modify the program source code to keep the hot +object in the main memory by invoking ``mlock()`` or ``madvise()`` with +``MADV_WILLNEED``. Or, you could proactively evict the cold object using +``madvise()`` with ``MADV_COLD`` or ``MADV_PAGEOUT``. Using both together +would be also worthy. + +A research work [1]_ using the ``mlock()`` achieved up to 2.55x performance +speedup. + +Let's see another realistic example access pattern for this kind of +optimizations. Below are the visualized access patterns of streamcluster +workload in PARSEC3 benchmark suite. We can easily identify the 100 MiB sized +hot object. + +.. list-table:: + + * - .. kernel-figure:: streamcluster_heatmap.png + :alt: the access pattern in heatmap format + :align: center + + The access pattern in heatmap format. + + - .. kernel-figure:: streamcluster_wss_sz.png + :alt: the distribution of working set size + :align: center + + The distribution of working set size. + + - .. kernel-figure:: streamcluster_wss_time.png + :alt: the chronological changes of working set size + :align: center + + The chronological changes of working set size. + + +Personalized DAMON Application +------------------------------ + +Above approaches will work well for many general cases, but would not enough +for some special cases. + +If this is the case, it might be the time to forget the comfortable use of the +user space tool and dive into the debugfs interface (refer to :doc:`usage` for +the detail) of DAMON. Using the interface, you can control the DAMON more +flexibly. Therefore, you can write your personalized DAMON application that +controls the monitoring via the debugfs interface, analyzes the result, and +applies complex optimizations itself. Using this, you can make more creative +and wise optimizations. + +If you are a kernel space programmer, writing kernel space DAMON applications +using the API (refer to :doc:`api` for more detail) would be also an option. + + +Reference Practices +=================== + +Referencing previously done successful practices could help you getting the +sense for this kind of optimizations. There is an academic paper [1]_ +reporting the visualized access pattern and manual `Program +Modification`_ results for a number of realistic workloads. You can also get +the visualized access patterns [3]_ [4]_ [5]_ and automated DAMON-based +memory operations results for other realistic workloads that collected with +latest version of DAMON [2]_. + +.. [1] https://dl.acm.org/doi/10.1145/3366626.3368125 +.. [2] https://damonitor.github.io/test/result/perf/latest/html/ +.. [3] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.1.png.html +.. [4] https://damonitor.github.io/test/result/visual/latest/rec.wss_sz.png.html +.. [5] https://damonitor.github.io/test/result/visual/latest/rec.wss_time.png.html diff --git a/Documentation/admin-guide/mm/damon/index.rst b/Documentation/admin-guide/mm/damon/index.rst new file mode 100644 index 000000000000..c6e657f8e90c --- /dev/null +++ b/Documentation/admin-guide/mm/damon/index.rst @@ -0,0 +1,35 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================== +DAMON: Data Access MONitor +========================== + +DAMON is a data access monitoring framework subsystem for the Linux kernel. +The core mechanisms of DAMON (refer to :doc:`mechanisms` for the detail) make +it + + - *accurate* (the monitoring output is useful enough for DRAM level memory + management; It might not appropriate for CPU Cache levels, though), + - *light-weight* (the monitoring overhead is low enough to be applied online), + and + - *scalable* (the upper-bound of the overhead is in constant range regardless + of the size of target workloads). + +Using this framework, therefore, the kernel's memory management mechanisms can +make advanced decisions. Experimental memory management optimization works +that incurring high data accesses monitoring overhead could implemented again. +In user space, meanwhile, users who have some special workloads can write +personalized applications for better understanding and optimizations of their +workloads and systems. + +.. toctree:: + :maxdepth: 2 + + start + guide + usage + api + faq + mechanisms + eval + plans diff --git a/Documentation/admin-guide/mm/damon/mechanisms.rst b/Documentation/admin-guide/mm/damon/mechanisms.rst new file mode 100644 index 000000000000..16066477bb2c --- /dev/null +++ b/Documentation/admin-guide/mm/damon/mechanisms.rst @@ -0,0 +1,159 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========== +Mechanisms +========== + +Address Space Specific Low Level Access Monitoring Primitives +============================================================= + +The target address space independent core logics of DAMON, which mainly +controls the accuracy/overhead of the monitoring, are separated from the +address space dependent low level access monitoring primitives. The primitives +are defined in two parts as below. + +1. Identification of the monitoring target address range for the address space. +2. Access check of specific address range in the target space. + +For flexible support of various address spaces, DAMON's API further export an +interface for configuration of the primitives. Therefore, anyone can configure +DAMON with appropriate implementations for their use cases and even use their +own implementation if necessary. In this way, DAMON can be expanded for any +address space while keeping the optimized performance. For example, physical +memory, virtual memory, swap space, those for specific processes, NUMA nodes, +files, and backing devices would be supportable. Also, if some architectures +or kernel module support special access check primitives for specific address +space, those will be easily configurable. + +DAMON currently provides an implementation of the primitives for the virtual +address space. It uses VMA for the target address range identification and PTE +Accessed bit for the access check. + +Below four sections describe the address independent core mechanisms and the +five knobs for tuning, that is, ``sampling interval``, ``aggregation +interval``, ``regions update interval``, ``minimum number of regions``, and +``maximum number of regions``. After that, more details about the DAMON's +reference implementation of the primitives for the virtual address space +follows. + + +Basic Access Check +================== + +The output of DAMON says what pages are how frequently accessed for a given +duration. The resolution of the access frequency is controlled by setting +``sampling interval`` and ``aggregation interval``. In detail, DAMON checks +access to each page per ``sampling interval`` and aggregates the results. In +other words, counts the number of the accesses to each page. After each +``aggregation interval`` passes, DAMON calls callback functions that previously +registered by users so that users can read the aggregated results and then +clears the results. This can be described in below simple pseudo-code:: + + while monitoring_on: + for page in monitoring_target: + if accessed(page): + nr_accesses[page] += 1 + if time() % aggregation_interval == 0: + for callback in user_registered_callbacks: + callback(monitoring_target, nr_accesses) + for page in monitoring_target: + nr_accesses[page] = 0 + sleep(sampling interval) + +The monitoring overhead of this mechanism will arbitrarily increase as the +size of the target workload grows. + + +Region Based Sampling +===================== + +To avoid the unbounded increase of the overhead, DAMON groups adjacent pages +that assumed to have the same access frequencies into a region. As long as the +assumption (pages in a region have the same access frequencies) is kept, only +one page in the region is required to be checked. Thus, for each ``sampling +interval``, DAMON randomly picks one page in each region, waits for one +``sampling interval``, checks whether the page is accessed meanwhile, and +increases the access frequency of the region if so. Therefore, the monitoring +overhead is controllable by setting the number of regions. DAMON allows users +to set the minimum and the maximum number of regions for the trade-off. + +This scheme, however, cannot preserve the quality of the output if the +assumption is not guaranteed. + + +Adaptive Regions Adjustment +=========================== + +At the beginning of the monitoring, DAMON constructs the initial regions by +evenly splitting the monitoring target memory region into the user-specified +minimum number of regions. In this initial state, the assumption is normally +not kept and therefore the quality would be low. To keep the assumption as +much as possible, DAMON adaptively merges and splits each region based on their +access frequency. + +For each ``aggregation interval``, it compares the access frequencies of +adjacent regions and merges those if the frequency difference is small. Then, +after it reports and clears the aggregated access frequency of each region, it +splits each region into two or three regions if the total number of regions +will not exceed the user-specified maximum number of regions after the split. + +In this way, DAMON provides its best-effort quality and minimal overhead while +keeping the bounds users set for their trade-off. + + +Handling Dynamic Target Space Changes +===================================== + +The monitoring target address range could dynamically changed. For example, +virtual memory could be dynamically mapped and unmapped. Physical memory could +be hot-plugged. + +As the changes could be quite frequent in some cases, DAMON checks the dynamic +memory mapping changes and applies it to the abstracted target area only for +each of a user-specified time interval (``regions update interval``). + + +Virtual Address Space Specific Low Primitives +============================================= + +This is for the DAMON's reference implementation of the virtual memory address +specific low level primitive only. + + +PTE Accessed-bit Based Access Check +----------------------------------- + +The implementation uses PTE Accessed-bit for basic access checks. That is, it +clears the bit for next sampling target page and checks whether it set again +after one sampling period. To avoid disturbing other Accessed bit users such +as the reclamation logic, this implementation adjusts the ``PG_Idle`` and +``PG_Young`` appropriately, as same to the 'Idle Page Tracking'. + + +VMA-based Target Address Range Construction +------------------------------------------- + +Only small parts in the super-huge virtual address space of the processes are +mapped to the physical memory and accessed. Thus, tracking the unmapped +address regions is just wasteful. However, because DAMON can deal with some +level of noise using the adaptive regions adjustment mechanism, tracking every +mapping is not strictly required but could even incur a high overhead in some +cases. That said, too huge unmapped areas inside the monitoring target should +be removed to not take the time for the adaptive mechanism. + +For the reason, this implementation converts the complex mappings to three +distinct regions that cover every mapped area of the address space. The two +gaps between the three regions are the two biggest unmapped areas in the given +address space. The two biggest unmapped areas would be the gap between the +heap and the uppermost mmap()-ed region, and the gap between the lowermost +mmap()-ed region and the stack in most of the cases. Because these gaps are +exceptionally huge in usual address spaces, excluding these will be sufficient +to make a reasonable trade-off. Below shows this in detail:: + + + + + (small mmap()-ed regions and munmap()-ed regions) + + + diff --git a/Documentation/admin-guide/mm/damon/plans.rst b/Documentation/admin-guide/mm/damon/plans.rst new file mode 100644 index 000000000000..e3aa5ab96c29 --- /dev/null +++ b/Documentation/admin-guide/mm/damon/plans.rst @@ -0,0 +1,29 @@ +.. SPDX-License-Identifier: GPL-2.0 + +============ +Future Plans +============ + +DAMON is still on its first stage. Below plans are still under development. + + +Automate Data Access Monitoring-based Memory Operation Schemes Execution +======================================================================== + +The ultimate goal of DAMON is to be used as a building block for the data +access pattern aware kernel memory management optimization. It will make +system just works efficiently. However, some users having very special +workloads will want to further do their own optimization. DAMON will automate +most of the tasks for such manual optimizations in near future. Users will be +required to only describe what kind of data access pattern-based operation +schemes they want in a simple form. + +By applying a very simple scheme for THP promotion/demotion with a prototype +implementation, DAMON reduced 60% of THP memory footprint overhead while +preserving 50% of the THP performance benefit. The detailed results can be +seen on an external web page [1]_. + +Several RFC patchsets for this plan are available [2]_. + +.. [1] https://damonitor.github.io/test/result/perf/latest/html/ +.. [2] https://lore.kernel.org/linux-mm/20200616073828.16509-1-sjpark@amazon.com/ diff --git a/Documentation/admin-guide/mm/damon/start.rst b/Documentation/admin-guide/mm/damon/start.rst new file mode 100644 index 000000000000..4b861509565d --- /dev/null +++ b/Documentation/admin-guide/mm/damon/start.rst @@ -0,0 +1,117 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +Getting Started +=============== + +This document briefly describes how you can use DAMON by demonstrating its +default user space tool. Please note that this document describes only a part +of its features for brevity. Please refer to :doc:`usage` for more details. + + +TL; DR +====== + +Follow below 5 commands to monitor and visualize the access pattern of your +workload. :: + + $ git clone https://github.com/sjp38/linux -b damon/master + /* build the kernel with CONFIG_DAMON=y, install, reboot */ + $ mount -t debugfs none /sys/kernel/debug/ + $ cd linux/tools/damon + $ ./damo record $(pidof ) + $ ./damo report heats --heatmap access_pattern.png + + +Prerequisites +============= + +Kernel +------ + +You should first ensure your system is running on a kernel built with +``CONFIG_DAMON``. If the value is set to ``m``, load the module first:: + + # modprobe damon + + +User Space Tool +--------------- + +For the demonstration, we will use the default user space tool for DAMON, +called DAMON Operator (DAMO). It is located at ``tools/damon/damo`` of the +kernel source tree. For brevity, below examples assume you set ``$PATH`` to +point it. It's not mandatory, though. + +Because DAMO is using the debugfs interface (refer to :doc:`usage` for the +detail) of DAMON, you should ensure debugfs is mounted. Mount it manually as +below:: + + # mount -t debugfs none /sys/kernel/debug/ + +or append below line to your ``/etc/fstab`` file so that your system can +automatically mount debugfs from next booting:: + + debugfs /sys/kernel/debug debugfs defaults 0 0 + + +Recording Data Access Patterns +============================== + +Below commands record memory access pattern of a program and save the +monitoring results in a file. :: + + $ git clone https://github.com/sjp38/masim + $ cd masim; make; ./masim ./configs/zigzag.cfg & + $ sudo damo record -o damon.data $(pidof masim) + +The first two lines of the commands get an artificial memory access generator +program and runs it in the background. It will repeatedly access two 100 MiB +sized memory regions one by one. You can substitute this with your real +workload. The last line asks ``damo`` to record the access pattern in +``damon.data`` file. + + +Visualizing Recorded Patterns +============================= + +Below three commands visualize the recorded access patterns into three +image files. :: + + $ damo report heats --heatmap access_pattern_heatmap.png + $ damo report wss --range 0 101 1 --plot wss_dist.png + $ damo report wss --range 0 101 1 --sortby time --plot wss_chron_change.png + +- ``access_pattern_heatmap.png`` will show the data access pattern in a + heatmap, which shows when (x-axis) what memory region (y-axis) is how + frequently accessed (color). +- ``wss_dist.png`` will show the distribution of the working set size. +- ``wss_chron_change.png`` will show how the working set size has + chronologically changed. + +Below are the three images. You can show those made with other realistic +workloads at external web pages [1]_ [2]_ [3]_. + +.. list-table:: + + * - .. kernel-figure:: damon_heatmap.png + :alt: the access pattern in heatmap format + :align: center + + The access pattern in heatmap format. + + - .. kernel-figure:: damon_wss_dist.png + :alt: the distribution of working set size + :align: center + + The distribution of working set size. + + - .. kernel-figure:: damon_wss_change.png + :alt: the chronological changes of working set size + :align: center + + The chronological changes of working set size. + +.. [1] https://damonitor.github.io/test/result/visual/latest/rec.heatmap.1.png.html +.. [2] https://damonitor.github.io/test/result/visual/latest/rec.wss_sz.png.html +.. [3] https://damonitor.github.io/test/result/visual/latest/rec.wss_time.png.html diff --git a/Documentation/admin-guide/mm/damon/streamcluster_heatmap.png b/Documentation/admin-guide/mm/damon/streamcluster_heatmap.png new file mode 100644 index 0000000000000000000000000000000000000000..0ad6cdc01e693c7954471ee593c9cfa7c756ce7e GIT binary patch literal 42210 zcmZ^~by!qi)Hb}&3=G{3f(U{j(kVHLpdu-VNRD)OOB_WJkre3~Kmh>(0cmhhNokOd zQ9?Q-CFC1_&-;GgKi_j*lV{F4v)0~cuek5EU+8Mzq#|c02LMprzIEd+00YkRfmL6Pr z&>0$g#@*c9Ot6j-kaGe?Uce;)1cZTv1W=R#>Q{k|5_o_Hma4!>9eCdc!I~gk8^q{= zWPOlr2=a|UsWJ4?3~IE1zC46Ft)M;|=(`;>;Q-A#LGvz%B{#&H2V%<;vEz-{_eC7~ zBaZ`+Cy$V)!N|WML}xJ3*<+%!Fyg<@h)=_bk0Ve=FHi?BN%o>he#ek(#FDPWkuJU> z`e`OuE?S;%BIQ7p-s=FOUR>(d_(^vpCO=t(Y=t# z?j5sP5sN_ytHyh_Yo+Y6WgNog=XgJGvQ=`?RGlaO$PKD_Kn*Xby$I^~LH#ArC;*x+ zgJvP{Nf_WnK#LgoEDpX%g0Iq`RR*-lfp&S&p$IyaL6-{nb`5l^f*v){dlU5C2K|~~ zKpPC|f}y)$*Z_RL4@Mq42?F43A!HheY^#XWb@FM{H3<7hZ zVEzeMcm@_Dz^|9!S1ee11(p-RN-9{*0BcyVo(DDxz(x_+ECpK?V7nUpt_M3!V7CSA zwt~Gbu-^*~2EpM7_%i|i%z~piaJ&Rg*1+jDINbw(j|g`bVAv6xBFr(F`>h9F08oGZ z_kzLD5tE$|I1FPoP` zo;EMB$l?5g@RzeIE>fq8uLaB~blOvitOJS%l`R%U#~%rGo}_Yq_tiB_t&d zL`TepB*Jx+uYSEi?Gd4R?kDYYx&NP!yb*KlG_hPt!J~IPBC=f+L%dHq=bmo{3Q})Z zMn9*CBaV%k-M)M7XM0pZY-~(SvPrDiCnbN9ms|LOfiIg{C9-!uYBn@X>Ifi}1Jl3x zMr6xnDF<%NFZnn7>>eH_T)OFIfe*W+le4x|jZ1%fedLihWmtn77boYs8`n>(7t^{b zS98l~X77bu4LmtIcm&fdhQ?-f&nl*k#?RGM>pu7A*vLnAKL!RG zIzodc#_XQAHALI z@Hg=g{=21WJYoXQJ~epI;LK1b$Fpxjuh^amfP<}>TDC>4?Q1bT$7iqI&f~-0qE5c? zp6a*Gk+d&~6z-=#+A^ctTBI6qi=gpzo;fQBastVz0W;trMmaLCZ^Hh#tC1;OW?!kf z$zV9!G?eim|B$?|INB^DoKga(2nIJOzzl!+$Zz9?MtBA+{CbIdziG)r?$GTT3e@@=L)(+t> z%s89_Tt740{`)wEIQ-SHO&2qA^YE4Ok{nR{st}FDsWuMP5^sezah(5QAN*8wIV|SC zWB>CrVf4NKKOI7aa3;ybK}SnVKPFXFH&qD3v3dU_8n~G;Z?z1B>BM9QxsSHUgP>18 znQ@yNZAzB}RsN#OT71D_Nb4-sR+dvs)EoD0JBiM7yM37tHF*`qFBS-^g%W8?^elBR8 zB)())RwkGeOIrh)8#xX@P+-avnr+^%vsASES-l*2D&=zjqf~K&hhLQ|LEY_#lHC{n z{x~#&Lbk-q(4P+Zf=_jBeyaCz*|6X|ebs!nAnG@J_H!t2y2oN|^Y_d2M`uH`egRwA z{2_azo4;M>nzw7~_9~Hw#iQyQ+08Ut+0-Dkq}EN*)zyqJ1swS4_~V5Eu4$)mQQ@e^!aiumsv#N7(vb!ZQw+k4)xPZLxXp4rMf;GqVy zLvkS}TT25&qoL}X?V_RE$+N{sTsHRxTV5-a~0*8&|~iqwb?sPvh6E9l^cJ1 z|8qR2rMsgfu-nxQwwZ(T-0QnryFWhG7Tj$2+wfd9YI+_U^Uyss`6l+LsbT1tFgZzc zSwEVH-Rn6vk+^_@vx3ov=|(dArP1Rh#gw!0^8#C>FLsU}T^anp=LVlay&gfqTb)$6 z=dsFl@pZFW`#*Z_pV0Z>W+$zpW@3@JnsWzHYb1!RmiLskqGZ{Kh3zk9FKSfoE)4wc z`K?+%`-;Ks$^UzvFg!QyZvnVwi=gat`nUa0$gVy*vbXrT4kH6y~}!>{fQ2r{(rAUxlRIYZD%T-CBNHm8Q7Ca`sby{162qU z;5jA9Tm$y~=1#%k{6_pGNhwBn^izO7!|yiiXwZLy&~gfZa^0h&XEx;OkiXJ+vxj8& z|9k1Ejbj>uIl^KcezE7*(&H`}fQ?v+;GOU#S)!)%Bygxw$hMX8n9`R=6c|6*84{Q# zd34L4Bu?2WUG|G&=0mO_QCH4lzmvnI#k^YC)}X0&(GcA6armR4vzwu(*JiX&lF#g~f)@$md_mex;9wp8P3WUj2J1`@do6nm%NO z{_60j^gr{xlKQPP-vuw47co%(ZD!%yo%S@nzb|xTh&HbGXOfOIl6&TL%dkw24}>cl zUSkQi^_-hi|DHLgS~XI^YGtE;w{vMN@m8c7N1EjwSyxqEUx9d8*EEAaR*dx(t&;Z) z-oDn-l#ai;TK_)!u03Z)jnDaLzsO@&GZR6<&w|O55`0`usXqm(c7m(Vet%;9^>i=t z!R&(qeYbfTxwo0O+oW%A^UV2X=6M?Ll+;+B%lw$*9dLP%@nu)(_%hN5_|O^DeA?< z|Ggdh=CHX-^Q`nj;nK+xjZ(QptLjnB!44O*=byznBt+dEzqlOvaf)M^6jPBmyDv_{ z`TkYT_KHnd)H7Xst}%v%qEb4wA?#d2zsK?bC_4vFK3s!?t_5Rc#W$ zS#!NgHjC}N+yRTkewTQA-HgoJUn7T83nKSQh4}#1Cf#SYajlID^iYhmx70~KxNT>` zmlt}W_N=l!Fh65KFt&SHlIvA;m#w5@vOkOEItOAY>SLSh)BTo7|9>mxC>2O)|7jNX zG%$XHxqPoKxy}1pLreXd1$(QgO&`5yWD?I@ba$_Uhz*A?@S26N(W=9P$1652Q4Urz zxvlbUwQj88SKbQsA}i~C?>p&r)%whkGKJ|)W4!* zP{B)xrGWm)>Rq}Qf_`R?V^IzxGA1)^)x}UbEza)e6zR+RBZA9hn7g8i$)Wp#-9U{k zRn{qtQckd2AZ+_qW#e%EjGPRc19uSbFtgK;_dmlm{N}8P?0N4iEk4tG@XWe)}&(3(k!7vL=oioq3XNtHoCpn)iqFgGaU5hChOWDJw4OZTLG`(nzZ zS}k9ns9z0n_`H5Fg_)E)9im=Q`?=Bg!CUIQm?3I0o<)4IwvpSECpXFY9q*+?MYK!cc%%A8F>u-YuE3b9(P3ZB}3%+jpJw%Nu zsCNeCH6n~UUgL}=Q5EIFI@gO)3#kjL0~$Gvo@+(MG`=S^R*nC;y?n;=Q7WlW+qZ`|i22oVnDc*Ydf7+uy)MZolX5AW8i%aSLMJ+ zi0A+;+9r5a=kvnPZX)b8=snlHrsiq;&Ouf8(40SP=%?h)K@y+kkk@ zg8~D3^XJI|PJH*zv6WjE^0L1c&^u6`LUiQ4qssSfuKSmKoN9s)hoj)(Brbq&ivzjr9K=$|5)Roz!VMzV!L`zCOw%`LxE=O!a>j*?alTcSQbOafZ9ClXY?K9q(MB4hraXOGycPtsH1%x|xy6 zHPlMk&nVc|tC_!|o}Xwah`{o}^*X=V08b66N(EhGlrcd0TmyQie)&-;r%%2M?-eIj zyy5`oJcl|3F_ATC9&ggIZJsgb-hE7elVI3BOVcJ~ja`pRyz|f2$4iDYyB!A&fkPlN2Z&xT#Bev;0^8egRv;0f=bzkE>7I&f|tn?Dps1}n1Rv_*MQ|HCqAG7=I=iPz8URup3=xQ@>8ok zYcTdlKz!6jde@p)#{HoOPL_{+%$K8! zf9+!vKl?ta^DRz~nF72v8+1aXj9%E+`-@wLRfzEK=lYcY%(r}@q351yZLvl~7%*11 zDr|Nk;%uojokY{g^q)WMxC)kT%-D}_f3<}lYV6evQI=ai&Os;Sd{|eLka;#Ej;NJR zte$veePLpWcb*Ez4w^=`ySJVWPD7aCcM^@CZ%oqRBzsKjQDYyOzh6o!O=g1T8;25t zsPN18Kk&d1c(&SGJY}m_@s%+u;}$qhiKnZ{0=W$JROQ{Em<_Y9TAu+Oc}rZZsSu!v zl1qoe*|4649N#J9IKDL6cC6A;UU3F*b&4L8ZZn0o^7CQC6F{iQUZW%wyMWr2^^*doqkDD6gQ`xK94z3V86= zYD;n0=ej3GUZ4RO>=8>~hqs@#R*5L(W5QBD#S^)oBgG4He4!Kr7>($39-!V`LR9{} z1`OIdBS^_4(dKd3(VnOLSnDr0yhBWZYrUm;4~Sl3h65luI-E50x|1f}u}2eBs@V{_&_(^}saJm3tfp=TJZG4{IF+)h zczFP{MIQ)I%K7Onqg-?HS-_(IDJFa|5S1cSb&KFf09Y*qxWg8hLzVQYyy%08XjF>WcpsbYf=^7Vv?UL~Z$@wHFHQl{?{^+VYA znbtLc8Z1F#mJ3gK+pd7DJIv2kR*QCmqC+~dDgJY_2v}U0kAQT1x9CR_^;~}NnFg?A6iIb zZVv02{POF3t{3fio@3{ZZ4%epXtn);2?Q2_Rwg2bnb)(O^;Of(ru$;}-d;wm(azPC%R%{0^IVrR#28VDXu6=jG=uc8nU(&!htK-9^exDaN1p;GFi021*4z!@3=27-^Eb)sP3;lqpK8;4vVcz`7%A0H5A z2%%MheimfwhO#*TP8`P&xVOMGImS9=Pj1`&F}()qs_=%UyM3pi!%%6~I_8x!qXixA zf=+HIKPgs}9teTm9%*zdh8URXY9)Dc22CLqrNI&@=4t>L?4QI!`E=r-_pj7w z(V8u#zolb6Xvzgt0mC=*^mQG`z0Q2^IcfiI;NIMCr$IGpe0gxo70S#flVT3Sl6e^1a(B-O(-%EXI1)soLizEt*#gVHK$HN)XQomJ*HHI}acdBoWM;1r@z{*&4}ILqk5 zMMC zk1-jSOB!VCQRD|~Y62ew{iuPu*f2f0%#t{&;H+DqT?}nk_$FR6D~%4Ur><0HJ3n9~6;{6lfHjtLH<0?@6xWI4AC| zOGfwgNpno8b$I580t};IP7U3;ECyT^$yiB6$r`1uHH$55U;UmfmGRZGKI2=(4u*l7 z2f`pcJl&nD-$jw#7?cbQ#H4~FPPA}cv{trF{N3;Vt{*&sfGbjy4kUJ`O<>u$2OX6O z$tmcVyjI;hXzrIORW)Jtx`1B~rD=cdyMO~yl#q*blLH2)AP=gU`!=N5Y)D4!gM8i? zJ<4HN+-Bfm=p7(w1>}rB$d|1@m{LcKA+c>l8Eg5ekcKMAd#51mpU0i6VK1G?D2X0Y zAK`dW&9lp62VPU4omTcPS{zkCxOrp5(#acJ`1dn$p^xB(-E}}Rk=Gw(gC-v?2)ce> z6-4iVl$8-vQzWB4zzx>TcDk@3UoKsU`$oon_d|5wkpPIhDyNEk(yU4O0!Wz9$S~`G z9%0 z_r6&^25UsfYNLqC%|j$w(;`)tUz_)&Y&pm|Q&4fE49FW!OC zZV>P3m>>Asp;Ie$-_nAEP)_rvxsbqI>uRfxi4CYQiwFC6!-ch4n*UChC%LjG2^tkp ze0-Jfhw}SG+5DEyz1HCpDxKO9@3N)udNf)Vs8wpfJ4DOLp;M&3qmB7zwH&X7& z3I3-}uqPC;8&Ap|xsmaWQaAN4ag9hCQ*fJ);~TBjxJ2@MNz&^p7&Wh--yS@-ET5ym zg@Z+6?mM7%zsPm6RAv;|{3;{mHVk9KIQfAFLwa=4S~_9c`adJE?i>vy{({9)NNb)4 zw}H>QM7Axll1fP&bb3UEo}uWwE=iB;+9Jk3BX8n`MXmr&=!W_n5$%JF(M-wrSy1>U zWN_v0I@tS%)kW?<^8o@Bj@abE;ld;Pu+r+b-y;r<3;yki2tpn=RiD@kjL1mfAD0)i zd^NQS+o&!IZv1p~d3nx^ULx}jaMnn=B$fQIHxsfQ0s))O(gu^({0hjZJ!1Li^*^MC zOm?qNJk<(0oxtLB#j1T5iqHQev!qQnPA$a!glxpWH)E1O zj8g)y)!s#BSg|u)O-;Z2<68v+;!lb#jf)(f&p4~3esX&h!kDVX)+FTk`|5F!!Lvq$ zycnx=lO-Zxb%6`Jp{Ei!;|~;zc@$%fMu+O$H3mome_2tVi~jv0_7@#+pjja1*6N|a zt&jr~ga8?K3aS#_7Fx(7Mz^OL4T9lh|vn5N&#S%ipj=J!ln-tZsiz0;dFKkYV57`e)SO%lOY6UcMK!83OUw1+Bwteguq8zhCbSi{?h zORQy#*izm{PEsn%=BS4wamE$*6<~h(5oaM`u~;wW0u%5Z!aI3<&7|1X3)CFe((h_FeaAAr6a2=u%JqHqk+PftL1K1Pf~SYMgU9L_@uu#*9uF})`}?_WfU z=NCjoEVg+4C^7z)L5gjX08665&zMBDf+treff8iOfcB$upF-{$GLLivmy56fu`SF} z11i|C@uuMvpo#()txQ-LG3o}6t5@=h>-HxH!>jXB9jc@+dq3O_a4TdoS9mr+f{P1l z(#Vz*==i_A07-diWhjIIq{}8bXf9;CWJT+UV$CenBcem+$n}^_v4i1Ds6C@AMEwB) z;^vDn1!a8~jcva?pnyB7H5kxRBuEgm>tO6#c*~)ErIH^v#R+>eYFr<;OFN7F+Zr(< zYcs!HktT_0)p0S)S$tG#`Kpt2;VYtiJ4V~ApV5m1PY#x)cokqlB=#mdl_68x2oY_u zW?^M1Gn~$bkwg+Ac!}v{e-y05)DwiI3E1F>O*i@J8WaTvxb_R%cx^Eb`Te#s7|pq- zuYW?{WP1;6A<|cpXcCrlZ?!Zx78bZLDg@Aj{$jH#Pn6;C+jwDqV-QsvTSX480+}`j z+#1hf8-@I<FmMlhu6?mRDz`!%!8hS_6BFV{vZz(3be+(CW7 zqFUZJ^#LI({xb^sLf}E!9|*rLr*S**{OZpX2OqsF1s=QAM$+L1;XFQK`=9bHfLM`ZBuy^$twN!;a!xs<+; zkx3Ukp8YQWr%&+!xe%xd^DI5Q!>i|8F1S66l0=$9-11J<*0_HlCAw;dKkzl`oLfH? z`fYDJF}Lxy^#`%u4>2H6$_FeN+|}oS)jXUw|IL&N1MHI{SzC4Wy>~Ia*TfMQs8jur ztQuF}t1S#n>|Ww%lu(&MW~BI$b%;l0er+8MIKZ&F`}EL->UiTDF^Z0=m7r0otRfql zD7@b9j1tI*vyY4C85f2f>xtu#n2!~uR7+|K49kVOQH3pcZh1^ATtzA(LDf73Ze%j0 zduba1`f1URHD6xNIb_423+lDbt(P>Q?AJ0N0=Pnq4xQ-%CcnaX;os~SzQp=rNp!gu z0Vn7MLzH6e_!~MTKR#T+9sEaNOF|ttFJ;h!LgR5q{a2#ik>Znn;ylrUl&jR-nHS!F zxXQhXRWXpnMK?oBcf-SAvN#r|Hv>~i^6Ow_W+*=)Aq~R&?dNKdy=&$)2FVVgubT#<8ath z8@9y}28PKo`TCE$y6wNL1Z;MHj`S!JI<>cYgLJ*<({h;I6+j~PZr*$v!K$QwtCbC3 zsL3I{>e-_zBP8#jQS>Zf<4rYKnzwF}fii_&8I4*?U10v6yeo;8kIZWs1l-1t+3>ys zV5zoW=KYo_C|t6)zN!w^#22rXR&KF@PckzsQc7<>kpfl(aCHu2f5NPu%Mqhq;%W79 zC%K8DBZ#Q{MazK_Fb5cZ*onVSD+5w~NhEgPsPS%GwGuSkgQbF;yYHE-L(JjKf+G$u zxk?150G@A9afkx97J}y*Alz|Fl)pP(65&3^PgLza-Ic|5$n+K%s59Hl@j+FR1W=@7 z98nIT9iV)%UWUV}2e9q;ORVRe#=1)R7@ zVQ@)Yr)APZ5^0dO|9p173Uqt!29-j{#zS9}zb&7wUyK^N8&u+`Zs*fa6vyUK4H~Z- z#{^74l&4$lXlmF78e505SQhezuNxBSZkvEVR5&icffyq+fpn_%)kaC8MgQ|}1RI3i zg5EGBUzBon%Dlf_#jam|0X9?7X>kIKSzzL&^H-RC zRtj`Pnp@yK_C{{_g^08AQ7EW5tf%T%PcRqTOm(p4t$43AE7#?#!otlzFFMhlfGc| zUjqko1IPJGI4?$RbLoNGfYOqLb~Aa>yTW}w*9orfFBPr<<}T+z%x5|R?$V${7&c9e zh_LsQr=?bn6t94B*_BB`CrwOVuR*5p^yo;83-eyY&&@Z0n-6ytDdFV+l{>Fd<7w)}` z)>`pmNkEyU%vCxH1Y~()ieA1wMx9<$o2In@^6J@-ZHoC9OD{qh8XU6cu9lQhttvvh z12U56VL0c5lOs0LVTJ$r!l)K|eu?OPNnFSciAopyw`#-%ws0V1Z^1mU4hW#L5h&WL zUvfm*gYr#DsLICXQqKY4g-ov^#yW-CWnS=;A(rGd4;YHr5r>~8AFf}f^etchKhCb! zw^8z0OeQf-vOqsN9|qx_d&E??9oJmo`;M~2d6qgNF2;z6jQ|o_hMIdO>W>oeGv;G$ z7a2ZGWQ$S0;e#_{vpABy$qJ(=A{Il=-Omr+AzN@POX}>2c!tUCz!-j85d_rmO-R=tp#EK~dSzx}OTH5;3$aw`g zE9Z+YhgBUjQCwbGH9yq{rHAqbgyr8d@L({UJ2C}Y+2Ck^aBs{@CZ(a_lK4^JGMGN* zcg042rK#+@W7d;L-JuldDy?fB_YjUZfdWuWY+EG*s~B!Gz%5^5>^QGzD-AqRaQfiQ zG($I#MMM5FznikZM~E7FT}snTPn_VsG#&^=d=!7Wb5xQE9kF4IfTIa1Hb%&zDs<6; z)n;<=JZR>+`KoxGl9n&q^yoX^O zrk|St4i^RQpTU=Ot`8nP3k~N<)XC&3`SgEyTjTsvv3(-JC&U;%P7c|vEs^?KpTn-NF2+X4kPxv>>n_6AkoXkf zepwQyfxxqtL0z3=+WeBpwc?};oF4avvYvx65im!E6NYZ+iMO(hvh}mdw=7WVCTVgs z)hBq{*m$D#RZ% zrHGXJ>^z0oiqcLcD|lh|_P;Et=U>ro+ZH)a1a^1n>R&PSB@aFo%#S*1!ivdiRgUxMT;3Kw2Y2l z>an(6zeY7OVnx_|j2JJ5^d{Phr9eL_WP|_B=#k*)0NP>Hw&mwYiA(>0Lrff?WR19A2}>$!@jF%4ciIdDe~P*oD04l zX#%u?+%93yFlbJ=nhrek(tcm9*FV1I zrDlgsC={X38=`c*TSlU07kvTtwa<5i##}+anK&P6ln#e3U}t9D?06>G^L6FgL8)Kk zHM^m-ddIxL7p|R(8=vZJ+lJX9HguvmQae#G`y3Zj<#$8e9XA7}bm=ho`4Au)# zq)Ad?g&(!)k_FGt>3driv%s!Vb?O{B5}?`-njq$0|B+`0BlFd{3`OMYZ$yiDpdy~C zT#QB4&qn=`!Tm?#twi2_%%m1dAYOtgS=j3s!M#kGnaF(K#CIkH?Zvm)xQkdHv_~tm z(HF&KltF)SnRJAnJ%-32RlDt-wz2o_+gEGy$r~tZ^?(2pSFIAa|1x@?Au|hrm-qML#P1t_ zupyLgPGSom#{PszT_#W#zW;zP{_nTxn)u*#%sPVBn+1L!^9k@Bj%56gc7qQT}D*7;SMz@Y?T4BuBtTG*~2@J7{*1x7IX0sYIuXDC3-5QK^d z*#5lDhA$pepRvR|9fK(MB-L0roIeL@wUaeiia5TQrD4ci8MYCAhjM=9`5T2#(bobN zd=vYma5HlrkyJKfgYoGDKtAnh!>ZK+{HbGeQ8;_Z9bT^TC{{wx@kEFM2RBJ1%m$c1 z>D9b__Vv>B>G>;=6jAvf@?W4Q5sVens6nJ))o#QdLA1r$@-%GhEkC^;O6?~ zRtm*Lw>=7Ljr(b>3c>{YF4yEpV$Oq^$UHHm6j5B|$6r6Lk|dV2vEiTT@NG%AX;gH5 z2mWj((>CJBsicvuiJ_~%3tJ!-P*v^61rO#H0d10$-#wA*PWZYlIP%Lw8 zBWkxlxdzkbjA;Q0;GLx{lDenCe^pu|f}f^rOM%i=o%!esWxu^}*e1Wc!V1FuOJsy3 z?n4;;1q}rH{%?V-p?4@qjK^yhND=klw(b(}-1y>mZ~aEsKf>G$6o~}c-E&+YnE`L4 z9U;|pXT#T$q=;U28FT{rAtThG z6u9rxhd4<2fA*-9ovjtpxC)z5bdG*Yq{3(TxF9S z!`nHlOD;};R^ND(9<5*v)?Up!F$G1}Tl8K+TG;UF(L;Ez)HHfF7Oiol=EKXsh|1lT ztY`YMPQ&}_lv&E^5WiMh>L z0uxE$Hop~xZzB_~-)PPHAHk)Bl=9Ts58MYv8Ht}puQ<2~xg5V7;IEg9FrrbdN)?|@ zN#uo-_Ip_q922g6*_X#Dui9lvU|CekDgGV;O68DsyPKIgN(-0d746@gQa&+| zFx{#^cG$wXd~jD}u8X}b90|0$s+j?YuQoWP##}olfvZOroxgh(43nQ;6PWN<15Ng| zr3Y+cZ*@OYmREkB%&AJSY5w4{CF8afSBN9!o)aoT&Kj)ya6HrQ%h&W%fOelsE9I@ePPOcmxA^(&49|K?7+sjVZ zMwr$X9sK-`$!u^p%cge{c<1>jF(U$BtMhxqOK0eL`V?SA2K0>Wa|EFujCv9DCvy`j z3SS?~OGNHtP}FUMXwr%wztaO>O9{2jSi7wuXjAIm5kogVCA?Y=M37QPYbu%TRorF0 zy6HU8|3VF!%nLc!+D#6vXn|8o!XhFEiFdQ5AV&L#>(HBBO~ zNaEfUbk#Sn0Y|I)3)2WA2=i6+jk_X^_cj|sAk{1L-1U`sh}XWY_8Y85ppM_`m;&6c zDwH)H{Yb>{ke|ejk4q`&sB`FfnN~?n8RlSXV0$}=?8Tqvn>UZskLAb>-lgJA-cUTO z-6D$HRy?{m^7lFh&p4y`lBuo~qIGleZhCMq&W?y+7b8;x6i^fxdqa%-)H^V!2Awavmln|p&=6+uVll>c>lN{ymwXKDS@9uc1XrHij5ck=I zP&mznO)xGHS}**?4PPp>75`8mSDU46^Cf_6rr|NL6uwrR@WzlOtaSO7p2L+)4r^3x zoi_zx1e`I>Wfhf~tJE>+iYlC?LB{ zKMwM`L6|yJM^9wTBj+F$)-!9FO3=Q5TrosUj&97H{DwXwnP9Y zxU64!c-e>;_RcD?OKAtX4f{-q-hGX>#u|iC8OP~`(&H*W+R1{v1c5!%5fsCQdjJ)I z>9V@@A&+v*sr4lyh7HpW(O%UkUdNJRowcY1w4w5+4y(I-gz+{a#a@E#qnp#dBKj>q z;>|i0VBgQLCo<^K)@!VL)%WVY4Ell&dIAsd*8?gH>8A8o8g}#Ds{WZ0p%mm@N&3j{ zPTz+hrL3|G+f8bUgGL^AI29{fs~M7C0nZp)5wh3jxfX8|6TPEZOXhM@h%v9{xVUS6 z8J7Q={sWH?GQ9U{LWttV2akjYu9WZuw>`Gx`u^ZD12T8$KtOsP$e5vLiQ$JRLR-VMdTA6N{1|BDd}8-yiyJ7AA@0$z2Vw@)_ipIF z&tT{)yqe5AGT)0mLz$0wxY@ykVu!#5=^(b+S{foqzOD#nogV(k8%=Wjv)@^PH{ekm zJGDH2b9-0!|*gRhL=8ULG!B=p#t$iG0K>;P`*oCWli7`ep;lHIWl}PjDrCS`UWs(&Np2amB9Z0< zJ3y@Q@5U+ny(qpaz)tO;Tz%h5)ZUgfe~*aPE&d@eI-$ly66BS511Rvz(}zyrd}AJ^ ze`EKlPE3~a#CLzf`u01W-p-SCCeh@1=}DkZmZ-faNgpze97{Pg&;@;e-=8sfHIa{j z3@9Ls%7&!T^9)`h@yhoKuFpK>#4Qk!2z;*lMyQHE%)OLc)${1pFXs@8x=$WU-$NNk zeHcTRs_9kS7Y_wb1x_TOoz^Q0l?sa0C~QP9r#yqHiiZd>6NDTq3Dd$KX`Hx@eTYpp zQF(TaOBT0ZY`2Uil{a2LFV4R%SJHh0nE$N1^Fs(P6Qo@%6(2Ih_}=>>Z1?e7%9Ez@ zNs9KIAp)DYbNNvx?}hJaE0kJY_A^(#e1}bD&Xe)^pJNIC(N+R1AB>S zGo)Ku9#s%->pBDOTU)>kPo*`-H_Cpur$)~{QYoihL-BWU;RA_b0-(<=ZR>$rK z?s1Y~8t8>L`@al2%t!l7tg!qVwwDX%f1}RKLIHKX@1cGK z*0-DQ-BOS_u-E=@ivmYW0=`C~0ij3jkkN_;?pSU$UMEl=+nO=c_ID}ZBsOuR0g+hb zBrvNa=*9E(#vzdKhfly>?al2l1H^pj;x5>KvSW z#EH&oteZkgf!ZGplqupqBOOVXTC#TPQ}KqqVEN+quMUQ1g5eV7bd%gWiCKXkNN(8~ zuWyGI&s`PD?FKs&(pdqw!CPO>{9!sUKS0ZWVLF#7 zT;a*P(%5K84PH*s<7&;oQORZRrrAyLS<8QuoC>D=r%l!+BQg zh<>en&D&PDnAT(Vy8VdysW^={?b3iNQ;mSg%LF3Y_p8*{G^m1#ogH!O(hAjuZGC1V z^f)n`nJtNRJu3an4s0&zTXF)45GvYr=1hf)(Qos4?-ON1D6j_NkXQ>Kk#GfiBl*tu z*MMyH9ot&mkmS{z#qay~N{;>_Iz3Ufxzo)Ym%1~+$ z534}y7pzcujERXpRfKW4$*fNc8Mx9Dpy3PI>q`|SuH^ccWlSTjFPZ6ARtvtk4Gbpo zJD;-hRZK;c2xW+VST2nt32OKUM2ftRYxbpwdbQXS5ec#tVcV>v8JBAQZI;FuHX({QBQR!|8(Q6=`(gKcjgEWXZ ziXZ~g-3ASc2#BKi+|T#-dj5o&d+*tMul-)@Y!viz5*dSa1Tf}*1XQY;b=kJaeq0aw z=+NqTzVYT0)6;0te-EI}*8e=UyEVffXqA4w$ zm30BS@G;|3iG1+>nTk;GWllU8c)}UpR1J($pHw!8<;wYKNgso2wq2~*3tEdUM8Xc7Dynh{d z^tWFnDpd}0l-SERwb~E)ima3qQtrx#e8^PNXlg?Xf9@^(c`3+p!f-(O3K&Bs0dxaq zSg{OZh0+p1Pa?=yI-Nil7};}mHmCe~fTf1iwO^F*Grw*cPFn5G=`aU~FE3Liib}xS z^k&9w3d56Kks}Ic68!~a!?JrXnA7{3TkQ3o#>FHz@?LLXUM>iu0L6wSq-EpTF?x_Rf&#Jk_nyrBI zdrHFPurqmAg(s$e(F_j(H1y|ZBC9?~IaUIj{9Xp^nF&uo-RI6M+6ucs{qCQbVa+oe zOnJf@PEm*96o%a$fA4{(O9?GtcjK`Z{i#b~dy~y;+`q{9>>l^%R`}_8{IUKD)v&_~ z-h1y+hCev@els155?ZkY>GXk$3jt+|C4+-tp|8H(9j8Jywwn)}eFg+l*?Kg0pgU`m zWlKZ@v{;2piO5eXKj8Gs zLB3CYWggt8+f_iih=(M>%K{W_^y=K8kY)-FX?d=x?zkT|!yF%>!!;Al28T_G`dIu+y z$_IN>)S>a|0wA__c&OPfE;s7YgF)5~N78$8;2D3*`YD=^JKLFDIEr(b68?8N{8sbX zcPWc8opDlEINUV5Dj_prn9v~N=INRL(RPq4{zHP}m_8F%rgxJyPM&={HT!vM)He^$ z?55-eEfp2#c9nlpG9GB{y0Ay$hBt*8rH#^Uoc?^W;lDU3BJg(-@)`EfAFWQ6D~W(%O1} z1B+olcILKhzIFUvt+rS^csJXummM}y$sh=a396c%cTTdsycz(D(1Knp@%sTGs@U>b z9hVRyv_+|^XBZag1CJ5PbI<@YzB@q)t@MZr2>jlWC*AFjT<;X=Bw&&ybAQZoLqsx? zk0M~S5`;5W!6*Ly+2OJ}f3$Qh6tAexIBSjv=uUQKPQv=Vx+@OEt zx!`YUg%V^Z8}E7Sf%>z#vQtWabole+v(OwR;f?noM%p7OX>`&+@ceKr#g6~SQWox> z&%p*BB8;TTc;y7vHJ-i_ERl<9s{WxulcL=v6kY@cCffwzbY#)Xy!PNO9bCYo5G7A) z-Y{69@nhd<&Z-=gd;gswmT*l0+#7oeHQE~5X@fX)RQZUbXZeO zYpW7GyB-<^4UfF7xh>p?!S_(0XVV5o=KHB(&FB7zC$#-w*>a^TbLEo5?lyK=FYyAo zO*l{=(KQNwUNqq(pz>`Z-q=@K_g)d=JT^V!q!=bQbYvK~nz&}MlN4uDi$HMGVAwT| zqm_Jm|Aw2Gkaa@=waBc1{s}F7>CPB$hV?b@17oN#p^QhvXx-d$^T1FtZ_F?t3Jbx- z$*iC`mp1G!s@Kb#QSSHM9}x?`nC^YZg~s>0vqbJMfq9n+hTmzF#GTg7IUN2&$Pp7> z5&9#9nifWJl)wI%&(8;bWBG0bZ3&}{X?2ov5cI{>(gO{NccKZ;ljUmJK*d=u$6r{2 zoNLm`M;&CSW!sCDeyI*n!mlB4wyiCzy82f?x|gQsN*lS4oPw9p<|s2I;5a& zRo6x(U0>=pz*{@9f~Fy|&rP4S<@uv>O<4SB5uW0iQAVk0_S>M7F&yEURd@HtvWMwn zfjkN&z;9FVM`WiFqQt2@# zq*SFu?cs>8P=*Zn(+S*Uc8d8-mrmsI8Q}gAidt1DyD9@;Wpr?P7$~cGSf)vD#PX_M_8hj>wI6zPWQXE^I;c3i-OvkPX%HiM;4#R&L3s@ z+iAvKV2q)1Sn{SzIyvb9;k+D@gR*w+3lTl0KgC2MZ1%|ALDw<;$^w0)3R)7u(N(Yh z9w4`1AZgl+P%Wgr9xc%Af^G#CJefFhQ{uUQ2JVmC0MZI7Gnd?p_87IIFQ%aMVa^Xf zRP0e@pbk$-j$iXdzYMKZS`~K&>q~&UDlMu95Uv(JYcy&+S)_#5k#$=^ zZe`9q6Q1>mlXU&RuJ+Wb2W;J_$F9Ad-n8JG{F{F>b>gP~%6DoI`6F0e9`PN%Id_MA zxQ!YPL^)}KSNqXzGY}z|D(b#0rjCRDp#`SS#1w4+c%A^k4Nbr;2p@Oy!Ey%aK-6)P zp|gIA#q(le!h1k?6O}tJHk^aGZ1D7eMcY(bx70R0CuzZz< zt8y`*e%Q1^>zO|ISAN5(fqXgyiCNQre_)sst+o6yzW2|Nr81E{j6+^rd%Qc)yB+QNPIR3&v0n6c5bnSwOULNEtko8SWshQv{{?aTmRln2v;ji#?{Su($zWRQxO3Im76Qidijj!0)M%)lIt0tW!0R0 zKQq#?)58Y5>Z@yA-;taJuj+|i(GT?h%tQ%sk2Vea00*SsXt7l9tCF7Z*GERR#)t#sLb??Uf-K)=p-O9 z(yh4cfRv3PxnF_?kTJW-x2#(HROTD!2i~`7ljj0pa<2mcIHVG2j|q%ezr^^G`||!` z5NVbAjHxrKn6!XmvCDPj)5k?f64RqP3V06&C22{rsF_C3s>>^ZbZH+p;%CvYk@-9GzRKM4$Iw2onMB#SfW?F zmnKk8RY_X|I~eIK4BVcW(vvUgQUdT7GaKpNVtg6Gu79xwMC~}qx(;&7Xk8F$?FCMZ z6kgK>JpEiR-6o$W2n*zOJOPgwl+BXXtvPNsvh672gP>utoJ~%R8(8=-G#`xKe`fWI zcW8rdO>>Kxu_yH2tck4G?XRYqy9uK~(VY|{x6uiO?5^gxhbEJO>o4>|{?~z7LyObR zP5U>9ACcMB6iin^1(JFo3rlpoQaGs->4af8h1iv~=1Wr?$T-eV9&!RRK6rKjIUy65 zgd)oVvBbw@IoWLB)u5bn5n~mn^R_Q#sF@w$cDq7oP&5EfSgeM|%-}>(_;F;JBp4(_ z5At<~U|^#(Zxn~6SDVfFYa}Sh!^#0{H&A=P-iI zX&*eAT5gB8PuqD=p_H;%cCp7;f`y#$&D~!w|94#KgQd0uLHpJ1^BxyWZ21O&?-ag8ZY0!<7wMy`vfD-;S{;&$dIga1} z(m)2)9hXqz^-s&y|tJ<%js9uDM zFQmq9+&y40PNyWEhQD(Y8yDI451c*kPJ*SGc8#gJmk}nPD_dqG>*O46y@D(mNLkj7 zNMmQlE&Xf8bSE5Gsb@tJUH$9+->WclSWINs(Hzy0CK?#=N@={%5qmt6lI zCD4AontEG?Vcof6A~{6v12)qwXjuu;dyOXknWBdMQPC&2y5(U;(q1Tvv}U(1 z8#jC1L!)oDy!lgQbt5#Yo3JPzMX678nNk8wt0*%^v1-JwI9NpVQBc zz7rlHx!YsV)mS{d1~gEliVIh7&YkV1B(B^RjO2%#%(OqqEtHLdR#l5IEP)@szm)-U zutcQGm2&|b>ifoaV8F~Vr1hus#9fJq^gkBVF0&Voia=LviXhCyecljWX4SKxrijCJ1j?aP%UUit^#NUDH_~D$3nJf81a0JPQ zb6fzXpjI5c*x{Nc8J^^r>#DF}-f}}zO~9mKKL9lya%O7~y}W03{Q1-57StLP{#Vd@ zTfNavyaK>W1|Yyr-Jj!N#zqbCNGswncaRxZBv1bd@!~rHxj1B8UqDQ<48)kLhPd);=f@dgC*zRP|30!xhR zPjulX)c&u_5+gl{gS|&gg{!52^<`+>17=*XrG+J(#pI!V4B~0O1M%%O)F8&8J`97O zKy0j53zRi0tjhFKjLt3bFP2Y*s#7%Z%axOJeVu3ExcTlQ{h0#`U|rlw3(5?)%Z*y) z@ps4#(S)zqWi0d-`YhUhSQDe);=e!K0cB-Napi2YZhS`(pOiPKNx zmRnv&BgnDWB%lX%xw@29M)0tI-l)fz!dE8zSlE}aop!ow`L)^i|81=72paF&j6Dh2 zDyi|F^)EAWfFkzg;E)N^Rw9%9?5XJmoM8tSiWzIclR@p2*!eKNJfY7C_2Y!F=j zUhbEnEC*1(wD&=z)&uAXWkAbiLU7urqBC&1FHYcQR7OFngV4iCPhE%8=ywxU3Hl?q zh8SUbyTb5tVz0}hhhD49xTYRDlO)devBxI>|4(;|;$_CW6J`887? z;Eb|N_ke?SUawp4h%%DQ6*5KyhD@j~InBQJc8mg93V`uWx4SsiTTN7%SmTWvrDS4d z!f>F02foioLtqn&_}eL@Dw^$z5(9qzAP))O`q0v{*x3|xTza?M4Nbb)zsZ-RHs|Lu z2yG&d#r|3HrL0i}KU6dU3;zwg*m4@qt2rL7g8GK}79pasirl;Ac_izj^^*tO9?y&b zfx#(ZI5xB1HQiD_iwvVCu2WXl=y6Ql_{%Fxe&o`cF-&r$2p1AOAa+WJ8W!BbB53}8 zx8>bYvvV(AV=FK}$oZ^CQ!|{N0tIt>z#U6i`U{1;oJRGsaG51CDN@5Uwda-| zV-uqN>hW{U2c>s+YuL{1r*HN-UrIH-98G0Yq#yexE%RD}t!0 zq%Ze*T;CttRy7@Mntlmh%83D253-8 zA_276+remyFCzE`we#mYfmvZ=f@dqxXYmRm1ZAAA;; z0QICd0V4T74NnCwB>F!q&VFKiD6D7{%S|39=WWq?LGBlB zMD$_m0@_HgWudCTxxxuUOx=2@jq6F!l1AokOOq1Db@P*PO(QZs)R$|3SrHKfkR5Fdz2u)#YDX&m6ShP^39^VGQ-bu?Sgzuod7t18d*E z3K63Rhi^C74>2wbLa~yZH7^P;G?-wjCV>~(`+wdeDuSo2EMtph11RA51fDzR$6r(A z82{mOTwS)C>sVSR>VF(2-cF#9RF72RM*TeZ!#qR+6JeqNy*&|xqsAYOXR}{7ylb$K znfORQuCw^yQrbs~1j+XKU{%)*ixKtW|+!0b%e#|07jDLxoi=!EMfn zd#`?@i2~o@#vG0rke&8Oc)3lEM28Tm`{Lx0*!g5K58?$S&7 zC{pa^amB&GuQCQGuG`%rBjzG9ji%8lR~6=+UpT&s-Ph8Us~5b%Z`&ZuaARd*pC`b9 zqA^Ei65l!tA2fbH9A%k+x<~$kS>ulMUJ_{4(SLj(S|&N0B52H$%s*m*=KEy%7)ktU zR8qr)qIwO9D+Ea}dnID)g_J9X8WaeAz%V=ue#A15mCAES47PVk%V){wfIRzl`SG8!B?>ryHU|i22n`%jNs$AeN%N^58N$_(XmkU%Dq9uA%ezeja>6Cm`bZsM` zY!Xzf<`maQ6>d$`6Ip*h-?(Pr%$mZ%%>fi;>L?_eJTQ!b2#1MykOb(wW6-}#!3oHf5X>z452Y_^}1N6*WY zpj_k`{#Jo#?xTtPusuc1{`^#g$gdwrBi`_T@*aA5&d2BFMK48#iRBC)k8=f94`S~Z zI{p-$5@R{`fd78v&M7zpyXsDYbe$aV>FJCzilagu^kc~5fl9U0gy?&fUv_|%f<$Bi znsx1rACSy~-WN)`Cjj3J3p*KT{IVCnmh_@(@HQLGyd_x#W5rF%AV~1Xe2nrVq zyA4rc0-t|r<4M2?oJ{1e&KvQxL#>{2E`vPJRv^5nn*Y~^PK+TFzSe3v6RNrI{cuT5 zs30PU&QKP3K1>mKvzlvO4&kqY07_QKb6a&!O#`UaI5b_B^!|H-v;>;!|MsQ)oaXAi z7=(I?69JW@dU{s{M5-fKGC^HUFMrF$J+OXXq{~tWsv{?5QS#wzQI)%TNLCA@Jxc zH=qAqM7lCfzvjPMp7Swj*$>P*$R8O*RgczJje5f-gJKy_F0I8@tLLrGKU z-I5jw-OLt<*d;^<7g}Heh4Teyn~pJ#p(^V)dH1tINz3Vnv3i7|ND5VkcjPC)ZQDI;5SlQINPxnW3Rl`z8Z8Q=lw=TFEmr zP(`znmQ;(A0J}Ml(_ZnyE#4vdOA=4QGPyFt&Ov&Jc-f65RLsWK)B-&I8H+8p6Y8DE z)Y+ipLuq}#JkZd#MOTfyEZYAjsD4HE@2|mx9!oo|K{mFF(aOF?r$$usHUVbwDL7FhVP4&a3 z(iwpit~s7i>M#`pFjBmUMUbrOOEJn#lT7$d@+=jA=tuR>jelbz!ZO|rjHpH zmZxWWqFtp?HsQhOFD#NzsXx}xj{}vphKirGU#JAtLF=<|NI^Y z?D%Lrd~Ddp@+%f-`j~L%Sn7g5&9p(1d?X}A!1G}%>Kq!7y?Hx|CcMJnZOEgVGtq=N zN=uP!WytZli1F{c)$e}L-kapLfBz+DCKCr9#~w>=PHx`+`?1bXEa|J-fL(II51uJA zxy2_{)5AwSP5bifcQ$q19}2;&rylx57Ltt#!R=G0XV8@I?>);>$2bNF7=`9LH_Bjk z-_F^_VTt!IkGHEmC(qLQTWdFno#6%Fqg;S(^9>eilfjIrAI(z(!mn*9A57Jsg+r9N znTFWk*gUGU;)&&qql+<@dj!a@lp0pA2K4YGHI{k> zet?ml?F)U^l?zlTIdd*RwoVqMsJ}oZc@p9Ql9M3Y=XNF~q7hlXlSed^y2pB!rYCnb zxezlXK9(mmTA>X~FqZ&50`>JrQT#ZdzG78;`fd7##GM8 zM>p<8h(k7(&;O}wVUgQvf6*k+CI2P?If@Fuz+*g7Vfj*x5Z=`5{z#a@bnE-_ZeyRPBZ@$aXm~YxM;hCK{ zfHZ1(s>)x$;!#)UCrKv!B`ALElVkJ8QpR^u5(Aj5|NRYje(oJo!IxY4-Z|8d417|Q z6t7fFzooZ^z57R4wQ7?ePA6q&04sxk3Uzo-zQbfdV$hzGM_DBnFXr+%{Npj+IZNi z6+{lxvdjJZnUW1euSU=S$+H3ZHNVPfabeuNGgx|9$$v8dZrnd4U%n3ta`ZZWVx(yh zO@z0LT>&h+y7Z3#qGGs!gRnP6;C0HE3S$-u`W#Y#yx*?IC0%LlgWx;E!am0!FSB0{=Mf zpM{+#8=JhTo<45?((PTVAjG+ohmugGBfy67ZKmqolG_r=zk$jUgoQp*H(6?Lc#3U! zBu7y4!?#hr$m@Cqj43C-G3v&U00)K5AU=) zQ+c~PT*?~_0^Fh{k`N((>n-giFQmpcM%Pe~9L4puW8d%yL=?#K*%_6N6d5F8Q${H0 z;E{JLZ*;hi|9}))z|MLIuhgBcV}=;cOEBRDI&y$YVNj~w1H;tLeYtNv=Qj0uLw60v ze9cCt1qZ?K>67LKKbP)`oDYd4m#O*eSUH1s?R{t?h4p&X4bK7}y4lw(W^*S< zw%{uoX)Y=LAVU#7AirpSM3FH~z8nh*WG-F|lB;KoK?1pSEUInLa(1FZ@Qw^K{uQKT z3-mAye>KKvyOKBif|ObF*dz}^%8FRT-Bv6!Gn<(nP~clocdi*9U6nu zvTY#$Lk*p1iyW0?9?$yMe|>NPBAL0_dMGlda;~E+^9zDKbRVY&dZS!7;ajyt&Aj2LBDmo)QGR)jcko&s)zDSf_dU^dEa}j0*)-?6nGX6IE7Mmyh0#Mjt*-bdFMQ=p=muh7yNwmNg0}IYz!Qk!1G2dIS`? z1=HQr)3=ZqdYK|;s_KgL4QwDP#aOyFo{VH@04ne9ATc*Muss~;16DN;9R!ES`t&!H z!Jc7Yi5nUy#Dmezugv(asKR(TW1FtsRueVmVU3r=LMwVL`VhRAvj(}ni2pX>(fv1C zzjx=B_dm)Z{|{dMez#BjH9)q8yGqr7GD#X@nV~I|vOcpuE&+|3;kZG8ju8AEMd0SX zOesfSXhqEWMngMpTKYh@S=ZG+$f?jsWFmx0T@%P9x*z^=nv^>s2M>0_aPKQL*t{-7 zws|x`Sa>A5GX~?^Yj=GVGb}{8Tr-jj%%8O!@44PUR&E7Nv{X%y4O;#3wZgSZKU%}j z-<^g+pD2I?YxF?Q%1xU_F4!RwM_VIvg%4&XWlZv6@S3GPuN1i8>nO^K2|4>w=f|n1 z3QxmP?Pl0-n)U&S>#xL7C$6Q}U5g)aU0`et;wKx`ria%f|K)}`HX*8rhdRBTa#40g z6RjGf7PHaFUOBP?7zbH5(Qv59#PtV9W8Le%`|RyQ=G;^&a=iZEuD|;CnYH)d>yvMA!HQwBcXgHO=DqDO>NN>P51#)(g7c z>aAaRGX27tc9>?cY%;1||B4(&dR21f6AiJbd$n&-!489@xXPY^1t9e^re~ia2EMgy z1DuNbNr~z;#@JyQWk?+IWfX+Hky;vi;avepju}7E#0Y*}plYlfbzgYUiy7WE0ptF^ zm`E9+?>G9C5T32KwR~oHEjQwm?#k68TDlpQ9{fb-I~Ksw2Cj(s65l~66M>0 zK{WdlZbE4~AMB2@Z0prq4;@mf$9GSPa8T58C0@wXDH5c(G(Bx4w`u{^*#9^2*~y|# zruioSwPgoL?}t^VM)dK3N5i!13#j2ZsnB{<_ZL>);~*bQoCG7uI<34 z2P!iraNQg#7BkISqi48%WUDwDf+{%sI2DR?5b+4we?8_m>MChT86>B)^o8cS@!87Z;^%- z3nMHTT!}?+se+QGF)U%*kXBG)zMv4QeN;SyB>j#QCQ;5et!p<43&IVuu&^vNyw0~- z@!rj9D9#p1FHG%W+3g2$wPq1?!_qZ8!{ z4vE*9QfjvTyh3OU3Ruz}hy!2s+7;s|t~eQ)@ivJ_pPNE#806R;xj}JxG{#8ZBqXv~ z>#0B}KA>h!7XG+~tey7-cF+4PoNPF_BQk0Aun+^O==|I0#Znsh4vOOi_h0c1`-%DP zdOVdZwY6^oph{6>9A?6|<7}U~Q^!MuuUW?L-9v(IAiOsp%uT1x$sX)B$MGN-Srwjx z#tXcuz;@I6W-TP0q+$2v*T6wru8o=c{?(uW$hR4a=3dkcO8wF@JyO3Uc8zVR22xp* zeQF-1v4w__#^11n%v4V9mYj95zaik5ExM)%UKqa1=R8Gmm>omb~f%{oJG%Jcc# zP4U0Y?j-|#_a?8os+@?(JzzV2_|Xs15$i7bA7#+_@h@1X z$1dhoZ^?v)3?oZA2U6Y$2UVr45b7G>c|NT`xy;>KM=WCuJ_}1Ij+ik3FRHFdLXPY}020ruwY5f2h(POa z6UCRUN7~ncWvgrCpqH=CvSr5T`69wcauaqxUPR2l25Q@Ew}hm8F8{Lj%WQsnR6N*R z9Z$~`8P@fqN{2$T`E}w&tqr{;)ha{uYFqKgp;^UZ!x<)$o%TnCte3t{zJppHLD~V+?bc?yVx(p6HBWlah__hZzDBD*SXrN2(!nUF zfAs+>yoCCqC~AFr2!}v%&Hh*z2VSLFJ(D@T7;sr2*Q#Kj`Ls?j9F4<7IA zG111{?=BZ+#{-6o(h1e2WOdy-iojAI^cR*ezRUr?J)zZ|sE63sfY*%X22>{(|9cHf@No@Ry9niemf9F0R70!RQR@^4LAk<-Klq34 zX!LUGghGrFed%UNZBi8-Bu|)WWPu($kM^-u0uSp*ltiXO6ror3yUr6$R{5|iGZb!3 z`_;_NOnG9{>y_Goo?~{U72ybI{hyf-#kM*P_h<;XTzsAAFYQu~NuW=d9QCv=U;zW6 zKCzvPVw;O5A77EX3c`=TUNEiiRlrVV6k2wK0$TYIdD%Hz%aGx(H`6c6a^y*pipV8> zuyNm;F&geQ5PsPZV@nz1x=!8mV$8na*T+2?BJ@!npBW|S4;mBKc2Ojc#OOPRdxwu} z*>AhCY+f6txMo4yAw7J6T28Um6PZ?!iDRyO*#c>*o}ap~fA2e0I4?+PR=o=HhQaza z#?aZ~2UR+8eVh-o#VAv)x~IUK#B}Ug#nts#?>L=)_LSg)b#0WpQRx1X-N_s2(_JEv z+=#sX8#?>MKQzQGjzSGaqv*bp6kBEzLcP7o1krcWtOV!WEiJ)#?y3ZaWJdI}1KP5! z+t9LN#v`h6ckeSf3=7M{b@u`JEl$40_~Zak<%PH{y$Zzw*L2xanV-`;otbtk=r`GP z!%-mX*eZyQi8g0IZjd4GT60vHh-S=h)2pJ)zr+qlw#beqC#w~`#}aNWb{;U>=A2Ac z&q+ntyi5<_6om6wFbuzZOn0!!e3Y7);yX3j4h5Mkv_i1m1V#-YZwEqGw0DkKC#jyX zmsiMYrh1TBu-1)9MhyNQIjp~>A5JvG5D>=w9b+yMa<{dC(vc zHb)lU;-bLO0@RyeO8@d%y}qdi8f?%-KDSfN=+RX)`0K=KGCcPCi=c`Z$a2!YOLsi!5OmW#8) zk>TJ@|Cja65+<+G;!CW(V|lT!BS-12;__KNg}L5mJwFb_8rK-(+mhIvjLw`9EFxHz zZC||znNA?DSaWmEv#@QOwTyKs#;6$_-_}oefQUcd=v^-Fg*z^PdZAYxG}6d+s78;C zQ*yT1#jKEQ2?TP%IvOYhMEu1`x=ifKxmulRwmdp_mJH;-?Xm1?$o9J@E|zPBZe&?s zpGCPCv%KcPy`BO$cdysgPHSa?OQE4JQk-y$cebi6B-wBKonG9pJ+ON6?L~jO$H8I= z<&mm$>4@|hz?K6fLQ`5o8glcyze)R!qX;^VjDeEk5oUghzLdo-@4VoL0&bHWI zn4;f}YSv}{r2nrY@c8^!|Dfo%)z#JPQM!hhTyPlk#Pn?TnW-z#a)_#sIOZGuh#STx z6vK;BO8cQRxaD)>`X3yh-36O|D|yZ98>v20X*8*o$rL8E6Zh6{jsyZF7nN@wm&QAy z)l9dC%yM-vQ(vb|?l~ih$}--|JzXDNG#t)=7wOn|n4y181F@iP$-*~K!d3~|1D@00 z$j69|h(&oJZHq~_&x!bJ~S~8B)R~yr(Ew95U+)}n7o=>l)d2a z{EUB$>d&)<=ffn`#Pz+JvO$hPf~sWOo=iJKHp$QOP*>g;u=Fnf8-Y$fZMJTGLmkC^ z<2)No9s0NX+#ECBn#NI&281iyWhg=0FZgg)W09YX)gMG4`Ew}c(3Eq2k39ZoCb<1)@34VfDdGJc zm#u{UTWvCW!PZen>^2rrIPdo*+-8OvhznN!QbS*Ev*&=F_Jt97-;99q)o6AuxLENZ zrDh%@%|yD15=IC$k-66+lq@EkV1N>X@HHz7(b~@mntA4DfVl;9KVOUY2~dz@m4?*6 zuj$jn%?!Y@2Fkz~URnZGl2Fl%{Yzne>*6z0;#K%^=Xd&4yn6X1XtCnvrZ&_tgb)nf zxor0MfuGEzKR-3hcyz2zU1Ju_a!eG27h=4eltdGD2OH#|z?UVTea11v+89-9Yb?;< znnj{YI@=Cb$DemRz~4IlU}N#s|3DRLU*gK-)#cK_M}}2#{Ksp&6dfziN}8cytx9e7 z6C7nR_Cx!=`f~H7ze+WvY5lVU!sjr1FhfB&8!Zq}{WSa(TMFP;fBP~+D5%3U!NdNnRk*L;=)7a};@;5X5o>W(`%jiS zfU+;47OB3F6tU8XNZi%giwGAsytP~JZNz%KToY=wdY1wASvu)H?_-d@FlX~^UPbJC z=MUrZxt7k)7YO)3lj>?o;LO`y?xC48`0ni&+v{#QHV=4wIfwnxnv;qOwMi`QLQ?d+|Yzdp*tSbf5Wu8 zj>J@%NQ2Se4{u}Pxa?}jI?AVEZSMX9Z1P@2DRv`1uzZ1K{vZ?VeIsv;MRr9)spak1 zWkckW7d+XzSo=x0BM|56dWYA|u>yKj#jB)lPLhWvNek&;)!?zHXCMG%$ld% z7^g_8W7NK_>hd_y;#4h!dzE>V@(-sh`b3HM$*;_sK7U$66_!VR4WZK3EV7+4k*W@) zVnBK$i$X%Ic6_mfyI2BYa{E>#FUZ6U_m5=XwKhK0tZkqXt>Ey-E0wPnlaaJWLoq6$vpNV7@uJp*<=2hD5+N9_PwNTqR zq>Ee)d;RYo@HXDzvT3nz?WcFBY?;tOh1@8fx-b%US@Xw{5UMo|+w_9BKC;4^BS(sT zL^3);6a4zm-VtwHx8J7&XvQ}Axq*?Ez(_^m??76^xr;+d`}g!3cUM~dyE8jC8G|T^ z#7IKAQ`4=1=l6j>upQ2BF4^Bi@wEtHk`2_g{Ky&31)$2LAjibc#lKXyf)i2_;>5l_ z$h>skVY-A|4LC{w-pE`^LOaNkg#J7&xS=V|@;f4Z_pY4DD8{fZYZ4eq07CY>wE4u* zZCXRB&K8<<6$)3?zY${%b}zr~1pR09=&P*FnLr>6(b@Mf12J+k>5$=KG~M zV03s(zOOBqlj+i%Sr-9gv#Y4y8;>7O*J^}f`F6_hKPprDXvzlV|UDCLXYM`y-HH9qwVoKKj!=em_ilrVC=8FFnMs|)zdNh&tr25#v*X)dAN=W*nk%%dSusgIG1 zEJ#F2h~``eL0jT3)q{#HOG9Fy&E_aT!8L8e8eSE9ALcdbVH!aDpSOtv zG6X~jFDQoMXd)+-7$T2H0Ajbf>>-{+$2eR}F7FWMKYAAExz;{73PsP)=GGpcWU9Pk z@Ih%WHR+dLU(EhAzvD*J<=|Vr$mV{dI?U&Lwmxu@@{|la)58G`6Isp33cH8e{ON!Q zYOkKx@}>|x+sZdRI;5`8P8L(}Nt9lXFU2^JDolux%~m4yrQUBy+k* z34$;-DHX}3-^J7+5QT~&5*vDo?K_uxk(sX9rwwvG3Vh=yd*X3bHDX|?siA-Qv%bhf zpy3%=Y+M03{uO!IVbM1!APNF8(#y*;ps3Ssw7`2_z%;YkNdoM3LU_gr%W;HFMD8C` z4pYb9W-}O@IQx7N%>TqB4>IRMdYHPY_1njVYBdr7SsxQzi^=DXGOkg}3;$_hId{nY zvE^Clz5hWg5l4H$%ne_b!XBu^K6xQ3sm*AJwmt>w;;B|;B<6(-+&UdQqh;OL1IcIc z<5%;~oVy0j3(CtBxHZz~7pka}R)_sft|g8;|o*d;bK~1x5aa{3JS7i1a4CEyz^n>6mT-n=p|H z*9qfmV8dGIOduXO#|w65w6B;@^&G3X4?>3`2933v=IX^ff+8~>>ZI?WTSgA&5JoN{$=A$>mg~>yPG;KC!xZcN7u{8?kc}Hc5BNr2E72+}S|D0wfx|*K zjNM+EMPi1lx&Q*c(z1<){SBaKo$+28r}paZ87L=Lm+0k>Q+Pk=w59*iWDjj|PuZp- z@s_diZmP1B?$GJ0WYtyj>E>670EN$cAR<Fdg+|IyAq@= zrz1UFuf&>{`>4tLn!M;frG6Q^gh`oV`eBZI!A9qEdDWZhyJ@;zBaw3TbD(I(mR0Tf zDlIS%R*&;IU}tHlw<$Q(vRSO?;K+QxQH<}Zb;)y$NH{JpQS3Hqum1M`SpYv@X1r78 zQFwrco%%b-P$Mr>hi9(Sw722An2qmHK&n9ON^{Kn22IrRRn@C0KQ0v$daTbsxQ%6ue*(lmX(G- zCBIze6scKX64P94dc$d4yV|q&=z?2og*>HqOlNngKz7Q4XK}>9$MF(Lg9Sga3#BWr z2IQ{;=Z>>KNbHq2&rq1=GQF6^J~>P~C>wUQBQJ3+m-jH1ww05-%e?xUul$T7ZO)t; z80crd0-iLVGORsd#z)Q8IiLr`<%3bKPW^kg|4d>}<*-&urF8RUuUW?!n7v2hFD7r5 zr>6pbBiVGbyU_5v%d2ERn;?8zFj&_t^jiP(+^^%7RyZ@Br^&8PnkvAOc+*0YyM5R*E3S z0;p6e0Rm(d6f2;ps7Mi1RFoo0hXm=Cfbb(ANr-ebB%w&C$wlw??&jva%uZ%^-}lPC zw%_bmUK^!?yFlbs67ln+9X*}7YYPPdotCkyiyb|7xt0m@3=Y4>;dD=@+8Vb{&0=YK zq%bWvae4Cd$WT0YYGlYFVg5%UzXiJ^!>)>z(Uz8Lu{2ullE~{|S>opjYkkCo`O!WX zewoCX48r27j_S;m24$+pju<;XUl71!50s}_aYxJf6h;RMJ16~ecfO0*x%M+9Ew?+H zyD)?9RJCZ?N0^=LbID1VpZr|tqDokp>T$`nSYh;Zs#+!Nt<14VSZ0VY=RUXPau+yh zmaB|yE~*I&oLmbo+ch0E#8Ssfn=6I#m<6jL$=B z<5O*XLFWewT&$K?o6=MhRwtzsxu4rQxovixmiX1PbxHiH*7Gw3X_jk~Qvqs;v&K2! z3px`ylb>0t1nv*J+;0xtK|2>!tF=#sx$&!6{J+JeHkX{$1s%)tF1f?DmaD5XL%ijg zjLsQJOZ@T_zmht4wZmk!H`g+DeyM;jPuwNqNJg5fRb$FexY~S10Kb&>zf*kc$gw1~ zh{3Pt7lN00ICk!IHpUajq!4Sl(raNtl}rkiR$W{Bw*DHXE0jGljTszVG-h5GqVrg^ zmd*+$wUNc7JyOfz`Quk;Yg1;E0fY1Qae_4?3e?~g^77UYdczMNhI1!AAnwEu9~P~e zkfItJ8qxSJ`xBvZ>RXT4G$otXKd&6Ou}-tTzIKsAmRy@B&nMM-*7jsi4&Wy3dU9*( zqo!gzbMRl;UA0+5L7}uQ+-`i2vITykCxdEEAVuPrJ--m&@q&5=?5n z)~)H01-R`cl~E$}Y(*GThD6nc;fi6DYEd?G9cj9wXB|nX)(v1j1Ssk@Lt<<)RS}>q z`5>zB^7I7+2~Pk>_zbmpRaS`42IW3PRe~g&$^SxrBUh+SJPZ%=7v2vw3DrjselW>j z*7&3g>)$5G)0rP&I3~l3zXyW~K{71h>;zdJ!gazz2x_Eh3DPvR)=HdhpkjkjhB#yx ze{da!D-mJ?YjfOFP%UoC57X20Ih^~)I5TrlPsqayDqMV8&`x)O_)Hj+5kycV;T2&x zGC=bYXfBipL>7Y-{cN|yktDP@RRqw8;_OBoEFLGow$wU6 znc^Y=O1)^95UjEf#tMcp8{llRWFn5!m>t2dl;sVuna7s1Dk@ksEr`RbgfZ)WEqJR@ zAmIYjXIzXP&GA2Q>Jae-*$yy5>^jtNs}Qb1nQmBUIeHYvN;D+laV4<2mCev$c_|XU zPp~S2wE|;O_$w#z5@9Qb@TvqSh|>ws+6p5$XmFAICGydLQ^_l~M#vC?4Q$Z^nkVY! zmOLxZ@U=YGGFhox(I+xTiXZ2 zvDQw(SoOke+L!gTdKP?{_8p8=%4{^)UMu zDjvz+!m*cGz8wr!IY~5;`HgV)LI9-!3|kQbEc=(n1nFdp<{1-cFaXAKlw_6wG=axv zBzy@xhSvyZbIq~uTY74%dD*a`p~$8Hm|6(6&>NZC*Qrd$cven2goY%N`1?yBE0|0g zk$7!(nY;^D$7z%V;&Bu8*(MMY#wM?)CkY|dUIN@tzr5Zz9t5-q)8lM`U;Yj z3s4e~S2k?*7Oa~45CI{zAq=wexiLR>xcm}NKd&H9Cc@$g6ErxRif4uT!0K?wmKFmQ zEB+T=_zg1>Rf1HGCW^8fYYkx!H?DqJbAvrTB!)Bq>Nq0;{B1ODUm!H6e)}@`BjP-N zM*4UoH5ytd zv8Y39urxHpVlw&rZgfui&CmA zeto}y7mXEajF5Vs!5J9eua19fq+*aj7$;%pWf*ssCPa)=7K`QK@SR{?tTg!{x(Y8ca-NH-auqd@IDVtL%*=AVyAg`p}$X4$coro~QF5USW5+?7M~oxcBNP)h;s#7P+Y@d?3>n;m9B#`=NyxIB_EkaQjF!_3G|G2F zb!x^1DAX1Pa%HU1rLOa-=uX{+2}hD%*)I2W=qFGt4170fd}`3MOD`fddc(Wf{;&*@ z#XZz!A)qN)Y*G|3L5JOk4#0J3^iOy#zj$X+o;bKCMCug6)67Hix|L1be7?B1=(%{@ zwCo_EBR+k{V-LYEPk=fd#=YI__$$YZfRWWKm1YI~!1#C(+Ht8;VTwi_@!*6wW+S`% zY4}F?^+#C~6Ojjj0q4u+Q_jN1p>=|e1rfrb%RA-4Wk)QFF{Uo8q;f)vZ5N_wJ*;vO zBAx@TO+Lhyt65=5n?7zdypccLo8QLJvNQq2zK_{@!@WqWeGGAMB3xQ1T%ir+s~Uj% zIrkaap!kT#h}R^WUwoS@RwQi5Yu5>~zt${5%`_4K=9&u=H`tw5Q`Etz9uv9uuHe)O zSf-eL{fXl(CF@=u_LE(=!y!G_JDcR(fH#E!K1puw2TTCj9}k;Nqh+Ox=KH@Y9Zja0 z?993+Hfe5{Jf7MH7sryo!$?u&?Nn=0-$wm9lTWjYO_s3=PSucu_~20Jd0d>!eW-Li zO>djU79hdh;3YlR#}~fk5j3o!)S6tOgyEv1dk7d`t5LHZZ4!WhcAk%jyC#Yrn(8fW zheEk4j-&EaE)|Y?uJ`xCdJ0e48WN8(MZVk!_|R!hnv4_&weygmf?@bx*>oYeqm6;@ zr0vw4XFFnNjoR}$aqyj}B>bc0xr(vPw~Ugt#!+E7Ty`t`;D0-GuX;|PbD`K38!#P$neAT}m4=^4)>`4v- z{Py2H^<2&zj zr&&MMHmKGxG4iBcE&Y_zW;*ML!7HF}L+O3N`=i^4f;J6>cJ5=wzz9Qr50>Td5FNl0 zAK9sm3L;ysK7*(!nQvr_{6MaKnj^OJQi5Y*jOKenfUOO-zeE4w;$bO6j)gi3a}#;G z>8nvvnQQnCJCyHkAAxA;8_Kmr)kfjcf6D3^T8paiaAD9N4%Z8sh`DsLAv540b{KJU z#Iv->=_t78qB6Shg35#WgMIn2z_2iIqI1Q+{y;@$ShVc9yvbVHBt{)tLrz?G4-=PK-(D;MWZ}cvPg`kBewq#&c!%LGLVjT z*7AM!VV5hf=v>mF@Jj9iQr-9a_yMV=qV?K_g>1QOiUx!CG?6~xF(Ye`NaTuk#c@w20D02+)fT&Uu1Z=WD9oXVWhHFN{ zS+5V0(jjOy&r3MXM_BmBY}d4?{lnX7>|3OY?0{__ssB`HymQ&tEl!J7uM-$B+rcJ( z1LJJ4rDXFqFRYS~LRX>M>wUWYC*}_wQ+>T&lRipeY)+P$^%85(f0;&dRlNMzW9Ims z6#spIi{;Bs{>Z|P{etITUvtjCEef1YgTl3ZZ@+=-;b%K4-DX=$#74mJQip2^+IeGf zcY`b8kyTK#SfCsDwe*2sN`1k@)|<-iA^NEE3Fiz@S}5Q5h?7FnhY~FxE2JM2Uw0dP zT`*|0UkW^^?JLJdOC?9Q77NRuqp&1C{4kjF%{I*B-s}cp;1NGR?%D@fxMiliyeqwb z)s-@h>NZa_$kWZ+w_GM0%NQClv8%ZE4t`Q>GvegcJ(4jfxL&71dIhw4VRyxG^}E5b zu2SU@Eg|hLTaBWs7$L&Am7HF=^jXX7*=3ja1|N=3BnH!-${3BKH|uRKzB)Ebg6DQ3 zua@5>_Ojw zzBdG!AU*UoP%E#tb2Gx{a4n}jdeh5f(r+au@6Ck2(FK;)3yyzN#YhK>1inQ)uB*^b z$#y;=_WCP0+jG|P&|{e{U4&)=*nJdR*wE!N-!E6nSTX_DEP;%>kfvz=!2DB)LCN_c z3Q#|T+_d)+@VceY7AJI`7DDqabX-J|C^i}>vI)Pr#LW=#62lUd=zld@pb=zy4zFLRV9<2TU!Eu ztPh+nfn43N?+yI*5%bM=jPhNMMoWOR2)540q-$W!LF_w3aG(sLZ2aO9hy!k%7e^YPee?6p;33TozbYlzDZ$wdpGQs zTqr*iFW(ht6yYYHDX%O!E#Fn+iuFY2;dBB0el89nSHH6cI{85|WGEN%w!@x#1-%(~ zsqRZvS=)`>VuRi1Xn;65$UcZ&c=X~DbV_K|nr2Vp9jtEDUL?5)WNlkLOzAYS!IqGR z(Qr*!QFl%z(t*0%joaXc`J-yks@^)aTPoM!bPCijW`Ng>VVqmx7Ce+k>`@zmRXLy~~Nzh-h6r(G; zT^5(1eKH+#UGkCMf4pPrEtGvxAC(vA)S`iYHG#>5c*Pe2Y~t3b=dD`p#^P0h7GWpd zyn1IKm$~HFE)E`)V~>ND&G)7F%=$Km&hW}b?NzNwghP!c>hv ze;t1W?RF;a;`3uW+GC{RPD_5>cA)WA&pzNzw3Uu8bx9u{M8%^)FeNL&Vi4XHjhgS? z@Fq0>f>vRV>e_zNk}=0~I_cr9keWX1;Q6k75nT;eMDIRq2myGuV+yYKl$GG*nI4U` z$ewz^uae2$50Vy~4)X;b6bWox^Kq^h+;D z+?ms(P=fDJvl=o(^p429St<$Oix++)aLZ9wZ2x=+Zf@=HIN2ySI$PlSp}L$mt9Dtu zxbeIM*}lu#1(zV+%Pp1Ak+n>$&@au%Z3YlfwE>(uRbYv*Lt9*3~~#ucFQm@_p)>dPrt3-Jh(9G z8}#YRJaP_k5BHh}PL*i>f`FQso*N)_OyjG_V70th) ze2P%=hYy>tTiqm%>95Sl=%3E!R&LAO=hrgXeVcC=7JpkUz5qK@QdTG%bkO-TF&cO>oLO7yeU9^1C3*C*mmG*qL~E;H>Vb2m2Jy1Bw4*e&$*C+b}#|Afw`78;^e!Vk79gNe`LodBm(x5@_G z4pm+{JzJ!JXl*cI{r1R*NOjbHk*aUJ4({;B-vJg*D~CLjhDMX>_L8Ws+=PQT>-{9# zuIAV@Fr*_oVD=2}WXQ$icc9_%hB!L)n3P28 zqBS3ox;a4~$Df$B=hvkj=Kj<8HGK|eT1~VH&JS#}K#3~igoSVGzmjY|eEV(|o`Xbq zfxqRVyXvu133?#|nyy!W#eg)M`zMI1MMW<=p8s$E{Nt{t{_@y&p8el`edOg|ve3=H zFhKl;!JmBB@E5$;zu=AiCE)H9{6(<4SHE^sg0J{Fvs%RGt^WD_F1g(*T4m-a?0V&= zZZ~g#Ci(cCz=P{`=$A^dtl?|^l|0H(&jPPR0?+18ruT}JNmhZ`<^Bs|g>Ur5xqs7&{CZqO* z2#!$x&6QTl%@uQb+%}}VwG{mKF?uMv*ROl+F5{!_dS+Z|Ir$AVh^r=Lc&Q0--!m$o zo>p=*z0E6O#C`-rhQ>gbbHSJ0{A*+5wp=l9P#4`k2R3ZXO~|K{T(A``Ln^+yz6)j( zCD$D!*_lD>BP7no1O|I8?+DS$o2#oK>c1>NQs z=Y7G%=;%@G?G?$^rwxyvQtRZ>sT8Jigv6P4R)uNvo$+Dd7dTFuKg1!F;E`|jDa<2 zS%$uyTs8Mjfm+HOFDgm5Moz3>f?3u>|EjvOo{G-(R@d;^m}usB!Ic-KaI~F25XrDTHz;y9BdTHJ$KF*rs7iy_eu_`~9 zk#wLT^5;HEP{TKkGZ|_^rXO3qb}4@mtD=NwJN*bR607pa+_UyrXd%#%qp5b@FL#qs z%hsl6cAfQxWi}QGfxAvw?`>NfFg1E_E@b6eP%ew9*%yHx-#|IhOaHE_cIABTCaLmZ z`>2?ucYB`hTI-ewe%A1XnraxFy*avfAu=$tmTCE6(x4waewMH*Br;cBrEDK?q@G+VUdK8sKl=9mT zWSi&3n(>vu&(1d%GE?^-#<|K0MpwJ3Ew>x2^U5_6q?&YtMxG|#c7S(a|Uc8)nsoenmD|#s~KKi~T zM!6X&w;{$|8*^#1rCMNocin@tVNJ{4IZi%pHxTntb@DitAkJf3RmF$TCC*g(ck~sM z@P+^9xc-ED@%nUL8fI(15ex=6I_$PSJi%BdW}dZT{-}|9+`;Zlxu5Og@!ob*uBA=- zqadLYqQOhpK;alVPOn|`=2_C(7t{6ycI2~$D)A7RUBy2a?UenVsz?{@qi~ zRM%c-;i56=bs;*;WRzQk2ev|S^(6Cb_4_8HmnXpot>+V-j95n`ueY`wdkqc$II`ex zhW42@`vM6S?3BFB&oZ7+;c)t?#%t35rkLM}Cf)D@PvAOfYs3+-M;* z3_kDye6ip4yk1&}`sD)0avrY^c3}iB3OxVl}gA9B9 z8m4vW3zI>Bfn)~ARImeY!eC?&#sryEW_Om13?xb(?u6uogoF+pIACOChYwPIfDAdKAqT-#IJB7X9=Pc%inar7iP-c0<)0r7&4-D!Jb^-dOir*BM zzj^A{^-V!R3Siv`Kh0fKf6(2fbR>?bqn&|$}olfIS$d|z!aTIQq92p+l$e|z`dAs_LO6T1v=?SP% zztQ!>VtwAsP5{#L*9YV=J<2(0tp^0}JpaMYds2$g@u)cui+y}mVEDPm(W%{@mKXhf zj^$o`-{ra1X7yu*Jfx!DQV3=Uuuag0U-Cq6ZFnW z^^BrVV(T(1D4;shoLqudQHCXoHl2|CX2tl?r4a;d03Eh=vh*|g!+Ahukf#dQf}uCB zEsV-oz$H!7bmT3cD+o-l6Y};~s#W8of_GO(5<$}N1ZQKwfc*L3y*w`~P1~jI%Okm3 zqJCQT^L1$iLn_;d6wbk}5cHh!{@HOd(k~~;*P9NnF>6&lzI|Erj`}w94)r+?TYOB$_(bSmEYx4MM;VIqmA^iGcz1X8#6{Ijr>QMdRIhp&(%UC4 z5YS8Gs8qdBZ(GDKKBO)HUo!r=gHnHf&HVMmej;GvV_0$2aVQ+z)R7-H&)Wu+Jv}>S zRPbJ``0ak*!*N3=uO;ywBQpjc?(8(OMgp4&6Z(AvV)I$4I-O=5DvuBw`*V@qu|dV? zgulZ!w@lcnu&D6;;Ba>{WnE_pE(!pFk(Ols6z>x7AHaRd6BV-y#SlXmuOPe~npS6u z&IJhQrQI1p_co?9bQwJ8e zlvRQ9ab7Z&mBNZr5+BcaL^|gSi`Z>D!ePY+0n_vl$`f0wbe;qq*O+2|cp?v4J5*L*#K0McX7Bb$~1*Bha6`QJxbGW*7BKPApS0XBR#DMRpeb9Q>WnRI! z?URL-E?t0TO?wKJ^H<#GY(Iuebx0W&mjtXGt(G+8L2THB`b=uaIb$`W_Y~lj@8Z|E zEkeMdx7O>H*v=M}ohz3FviD-vvS41Q}FXHuKd(`|^%^Roh@rbBvY$vvw#Y94arih8iY?-k8{7z|4Pu{V4dIh?xm6mJNx4RTB-IXrS5^GH{W~g9Di2e|Q6mVH-_3gEg_J5B6t%#&4)3hvwj9qOx{2v~}0cPuI&e_GT(Wh97{QA+ucUoxIX^VEW3w@85 zZW);9k`8P;KxV`U$DMb!kTY3^kKT;qWu6u*gW{XSQux?0XF&f;<06Hxcl3GvT>uzT zDRki_30u7#lRj3w6ecYu*vZK=OR>m%bjc~B~;2N$CIKwEa!8XHYB zS#E9DGq1z47T$A??P~?k*UBZA;YjJhEA4Zpq)K;y6wQV2t`NCYaq^CX#8mDE(fD&@ zcDAqVFm?7!kZ9Xv)IDsZ-oud=6ip^wqn2YO$9lIe<@psRWteys(uV{+QvCSY?GOd!`hln<0&eBBUp z`9xyqk(s%PEeRy1HNZ4~8yaFmCdC-1)$u-O$-Nv`~Jq=vNV0*M8 zA7%aHm6vw1x^=!qDz9bFY|-WYuGgX4k^MDgeToA@KYgZf@nADPlsTUSe(rf23>w*|KgG?nk8A1<$hX+t_rHF(P0|Fz0>xVD&b;Hiyzdk>9Mc})V8mjle;ySXU{P+;8G+^EG zAm-Xm5yX_>kIm2#wt|DVUigKyt$oDcJvO}$qIxxKi)P4KyQJX_HNFa&npr*^3c!8{_#; zHow^A*AoH`nE6Rw*qZNFT*VXxuC&tB;cy?55YX(^{f`k^{9H08O0|DKyIH+jte{EK^ICycuN6Vd-M@!ox%>6o>N9_05r?M1o3v1 zzl-Hw#MFZqk=vslsX&T(Bkaj$p$Hd zl%5cUZm-P7A%N!*9ba|l_p_H#-9a$zrK^W{Y@1&sIKcXMSHqbN(i-3)PF?7oDf4v7 zyAh}opR=E&GqCEJ@S(s8h$d{x5<3RL4wzBiO$w*_B|71f9b;+Mt- zn{Z-w%NU|@DOzB+ckw<$xKQnudX29ZVVIX(qjLA%L$uGJhH5R1wg(t7sA??>4D7~3 z{{8flGv>TYtIgAS6tUvEk=M5L;S3G|--0#5*Nmf5)Wv)RNr!jN?S_z3ZiHJyI%(Za z>Up6<$@(3*+0(Ma+jyte60l}Mb7l5_d6p z{xVyOm``{{%*#*Y#S;+!?ymzI77A5xEvloqpHVTjVb!OTJ4|qp%hEvt_DzSC4hTR{ z<$>Dk;%;lRchiJe+!`Ama+y2ytv^%zxQZY&xM%}1f3TE#+4mpKfAFwK@^5&FnYs?r zw}p4GHHoR`;XljsKx)3EV(dqyHbuA>UW>;=aX))SfcBsc>P*DqO1lc6YrF)&VQ;dTH;a0v}*_;=t$Du7aQpQM*R0?Q0nj(MS zeqo=qDw!Su#7ouSgz=4s#=}7UIL+u$_gBfzPFcK|Id-XiZ#Q=6bgZ%i>p7*0%AYmI z@DQtCKlWcA#X|_A5#8HEa_q?(w3)DD{Tx8NbdY$86zu?Hyqz)XqW-E+p$Qy#&hDj6 zHdBy9D21=5Vpe)o`&nt12JIfr&`5J97g$8rA9w(Az@BJ@a{c;bA_hV}$2&tvdF7mt zRC-kO`|0j24akASic5rs3r;#vO|pmn`r{x(vzZmLIcNI?E&i$Oy(7WBoyl3*X=_S- z1B%T-;4W4aY&>EOgCE?fAG)(EN89L)Tf_G$zjE86Hug#uyt90d{?J#ZJUd_gwweI> zOBj^Yl^^S=`|Q)zyqn%NS^qNdH=4fM1@YDw-P48?&rRf7?NF#YyZahZ4pM}Ii?7EgXl>+OawmkDbGD0@`1(lB{wqTN@8 zTb!e*JN9ZU?LeP!Y+SPJ*ZL&~0l9qT?OH!;8(QHFexbx?XJ;*Wt$E5=@ zDRzU<17^XJx?Kp*AXXe2mqXp5utqN5mWQzXPnF5v>X`ptwiwyV6E-GA{!`)eKM4F) z75)27JTSszTiN^C;P2C$9Gy=d zJzimWN=+sKaf|3k6PpbsGw`4_S>p{Z-|pW)fH?plKt!AwV`|m+I=J4T)w4wYx9Wfa`dX3#1V@T5Dv#{V(pWz zyK6(!ogsO`Kx-H0^OBJ9NF5yFc>w&nkww=pKPU%1%ZH)VLu=@%ET_tZRML(J)W zd^Yv-AK1$^2d(OsCpq1^XEdA>QY-~)2jsk3E1pC^Ns{US)s~Cxvd+yVo1t!XSCt{q zzGB%k&f1iuPbJMflSe_5V*0V-KWLYR>XR(~xgv^2MX$kE*ivlm=8yvdf5(O>rM|K- z&#qL@Fo!iAq+{FL+iH)HORoL=h@2SiXvTMLz|DHJD_6%h-rfM&gEk+;km#$(ZLuyN zPh=h(JkGxn4I9vzu<3SPa(97c+nT6}!OZpKukSNsQ@G))$i+r>YkDO=u?w{9sLx+< zI*8#WJ9-z483}~XON8!Kp3+`05iG9dXaDORB5XzV8T*oJx__E9*=M^1*XU{*zxBIw zj878<5^bwJs)CbM6Q0-Ds@HelPllW01g~Q6jBjiS8tl<~%&D?}b5_uOR zC3?9LHH&?E0GTWQXi)&MTdm$&bTF%m9Hjk*RH*B@b(-D~!cdIh@4F_CF=s2>ue5v7nSTbA}=kekQaHHM5(EU~P1)Qmxc|-L0MhQ4s zmFIQs22+!GEk?x*92egzJwbl7z&UO1*BvzuZzmnyfyhyz?x3h1@_6!8OV~VBn(TGA zGc}vPlYREVm5eHIHY~T3W7QEjesk168}{ai11_W#y>q^lHV&U}Xd|r5S~g^IzgIE? zzlqDY`BKcOVW$z9RpcF^;~PU=91o{bhzF1;qP8m6NPGYBbw=2#Xp>1= z=Y7a&1Sp<)FnG`H{ZnyYbM9BI4A$_Z9B{`s4lP;C1)yL@bX_-UCjDO&8qI!E!l^SE zM9aGzm7;L+_O9Kebv5H-32LeR%C1yb+%v4wD4M)yLBmL{gWH^$j!#IIf zM87a3;Ljm2$eHfAXLzmQIXGqc1ZMj(o4c}f_}ep>1%WwCCYNq8j%YXgOdxsnTfTjI zf~(nNg#Ao@S=3sN6XpfzQPV1P+wLQevknyBns&td0WaYSeI5*N$VmDa`p+HqKZj$B ZK{5}#DsLBdC5qK{{D`sc6K%WD{{rQqV}Aeu literal 0 HcmV?d00001 diff --git a/Documentation/admin-guide/mm/damon/streamcluster_wss_time.png b/Documentation/admin-guide/mm/damon/streamcluster_wss_time.png new file mode 100644 index 0000000000000000000000000000000000000000..84a9fd3de09fb0529aeb1051f1ce33a0a209e86a GIT binary patch literal 8830 zcmcI~XH*o?vM$}szy!ib&Pg&1K?Nia2m+EpKv9B#iJS!_G(nW0ASxhnL_`n~6a*v= zB8UPqNL13GWDtfVnYp9)ob&Fx>)iF;kN0}@bg!=7ySnzSZ&&TAngk0o14a}NiiU=U z(a2EGiiQS+XlMWuf|g1FnPms5563J_&gxSr6e z#h#Ks2Us3!zD6Ch^_^hbUi1y4PlDR^HBfk2^- zheGJC$s<$9`0DN!N=rAPBAkN5Q-qqDnwp!Not&H|CMLACwe$1y1qB7Uz4r&GV@CQK zo)4g*Vfygr2RxEt;-#SxqA}9bIUADmD;JqwDumI$%)k6G2;Xs7L;TTE{PhfSax5V7 zYcGq8zqW|qH6ALH|8DbdNr9A~I#!sI;}qnJR4(RdaV{PgA4?c{Y{kBo!@ zL5)rjlVCuwRSNDC%()xZ;otFEAcP@6{T&?**~b&e$d%2P^t^?TRnksUN#^E3@8NkYQiE z{KE4|lw|*<_I_YAMIoGC*fC8!t2R)SVmTkp;k!Vy9MI6JpoJP|Fgz^%YF9raoDs=a z6pJ3%@)$@;J$*+BW11_uK(743n%=bZqfHvWTV~2w-t|<${bl*kss7bZtJZve1)q!! z)|?(4inNKM3M7jhlZ@E(EKM3Y9!TCP$PB}Z_wLT+$zJ?JxE6uKk)<%0aDy81%5+l6 zkPAy0a}~6?R=hj`f`e$7=+jI+qiD#D?SUk}`5vWwdev>xW#KKAxHH~F2M#j)*A`Nk z)MOVHF*ulJX;M&#!yQXBddP+_Jgu0J1_GZX#VxIbgt*&8&!GN)c1nwddS02x{TcPY z{rDNW8}ZSQ67ByekeCD0?G|R(Qr*32-=uDJ9FF-JdYJ8OIaP2ZtdKe%QqfQ#bw0!} zsrOu1o{2I8f@1$UyZ@lTz@kfYVyvS%>6Z(fXn{Ur(v$gTs@P7wW;(D)=bp7p2c4^Z zdO{>4EO1h@ubEFOfD10r#4JlUH@3d(Tz%X z$>bZ={ZROVoe9vy;RNfTvam_d4Ri=Bc*aiYbqVv^;}~mM@l~B?rC)rLQ3z znN$a9Q=GUfH%;AyA^Q0bS-D~;&GA! zz3+dbI^vU#Gas1)L%fZ}XKocAZ~fYteNVBL+Y>cM3x+|Emh9cI#^W_h?grv7odY~R*w)rcfCR9sX;fBZ*aBK z!!~Guh?i#FwAi1^XSKcQh07LbP}gA>XnRj2q%*@d*iIg`1*X?tJ4JeA-@dfHG=(qX zZ|U<4-e)sNo0F#j=BeO1BP7KCKj;Vsex)MvAr7ZTMI;Nt&_FT43jD8~jCR1_wtkQQ zPX2fL{y*~izwJZ?ssk-hI(>e6HkB48%kyu<`rla%AUiGyKfAU=vD@|$!r%T3;}dgr ziBVK_G-zgkZCGSLy48ygA3|l?LYMZas0~66(`j4K0Gj<`q?$i?-9(<$_GYISerJ&j zxt0q=25Ky(VGwgq)Bt$?;bymCBmx8?0!_WZW1Gp?!vsbQ@tGDdt?(;)VBMC|+)e=j zhXvLO&Zj)U)G++2P6WslQ)2J}bx2V<qf;?3ry3%Z^XjN=) zBoU^I%h-iYnN*#@tvSUA<5PBFY`>0i+_{t~TWpR~mK7JR{<~uAgTC8v7`+9Hyn&mF zCmP=}{bgH_Z#qXcJ*!L$af{K94b=Y1MK69~5EZ;AnYm?x=A*74?Ks_;#%1Li8!Ud) z15K5|9JgtWP}ftvmGpW5#+IgX@w3Gpw$>>U$16GY*S6HLNV8QW>~iaI`U}txo z&d5EhQq8n-r{tOEYjjhg7VvB{G8)@kh4!&=9paOICEO|eF^r^`zdCU4)!*^B4er73 zK~LW#nsYFXTn_T+SbOczX%o?Q1WOVdKy^^QflS7N6_+o1Iy9IP98&~g?ZApfHsm=w zqmYcnpOY6q^3g`j5onZ2F|6WnXRYu$uO6_~gh;nI7%by`T%p_LP?2(e|BjmnJ`i}$ zb64}Inm4=%`ZY80FV`WIZ#)kwm-Nt|fO*#Fq+j```w+q#r-SWcYhxF;=C@-f7#;7~eUQeFGC6H~{!kF?SQ73gI_n3WXruq?_`+aH_)|9;j?-X;Fa|TQn@+sDb_cl&r8rAs0?*ffSv7^5b9lzX)ION z(!uF_gTcLvhtDCw8mPBsa^D67)4ZuI6Y@BhXtC63#5(t{$4tP%IyW)W__8>Ed_ssL zxsxjDOPHsmJ+4jV#?@FzfT|Tw^N08B7uf$!y?4D6L+cd5D1!15I{Kg?8Va>|3Uvz& z1k?yQGZnHZxLi@%d=T@q6{B zX{Hhk8BuCL@F_V>*gJqB3=^bC&HA662y|EyazPP}--RRi<$F zIXbE2cby-DyHB^>y#~m)R@vP&a{*Eue@lRQte6UZ9=X{(htAsJx{z$Bv1v+(hpGa~~AGr}6ImyQFj$_5*s5SG2=AaJU5bMrR>)?ap%m3(SL zuiWE~1z}5GSx$%WD)baVZdc1LR6|Ni2AGg%1?fIb85ya-7=rGij*yF_p1h@k|FMP? z5?m-PGa{JeJ%j<91GbVv_PCnSj3ncuq;6h0Jw$P-!E5xFpGHO)3nwpA@XA_YVHB#^ z>-CLj=(GqD-bLm47`grXmN*8+(5PQh52^+S&;Y z^3goh+?WQ2=;Yl5u0`%hiF%i)K+x7t-97RR*wPZRX@2Vthp(H}Mq=?Z$H{o8J*syF zrXj@!s_7qkaLu4y;8iN8x^Ew z*X+b7UT+&`CytH|9{ZB-gS_U(dQ8L`C`$-MpA72$a8U1mu3Z%r=xg7(TNYu`%WS4V zRa2E8zX&I>IiY9gHgsd8nY$TU&|zkDs(-LpHJ$3JNzTl_Rk^96@&$gvY^T@+XZKQ& z-9oNn?`rEde9HQiis2=b3tH74%5MY*l~?hj97GMlg|J6H$hm8R^3Kk{9v~-q~y0H50b<75PJ2wg(dt0`& zE%oI>R^Hp+gToU;63(QRY%uQxkG2Tj%e)$l>}ZBp$sF(lTcQgb7U(!=M8YGI9aKYc zt14#+t0Iekh`@7puUq||(2~!@;WXQ$N@FE~dJ8#D8Exc1-x})Lc-LVw=y*a~W%RNo$LJ7PHJEo1~+ z(pc?*F<_m`zK#iGfCABGd+!c&v!U;EH#kCO&*WM&TsZyQ)~>IV@za9<4h6E5-*gmmPHhxAFw_2$~#e zoPVE-tS$e}LJUAXtGDn5{l#Zz@QmJ^X)c*DDON==@ z@I*5k`9&BX=b=1a`M?h5*%W1Vr~EVHOBx=4H_9ecEL;-*C^k312#nOr1MAL*3p-zp zP(Iw10z%Jjl!i;;B^FF@=BF`4g9gZ&n#{2@ZiqjMUxf6F>pEG;?LD_H0dGe&o4;La zwoex!Dg0hLiWGKMcqIa`w@PX?>WD={Gma|;uno^wBJim)=cUf*Ybd}0t$VAGgKQ?_ z#d*^XS7yn8oRiWBaF~pJ;X_(u~12-<;7niRL1=4}>Z$ga; z?E;hQVigAx0n;b-g1=aKd_{3xeD7HIvhtD^D{=O0NYE_k*xz+OPo3bO z6+Q8&kArNu_!?57Oq?{%5W}~Wqla?_+!TPD4GwK*K;vbY zb39;J#tC@f&Hiab$eSr^;8cc^4lW|B7wdd7?+hWSESaMM8PzKT^kijN0|%!VFvP6z ztagFbMKJGde=}S|h65BZ&YK1RK_pj`o$h7olMaruPeBZaO0O#c@=+q&?0~s_;* zf{Vddrtd&-8LO<1=2D-(vNJ^1825 zGgUptx1hrvDHy{`49K%se40QS>e9#695eSgjs}iW`aYm*%kjF@b!EeN&b2j{-<3U3 z1TX*mOIB$5`aAorf73S)8e?b&P!cSNfSyuonS zpCRir+FxO`Pc#^Y};;TB_b#qeJ){ zVXy{E)~4GnaWIV$O$XCB==mIYZ++tBCN)>^9wi%Ka66q5*16eq^4Es>7tQ)~zklTtG0C)f=(%r8 z+TX#i=xZNRktdJKHa5(2#b&qI=Fo${UG8PNhF6HK$B5IR+F1O7R0CTX)cz`HPeZ`G zZgn{(qU-6&@3ZUblJdgT#XVTeKWVfa{(}}Mi;cKZ@_~x5m6vJ}NQzF!d)e>*ifj_S zfC)~rOK&X)Sr>PprK^I{)Vll?%xaZclnbFIO%%s3aS2Mnmo$!Bp+kC;b4?4`D&DO( zMsP>Ula3MJFe{XPl)?U&q@;ilX&|i!RwbSxLmIWSj%d&)Jh|Fw?zvD?9m*Bo^J|n4 zrn!9woxb=u`1SP;xW-H!sQlZT8nTS7tejWNg$93jthvDiC(vB7J#ybJrkN2s9b!~u zMe#chLPD?x`!#K_E`{t5-|$LYWDq84!}`%LPvN%sK{~3nI1HTygO}aR3(L>-Jayx} z8mr#d_$k*+RtM+GpDWJ-3gnL;sd$Zga~LjyZ${BtZovYrZb7p+LK2*JFDdu7swIZZ z=Tk77O$X+ExR1P+yVG}G5ux?RuIR2nZo5(4m6$}IIX#X=+Mut1J@ zNEt)c_>s9@2Q_*5r91bXv*QZR>jnUMWhK~&E1E3C8dV|Z28h!<)6edRV+uQ~(xxtf zSG5U?pN$^hq?)zMG1zoVOC4bBRz7l0xp)>GFa7{A9C!q*Ya0+DwK=X%$>~y9$u5_Kixi9vB?V0FyN(^odi#n*c^;O$!3@3%Fn>|Ge%i7c!dNuw;`WX(mQyz~Kj3IC zzil5Z@tctj;c^KGZ}5v2g_#%%_4b*rS3^fas<+WtXU==Wt=`Z0HQKixuN6SA2AZ*? zPaCPwA*YFA_a&-xnn@iTe+sjqe+lTLlK8J7Ui?I`> z4>DoYNN!Hms4x5DX#I%yz13K7*H~Lzhj2SvtqQfJl9USV=4}-+bIGs|aCxaXP|Fjnxsp2S<) zJn`xY+~ia==x7!LVTcJaP!kQ}oLPZT;@=994%^?1dW>Y^pRy9%lFgc0LixA#C`*jK z;-(%{KhO8A zQL|_UUpX*A+cvBDOEDMPZ}BA?y&2MCj0xlWOPtsMp2sqHn;nB~SZ-XwtqDHze!Zu3 z?2_rR;1(^eZPPTKtFON)2hr4=VwliD&|Dw(%vOU8ZV-<%(BfEZN9+dxHmiI$Mm<`x zR;QfPpJHG|xwd#okLE~J;-6CXKac)9k?#+)+p5z_0-C>;J()kB|1v}_(~~t_i(O4z z@~{^3Q}X1%OjMFy^8>^uUb)Nfm6H9BDCn;q<$X@kbuIX+n%!d@)g1CmH~A^4D+*z} zX}o%o6z&~dpv2&x12?hUugE{1C%QA;B^s3eb+4@)6oAi0O5?r;T&CM>WG#Eb*)8jw z_#-7?C`~j9u^qWu3K_J#%V!7*ttPSd=gIc+ocx&tNhNaXZaRO|duD!js{uVDl{h5N zl!Z34Z;|&6ZQ7uXf5)78h4DqdJ^|Kd{(JS(+3p)zQ}5h7=fNh&eQ}bb?`KqG?8BXe zodrnLBlG3}5jlmfoui$QyGb%iT)6=}@XYH@U~A*Nfk|7EY#jS+qQaEw$rZm+PZIr9jumQE zNER7TSY3Ve;BH(I?;YFLIoFu`KW_QW(^g9*%6ki`4IuaYzv=&;AFM!T7H&uK#~WGK z9a@i;wh3~&@WFp3GW=6F|NRbK7_D1qkF@xuZM$9zBYRXWg=k(K77*E_44m+9xI_Ea zkCNDGz(}ggQXc8-#GSEMMGVT*Ar+&RGfI`ig4p9O?q@!o8S%&3>6qki7#x3<@;Wy* z{bsq>dd80o^~Zi5?QimelosIucxC%<>t)CbupF(P8oS;bE$$1VM^upSMu*VRO|+@r!o(^8Z!djJL@s6;&@G0M`qZ|EZh3l z1ZvJ@%)#RTOR{Vj0Noeahv9Z#MBdA`5SCHb-o4?Eu?_hL4=8;5PzJr#J_I>SU2UjSOhgdHt8 zM_|@Z^V{;4{`vQtegv;W`+cqbtKFHLId-wp*T_-ngNu)Kx22f&^Plx>>)~>eY4T!n eCI6{X3*jh?2aa;Fb!_er$w=Q!@0G4|%zpsDRL;Nv literal 0 HcmV?d00001 diff --git a/Documentation/admin-guide/mm/damon/usage.rst b/Documentation/admin-guide/mm/damon/usage.rst new file mode 100644 index 000000000000..9d71f04d12ee --- /dev/null +++ b/Documentation/admin-guide/mm/damon/usage.rst @@ -0,0 +1,296 @@ +.. SPDX-License-Identifier: GPL-2.0 + +=============== +Detailed Usages +=============== + +DAMON provides below three interfaces for different users. + +- *DAMON user space tool.* + This is for privileged people such as system administrators who want a + just-working human-friendly interface. Using this, users can use the DAMON’s + major features in a human-friendly way. It may not be highly tuned for + special cases, though. It supports virtual address space monitoring only. +- *debugfs interface.* + This is for privileged user space programmers who want more optimized use of + DAMON. Using this, users can use DAMON’s major features by reading + from and writing to special debugfs files. Therefore, you can write and use + your personalized DAMON debugfs wrapper programs that reads/writes the + debugfs files instead of you. The DAMON user space tool is also a reference + implementation of such programs. It supports virtual address space + monitoring only. +- *Kernel Space Programming Interface.* + This is for kernel space programmers. Using this, users can utilize every + feature of DAMON most flexibly and efficiently by writing kernel space + DAMON application programs for you. You can even extend DAMON for various + address spaces. + +This document does not describe the kernel space programming interface in +detail. For that, please refer to :doc:`api`. + + +DAMON User Sapce Tool +===================== + +A reference implementation of the DAMON user space tools which provides a +convenient user interface is in the kernel source tree. It is located at +``tools/damon/damo`` of the tree. + +The tool provides a subcommands based interface. Every subcommand provides +``-h`` option, which provides the minimal usage of it. Currently, the tool +supports two subcommands, ``record`` and ``report``. + +Below example commands assume you set ``$PATH`` to point ``tools/damon/`` for +brevity. It is not mandatory for use of ``damo``, though. + + +Recording Data Access Pattern +----------------------------- + +The ``record`` subcommand records the data access pattern of target workloads +in a file (``./damon.data`` by default). You can specify the target as either +process id of running target or a command for execution of it. Below example +shows a command target usage:: + + # cd /tools/damon/ + # damo record "sleep 5" + +The tool will execute ``sleep 5`` by itself and record the data access patterns +of the process. Below example shows a pid target usage:: + + # sleep 5 & + # damo record `pidof sleep` + +The location of the recorded file can be explicitly set using ``-o`` option. +You can further tune this by setting the monitoring attributes. To know about +the monitoring attributes in detail, please refer to :doc:`mechanisms`. + + +Analyzing Data Access Pattern +----------------------------- + +The ``report`` subcommand reads a data access pattern record file (if not +explicitly specified using ``-i`` option, reads ``./damon.data`` file by +default) and generates human-readable reports. You can specify what type of +report you want using a sub-subcommand to ``report`` subcommand. ``raw``, +``heats``, and ``wss`` report types are supported for now. + + +raw +~~~ + +``raw`` sub-subcommand simply transforms the binary record into a +human-readable text. For example:: + + $ damo report raw + start_time: 193485829398 + rel time: 0 + nr_tasks: 1 + pid: 1348 + nr_regions: 4 + 560189609000-56018abce000( 22827008): 0 + 7fbdff59a000-7fbdffaf1a00( 5601792): 0 + 7fbdffaf1a00-7fbdffbb5000( 800256): 1 + 7ffea0dc0000-7ffea0dfd000( 249856): 0 + + rel time: 100000731 + nr_tasks: 1 + pid: 1348 + nr_regions: 6 + 560189609000-56018abce000( 22827008): 0 + 7fbdff59a000-7fbdff8ce933( 3361075): 0 + 7fbdff8ce933-7fbdffaf1a00( 2240717): 1 + 7fbdffaf1a00-7fbdffb66d99( 480153): 0 + 7fbdffb66d99-7fbdffbb5000( 320103): 1 + 7ffea0dc0000-7ffea0dfd000( 249856): 0 + +The first line shows the recording started timestamp (nanosecond). Records of +data access patterns follows. Each record is separated by a blank line. Each +record first specifies the recorded time (``rel time``) in relative to the +start time, the number of monitored tasks in this record (``nr_tasks``). +Recorded data access patterns of each task follow. Each data access pattern +for each task shows the target's pid (``pid``) and a number of monitored +address regions in this access pattern (``nr_regions``) first. After that, +each line shows the start/end address, size, and the number of observed +accesses of each region. + + +heats +~~~~~ + +The ``raw`` output is very detailed but hard to manually read. ``heats`` +sub-subcommand plots the data in 3-dimensional form, which represents the time +in x-axis, address of regions in y-axis, and the access frequency in z-axis. +Users can set the resolution of the map (``--tres`` and ``--ares``) and +start/end point of each axis (``--tmin``, ``--tmax``, ``--amin``, and +``--amax``) via optional arguments. For example:: + + $ damo report heats --tres 3 --ares 3 + 0 0 0.0 + 0 7609002 0.0 + 0 15218004 0.0 + 66112620851 0 0.0 + 66112620851 7609002 0.0 + 66112620851 15218004 0.0 + 132225241702 0 0.0 + 132225241702 7609002 0.0 + 132225241702 15218004 0.0 + +This command shows a recorded access pattern in heatmap of 3x3 resolution. +Therefore it shows 9 data points in total. Each line shows each of the data +points. The three numbers in each line represent time in nanosecond, address, +and the observed access frequency. + +Users will be able to convert this text output into a heatmap image (represents +z-axis values with colors) or other 3D representations using various tools such +as 'gnuplot'. For more convenience, ``heats`` sub-subcommand provides the +'gnuplot' based heatmap image creation. For this, you can use ``--heatmap`` +option. Also, note that because it uses 'gnuplot' internally, it will fail if +'gnuplot' is not installed on your system. For example:: + + $ ./damo report heats --heatmap heatmap.png + +Creates the heatmap image in ``heatmap.png`` file. It supports ``pdf``, +``png``, ``jpeg``, and ``svg``. + +If the target address space is virtual memory address space and you plot the +entire address space, the huge unmapped regions will make the picture looks +only black. Therefore you should do proper zoom in / zoom out using the +resolution and axis boundary-setting arguments. To make this effort minimal, +you can use ``--guide`` option as below:: + + $ ./damo report heats --guide + pid:1348 + time: 193485829398-198337863555 (4852034157) + region 0: 00000094564599762944-00000094564622589952 (22827008) + region 1: 00000140454009610240-00000140454016012288 (6402048) + region 2: 00000140731597193216-00000140731597443072 (249856) + +The output shows unions of monitored regions (start and end addresses in byte) +and the union of monitored time duration (start and end time in nanoseconds) of +each target task. Therefore, it would be wise to plot the data points in each +union. If no axis boundary option is given, it will automatically find the +biggest union in ``--guide`` output and set the boundary in it. + + +wss +~~~ + +The ``wss`` type extracts the distribution and chronological working set size +changes from the records. For example:: + + $ ./damo report wss + # + # pid 1348 + # avr: 66228 + 0 0 + 25 0 + 50 0 + 75 0 + 100 1920615 + +Without any option, it shows the distribution of the working set sizes as +above. It shows 0th, 25th, 50th, 75th, and 100th percentile and the average of +the measured working set sizes in the access pattern records. In this case, +the working set size was zero for 75th percentile but 1,920,615 bytes in max +and 66,228 bytes on average. + +By setting the sort key of the percentile using '--sortby', you can show how +the working set size has chronologically changed. For example:: + + $ ./damo report wss --sortby time + # + # pid 1348 + # avr: 66228 + 0 0 + 25 0 + 50 0 + 75 0 + 100 0 + +The average is still 66,228. And, because the access was spiked in very short +duration and this command plots only 4 data points, we cannot show when the +access spikes made. Users can specify the resolution of the distribution +(``--range``). By giving more fine resolution, the short duration spikes could +be found. + +Similar to that of ``heats --heatmap``, it also supports 'gnuplot' based simple +visualization of the distribution via ``--plot`` option. + + +debugfs Interface +================= + +DAMON exports four files, ``attrs``, ``pids``, ``record``, and ``monitor_on`` +under its debugfs directory, ``/damon/``. + + +Attributes +---------- + +Users can get and set the ``sampling interval``, ``aggregation interval``, +``regions update interval``, and min/max number of monitoring target regions by +reading from and writing to the ``attrs`` file. To know about the monitoring +attributes in detail, please refer to :doc:`mechanisms`. For example, below +commands set those values to 5 ms, 100 ms, 1,000 ms, 10 and 1000, and then +check it again:: + + # cd /damon + # echo 5000 100000 1000000 10 1000 > attrs + # cat attrs + 5000 100000 1000000 10 1000 + + +Target PIDs +----------- + +Users can get and set the pids of monitoring target processes by reading from +and writing to the ``pids`` file. For example, below commands set processes +having pids 42 and 4242 as the processes to be monitored and check it again:: + + # cd /damon + # echo 42 4242 > pids + # cat pids + 42 4242 + +Note that setting the pids doesn't start the monitoring. + + +Record +------ + +This debugfs file allows you to record monitored access patterns in a regular +binary file. The recorded results are first written in an in-memory buffer and +flushed to a file in batch. Users can get and set the size of the buffer and +the path to the result file by reading from and writing to the ``record`` file. +For example, below commands set the buffer to be 4 KiB and the result to be +saved in ``/damon.data``. :: + + # cd /damon + # echo "4096 /damon.data" > record + # cat record + 4096 /damon.data + +The recording can be disabled by setting the buffer size zero. + + +Turning On/Off +-------------- + +Setting the files as described above doesn't incur effect unless you +explicitly start the monitoring. You can start, stop, and check the current +status of the monitoring by writing to and reading from the ``monitor_on`` +file. Writing ``on`` to the file starts the monitoring and recording of the +targets with the attributes. Writing ``off`` to the file stops those. DAMON +also stops if every target process is terminated. Below example commands turn +on, off, and check the status of DAMON:: + + # cd /damon + # echo on > monitor_on + # echo off > monitor_on + # cat monitor_on + off + +Please note that you cannot write to the above-mentioned debugfs files while +the monitoring is turned on. If you write to the files while DAMON is running, +an error code such as ``-EBUSY`` will be returned. diff --git a/Documentation/admin-guide/mm/index.rst b/Documentation/admin-guide/mm/index.rst index 11db46448354..d3d0ba373eb6 100644 --- a/Documentation/admin-guide/mm/index.rst +++ b/Documentation/admin-guide/mm/index.rst @@ -27,6 +27,7 @@ the Linux memory management. concepts cma_debugfs + data_access_monitor hugetlbpage idle_page_tracking ksm -- 2.17.1