Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
IronPort-SDR: l7FgCOW1uOUOCBtlDuJ60BwlAl1BMk1XDTlOgDe0mrU6MYyPQvHMiwZ4w5L6qf0rqq6uGQBUfj
 hd23/eTF6bgg==
From:   SeongJae Park <sjpark@amazon.com>
To:     David Hildenbrand <david@redhat.com>
CC:     SeongJae Park <sjpark@amazon.com>, <akpm@linux-foundation.org>,
        "SeongJae Park" <sjpark@amazon.de>, <Jonathan.Cameron@Huawei.com>,
        <aarcange@redhat.com>, <acme@kernel.org>,
        <alexander.shishkin@linux.intel.com>, <amit@kernel.org>,
        <benh@kernel.crashing.org>, <brendan.d.gregg@gmail.com>,
        <brendanhiggins@google.com>, <cai@lca.pw>,
        <colin.king@canonical.com>, <corbet@lwn.net>, <dwmw@amazon.com>,
        <foersleo@amazon.de>, <irogers@google.com>, <jolsa@redhat.com>,
        <kirill@shutemov.name>, <mark.rutland@arm.com>, <mgorman@suse.de>,
        <minchan@kernel.org>, <mingo@redhat.com>, <namhyung@kernel.org>,
        <peterz@infradead.org>, <rdunlap@infradead.org>,
        <riel@surriel.com>, <rientjes@google.com>, <rostedt@goodmis.org>,
        <sblbir@amazon.com>, <shakeelb@google.com>, <shuah@kernel.org>,
        <sj38.park@gmail.com>, <snu@amazon.de>, <vbabka@suse.cz>,
        <vdavydov.dev@gmail.com>, <yang.shi@linux.alibaba.com>,
        <ying.huang@intel.com>, <linux-damon@amazon.com>,
        <linux-mm@kvack.org>, <linux-doc@vger.kernel.org>,
        <linux-kernel@vger.kernel.org>
Subject: Re: Re: [RFC v11 3/8] mm/damon: Implement data access monitoring-based operation schemes
Date:   Tue, 9 Jun 2020 11:17:25 +0200
Message-ID: <20200609091725.15859-1-sjpark@amazon.com>
In-Reply-To: <ed4b0be0-34ad-511c-7817-e4506ed2f891@redhat.com> (raw)
MIME-Version: 1.0
Content-Type: text/plain
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, 9 Jun 2020 10:47:45 +0200 David Hildenbrand <david@redhat.com> wrote:

> On 09.06.20 08:53, SeongJae Park wrote:
> > From: SeongJae Park <sjpark@amazon.de>
> > 
> > In many cases, users might use DAMON for simple data access aware
> > memory management optimizations such as applying an operation scheme to
> > a memory region of a specific size having a specific access frequency
> > for a specific time.  For example, "page out a memory region larger than
> > 100 MiB but having a low access frequency more than 10 minutes", or "Use
> > THP for a memory region larger than 2 MiB having a high access frequency
> > for more than 2 seconds".
> > 
> > To minimize users from spending their time for implementation of such
> > simple data access monitoring-based operation schemes, this commit makes
> > DAMON to handle such schemes directly.  With this commit, users can
> > simply specify their desired schemes to DAMON.
> 
> What would be the alternative? How would a solution where these policies
> are handled by user space (or inside an application?) look like?

Most simple form of the altermative solution would be doing offline data access
pattern profiling using DAMON and modifying the application source code or
system configuration based on the profiling results.

More automated alternative solution would be a daemon constructed with two
modules:

 - monitor: monitors the data access pattern of the workload via the DAMON
   debugfs interface
 - memory manager: based on the monitoring result, make appropriate memory
   management changes via mlock(), madvise(), sysctl, etc.

The daemon would be able to run inside the application process as a thread, or
outside as a standalone process.  If the daemon could not run inside the
application process, the memory management changes it could make would be
further limited, though, as mlock() and madvise() would not be available.  The
madvise_process(), which is already merged in the next tree, would be helpful
in this case.

> > 
> > Each of the schemes is composed with conditions for filtering of the
> > target memory regions and desired memory management action for the
> > target.  Specifically, the format is::
> > 
> >     <min/max size> <min/max access frequency> <min/max age> <action>
> > 
> > The filtering conditions are size of memory region, number of accesses
> > to the region monitored by DAMON, and the age of the region.  The age of
> > region is incremented periodically but reset when its addresses or
> > access frequency has significantly changed or the action of a scheme was
> > applied.  For the action, current implementation supports only a few of
> > madvise() hints, ``MADV_WILLNEED``, ``MADV_COLD``, ``MADV_PAGEOUT``,
> > ``MADV_HUGEPAGE``, and ``MADV_NOHUGEPAGE``.
> 
> I am missing some important information. Is this specified for *all*
> user space processes? Or how is this configured? What are examples?
> 
> E.g., messing with ``MADV_HUGEPAGE`` vs. ``MADV_NOHUGEPAGE`` of random
> applications can change the behavior/break these applications. (e.g., if
> userfaultfd is getting used and the applciation explicitly sets
> MADV_NOHUGEPAGE).

Only monitoring target processes will be applied.  The monitoring target
processes can be specified by writing the process ids to 'pids' debugfs file or
constructing the 'struct damon_ctx' via the programming interface.

I will refine the commit message to make the points clearer, in the next spin.

[...]
> 
> 
> -- 
> Thanks,
> 
> David / dhildenb