Received-SPF: pass (google.com: domain of linux-kernel+bounces-189067-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Date: Fri, 24 May 2024 13:00:06 -0700 (PDT)
From: David Rientjes <rientjes@google.com>
To: Sourav Panda <souravpanda@google.com>
cc: corbet@lwn.net, gregkh@linuxfoundation.org, rafael@kernel.org, 
    Andrew Morton <akpm@linux-foundation.org>, mike.kravetz@oracle.com, 
    muchun.song@linux.dev, rppt@kernel.org, david@redhat.com, 
    rdunlap@infradead.org, chenlinxuan@uniontech.com, yang.yang29@zte.com.cn, 
    tomas.mudrunka@gmail.com, bhelgaas@google.com, ivan@cloudflare.com, 
    pasha.tatashin@soleen.com, yosryahmed@google.com, hannes@cmpxchg.org, 
    shakeelb@google.com, kirill.shutemov@linux.intel.com, 
    wangkefeng.wang@huawei.com, adobriyan@gmail.com, 
    Vlastimil Babka <vbabka@suse.cz>, 
    "Liam R. Howlett" <Liam.Howlett@oracle.com>, surenb@google.com, 
    linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, 
    linux-doc@vger.kernel.org, linux-mm@kvack.org, 
    Matthew Wilcox <willy@infradead.org>, weixugc@google.com
Subject: Re: [PATCH v12] mm: report per-page metadata information
In-Reply-To: <45fb4c94-dd77-94c3-f08f-81bf31e6d6d2@google.com>
Message-ID: <1e1b89af-3b7a-7400-cfd7-d22a101955de@google.com>
References: <20240512010611.290464-1-souravpanda@google.com> <45fb4c94-dd77-94c3-f08f-81bf31e6d6d2@google.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII

On Sat, 18 May 2024, David Rientjes wrote:

> On Sun, 12 May 2024, Sourav Panda wrote:
> 
> > Today, we do not have any observability of per-page metadata
> > and how much it takes away from the machine capacity. Thus,
> > we want to describe the amount of memory that is going towards
> > per-page metadata, which can vary depending on build
> > configuration, machine architecture, and system use.
> > 
> > This patch adds 2 fields to /proc/vmstat that can used as shown
> > below:
> > 
> > Accounting per-page metadata allocated by boot-allocator:
> > 	/proc/vmstat:nr_memmap_boot * PAGE_SIZE
> > 
> > Accounting per-page metadata allocated by buddy-allocator:
> > 	/proc/vmstat:nr_memmap * PAGE_SIZE
> > 
> > Accounting total Perpage metadata allocated on the machine:
> > 	(/proc/vmstat:nr_memmap_boot +
> > 	 /proc/vmstat:nr_memmap) * PAGE_SIZE
> > 
> > Utility for userspace:
> > 
> > Observability: Describe the amount of memory overhead that is
> > going to per-page metadata on the system at any given time since
> > this overhead is not currently observable.
> > 
> > Debugging: Tracking the changes or absolute value in struct pages
> > can help detect anomalies as they can be correlated with other
> > metrics in the machine (e.g., memtotal, number of huge pages,
> > etc).
> > 
> > page_ext overheads: Some kernel features such as page_owner
> > page_table_check that use page_ext can be optionally enabled via
> > kernel parameters. Having the total per-page metadata information
> > helps users precisely measure impact. Furthermore, page-metadata
> > metrics will reflect the amount of struct pages reliquished
> > (or overhead reduced) when hugetlbfs pages are reserved which
> > will vary depending on whether hugetlb vmemmap optimization is
> > enabled or not.
> > 
> > For background and results see:
> > lore.kernel.org/all/20240220214558.3377482-1-souravpanda@google.com
> > 
> > Signed-off-by: Sourav Panda <souravpanda@google.com>
> > Reviewed-by: Pasha Tatashin <pasha.tatashin@soleen.com>
> 
> Acked-by: David Rientjes <rientjes@google.com>
> 

This would be a very useful extension to be able to provide observability 
of per-page metadata overhead and the impact of things like HVO on the 
overall footprint.  Today, we don't have observability for this memory 
overhead.

Andrew, anything else that can be addressed before this is eligible for 
staging in MM unstable?