Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Sun, 2 Jul 2023 17:09:11 -0700 (PDT)
From:   David Rientjes <rientjes@google.com>
To:     Julian Pidancet <julian.pidancet@oracle.com>
cc:     Christoph Lameter <cl@linux.com>,
        "Lameter, Christopher" <cl@os.amperecomputing.com>,
        Pekka Enberg <penberg@kernel.org>,
        Joonsoo Kim <iamjoonsoo.kim@lge.com>,
        Andrew Morton <akpm@linux-foundation.org>,
        Vlastimil Babka <vbabka@suse.cz>,
        Roman Gushchin <roman.gushchin@linux.dev>,
        Hyeonggon Yoo <42.hyeyoo@gmail.com>, linux-mm@kvack.org,
        Jonathan Corbet <corbet@lwn.net>, linux-doc@vger.kernel.org,
        linux-kernel@vger.kernel.org, Matthew Wilcox <willy@infradead.org>,
        Kees Cook <keescook@chromium.org>,
        Rafael Aquini <aquini@redhat.com>
Subject: Re: [PATCH v2] mm/slub: disable slab merging in the default
 configuration
In-Reply-To: <20230629221910.359711-1-julian.pidancet@oracle.com>
Message-ID: <38083ed2-333b-e245-44e4-2f355e4f9249@google.com>
References: <20230629221910.359711-1-julian.pidancet@oracle.com>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="2003067076-267791279-1688342952=:3716371"
Precedence: bulk

  This message is in MIME format.  The first part should be readable text,
  while the remaining parts are likely unreadable without MIME-aware tools.

--2003067076-267791279-1688342952=:3716371
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8BIT

On Fri, 30 Jun 2023, Julian Pidancet wrote:

> Make CONFIG_SLAB_MERGE_DEFAULT default to n unless CONFIG_SLUB_TINY is
> enabled. Benefits of slab merging is limited on systems that are not
> memory constrained: the memory overhead is low and evidence of its
> effect on cache hotness is hard to come by.
> 
> On the other hand, distinguishing allocations into different slabs will
> make attacks that rely on "heap spraying" more difficult to carry out
> with success.
> 
> Take sides with security in the default kernel configuration over
> questionnable performance benefits/memory efficiency.
> 
> A timed kernel compilation test, on x86 with 4K pages, conducted 10
> times with slab_merge, and the same test then conducted with
> slab_nomerge on the same hardware in a similar state do not show any
> sign of performance hit one way or another:
> 
>       | slab_merge       | slab_nomerge     |
> ------+------------------+------------------|
> Time  |  588.080 ± 0.799 |  587.308 ± 1.411 |
> Min   |          586.267 |          584.640 |
> Max   |          589.248 |          590.091 |
> 
> Peaks in slab usage during the test workload reveal a memory overhead
> of 2.2 MiB when using slab_nomerge. Slab usage overhead after a fresh boot
> amounts to 2.3 MiB:
> 
> Slab Usage         | slab_merge | slab_nomerge |
> -------------------+------------+--------------|
> After fresh boot   |   79908 kB |     82284 kB |
> During test (peak) |  127940 kB |    130204 kB |
> 
> Signed-off-by: Julian Pidancet <julian.pidancet@oracle.com>
> Reviewed-by: Kees Cook <keescook@chromium.org>

Thanks for continuing to work on this.

I think we need more data beyond just kernbench.  Christoph's point about 
different page sizes is interesting.  In the above results, I don't know 
the page orders for the various slab caches that this workload will 
stress.  I think the memory overhead data may be different depending on 
how slab_max_order is being used, if at all.

We should be able to run this through a variety of different benchmarks 
and measure peak slab usage at the same time for due diligence.  I support 
the change in the default, I would just prefer to know what the 
implications of it is.

Is it possible to collect data for other microbenchmarks and real-world 
workloads?  And perhaps also with different page sizes where this will 
impact memory overhead more?  I can help running more workloads once we 
have the next set of data.

> ---
> 
> v2:
>   - Re-run benchmark to minimize variance in results due to CPU
>     frequency scaling.
>   - Record slab usage after boot and peaks during tests workload.
>   - Include benchmark results in commit message.
>   - Fix typo: s/MEGE/MERGE/.
>   - Specify that "overhead" refers to memory overhead in SLUB doc.
> 
> v1:
>   - Link: https://lore.kernel.org/linux-mm/20230627132131.214475-1-julian.pidancet@oracle.com/
> 
>  .../admin-guide/kernel-parameters.txt         | 29 ++++++++++---------
>  Documentation/mm/slub.rst                     |  7 +++--
>  mm/Kconfig                                    |  6 ++--
>  3 files changed, 22 insertions(+), 20 deletions(-)
> 
> diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
> index c5e7bb4babf0..7e78471a96b7 100644
> --- a/Documentation/admin-guide/kernel-parameters.txt
> +++ b/Documentation/admin-guide/kernel-parameters.txt
> @@ -5652,21 +5652,22 @@
>  
>  	slram=		[HW,MTD]
>  
> -	slab_merge	[MM]
> -			Enable merging of slabs with similar size when the
> -			kernel is built without CONFIG_SLAB_MERGE_DEFAULT.
> -
>  	slab_nomerge	[MM]
> -			Disable merging of slabs with similar size. May be
> -			necessary if there is some reason to distinguish
> -			allocs to different slabs, especially in hardened
> -			environments where the risk of heap overflows and
> -			layout control by attackers can usually be
> -			frustrated by disabling merging. This will reduce
> -			most of the exposure of a heap attack to a single
> -			cache (risks via metadata attacks are mostly
> -			unchanged). Debug options disable merging on their
> -			own.
> +			Disable merging of slabs with similar size when
> +			the kernel is built with CONFIG_SLAB_MERGE_DEFAULT.
> +			Allocations of the same size made in distinct
> +			caches will be placed in separate slabs. In
> +			hardened environment, the risk of heap overflows
> +			and layout control by attackers can usually be
> +			frustrated by disabling merging.
> +
> +	slab_merge	[MM]
> +			Enable merging of slabs with similar size. May be
> +			necessary to reduce overhead or increase cache
> +			hotness of objects, at the cost of increased
> +			exposure in case of a heap attack to a single
> +			cache. (risks via metadata attacks are mostly
> +			unchanged).
>  			For more information see Documentation/mm/slub.rst.
>  
>  	slab_max_order=	[MM, SLAB]
> diff --git a/Documentation/mm/slub.rst b/Documentation/mm/slub.rst
> index be75971532f5..0e2ce82177c0 100644
> --- a/Documentation/mm/slub.rst
> +++ b/Documentation/mm/slub.rst
> @@ -122,9 +122,10 @@ used on the wrong slab.
>  Slab merging
>  ============
>  
> -If no debug options are specified then SLUB may merge similar slabs together
> -in order to reduce overhead and increase cache hotness of objects.
> -``slabinfo -a`` displays which slabs were merged together.
> +If the kernel is built with ``CONFIG_SLAB_MERGE_DEFAULT`` or if ``slab_merge``
> +is specified on the kernel command line, then SLUB may merge similar slabs
> +together in order to reduce memory overhead and increase cache hotness of
> +objects.  ``slabinfo -a`` displays which slabs were merged together.
>  

Suggest mentioning that one of the primary goals of slab cache merging is 
to reduce cache footprint.

>  Slab validation
>  ===============
--2003067076-267791279-1688342952=:3716371--