From: Sahara <[email protected]>
Slab cache flags are exported to sysfs and are allowed to get modified
from userspace. Some of those may call calculate_sizes function because
the changed flag can take an effect on slab object size and layout,
which means kmem_cache may have different order and objects.
The freelist pointer corruption occurs if some slab flags are modified
while CONFIG_SLAB_FREELIST_RANDOM is turned on.
$ echo 0 > /sys/kernel/slab/zs_handle/store_user
$ echo 0 > /sys/kernel/slab/zspage/store_user
$ mkswap /dev/block/zram0
$ swapon /dev/block/zram0 -p 32758
=============================================================================
BUG zs_handle (Not tainted): Freepointer corrupt
-----------------------------------------------------------------------------
Disabling lock debugging due to kernel taint
INFO: Slab 0xffffffbf29603600 objects=102 used=102 fp=0x0000000000000000 flags=0x0200
INFO: Object 0xffffffca580d8d78 @offset=3448 fp=0xffffffca580d8ed0
Redzone 00000000f3cddd6c: bb bb bb bb bb bb bb bb ........
Object 0000000082d5d74e: 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkk.
Redzone 000000008fd80359: bb bb bb bb bb bb bb bb ........
Padding 00000000c7f56047: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ
In this example, an Android device tries to use zram as a swap and to
turn off store_user in order to reduce the slub object size.
When calculate_sizes is called in kmem_cache_open, size, order and
objects for zs_handle is:
size:360, order:0, objects:22
However, if the SLAB_STORE_USER bit is cleared in store_user_store:
size: 56, order:1, objects:73
All the size, order, and objects is changed by calculate_sizes(), but
the size of the random_seq array is still old value(22). As a result,
out-of-bound array access can occur at shuffle_freelist() when slab
allocation is requested.
This patch fixes the problem by re-allocating the random_seq array
with re-calculated correct objects value.
Fixes: 210e7a43fa905 ("mm: SLUB freelist randomization")
Reported-by: Ari-Pekka Verta <[email protected]>
Reported-by: Timo Simola <[email protected]>
Signed-off-by: Sahara <[email protected]>
---
mm/slub.c | 23 ++++++++++++++++-------
1 file changed, 16 insertions(+), 7 deletions(-)
diff --git a/mm/slub.c b/mm/slub.c
index f226d66408ee..be1e4d6682b8 100644
--- a/mm/slub.c
+++ b/mm/slub.c
@@ -3704,7 +3704,22 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)
if (oo_objects(s->oo) > oo_objects(s->max))
s->max = s->oo;
- return !!oo_objects(s->oo);
+ if (!oo_objects(s->oo))
+ return 0;
+
+ /*
+ * Initialize the pre-computed randomized freelist if slab is up.
+ * If the randomized freelist random_seq is already initialized,
+ * free and re-initialize it with re-calculated value.
+ */
+ if (slab_state >= UP) {
+ if (s->random_seq)
+ cache_random_seq_destroy(s);
+ if (init_cache_random_seq(s))
+ return 0;
+ }
+
+ return 1;
}
static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags)
@@ -3748,12 +3763,6 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags)
s->remote_node_defrag_ratio = 1000;
#endif
- /* Initialize the pre-computed randomized freelist if slab is up */
- if (slab_state >= UP) {
- if (init_cache_random_seq(s))
- goto error;
- }
-
if (!init_kmem_cache_nodes(s))
goto error;
--
2.17.1
(cc Thomas and linux-mm)
On Sat, 8 Aug 2020 13:50:30 +0400 [email protected] wrote:
> From: Sahara <[email protected]>
>
> Slab cache flags are exported to sysfs and are allowed to get modified
> from userspace. Some of those may call calculate_sizes function because
> the changed flag can take an effect on slab object size and layout,
> which means kmem_cache may have different order and objects.
> The freelist pointer corruption occurs if some slab flags are modified
> while CONFIG_SLAB_FREELIST_RANDOM is turned on.
>
> $ echo 0 > /sys/kernel/slab/zs_handle/store_user
> $ echo 0 > /sys/kernel/slab/zspage/store_user
> $ mkswap /dev/block/zram0
> $ swapon /dev/block/zram0 -p 32758
>
> =============================================================================
> BUG zs_handle (Not tainted): Freepointer corrupt
> -----------------------------------------------------------------------------
>
> Disabling lock debugging due to kernel taint
> INFO: Slab 0xffffffbf29603600 objects=102 used=102 fp=0x0000000000000000 flags=0x0200
> INFO: Object 0xffffffca580d8d78 @offset=3448 fp=0xffffffca580d8ed0
>
> Redzone 00000000f3cddd6c: bb bb bb bb bb bb bb bb ........
> Object 0000000082d5d74e: 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkk.
> Redzone 000000008fd80359: bb bb bb bb bb bb bb bb ........
> Padding 00000000c7f56047: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ
>
> In this example, an Android device tries to use zram as a swap and to
> turn off store_user in order to reduce the slub object size.
> When calculate_sizes is called in kmem_cache_open, size, order and
> objects for zs_handle is:
> size:360, order:0, objects:22
> However, if the SLAB_STORE_USER bit is cleared in store_user_store:
> size: 56, order:1, objects:73
>
> All the size, order, and objects is changed by calculate_sizes(), but
> the size of the random_seq array is still old value(22). As a result,
> out-of-bound array access can occur at shuffle_freelist() when slab
> allocation is requested.
>
> This patch fixes the problem by re-allocating the random_seq array
> with re-calculated correct objects value.
>
> Fixes: 210e7a43fa905 ("mm: SLUB freelist randomization")
> Reported-by: Ari-Pekka Verta <[email protected]>
> Reported-by: Timo Simola <[email protected]>
> Signed-off-by: Sahara <[email protected]>
> ---
> mm/slub.c | 23 ++++++++++++++++-------
> 1 file changed, 16 insertions(+), 7 deletions(-)
>
> diff --git a/mm/slub.c b/mm/slub.c
> index f226d66408ee..be1e4d6682b8 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -3704,7 +3704,22 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)
> if (oo_objects(s->oo) > oo_objects(s->max))
> s->max = s->oo;
>
> - return !!oo_objects(s->oo);
> + if (!oo_objects(s->oo))
> + return 0;
> +
> + /*
> + * Initialize the pre-computed randomized freelist if slab is up.
> + * If the randomized freelist random_seq is already initialized,
> + * free and re-initialize it with re-calculated value.
> + */
> + if (slab_state >= UP) {
> + if (s->random_seq)
> + cache_random_seq_destroy(s);
> + if (init_cache_random_seq(s))
> + return 0;
> + }
> +
> + return 1;
> }
>
> static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags)
> @@ -3748,12 +3763,6 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags)
> s->remote_node_defrag_ratio = 1000;
> #endif
>
> - /* Initialize the pre-computed randomized freelist if slab is up */
> - if (slab_state >= UP) {
> - if (init_cache_random_seq(s))
> - goto error;
> - }
> -
> if (!init_kmem_cache_nodes(s))
> goto error;
>
> --
> 2.17.1
On Sat, Aug 08, 2020 at 01:50:30PM +0400, [email protected] wrote:
> From: Sahara <[email protected]>
Hi!
>
> Slab cache flags are exported to sysfs and are allowed to get modified
> from userspace. Some of those may call calculate_sizes function because
> the changed flag can take an effect on slab object size and layout,
> which means kmem_cache may have different order and objects.
> The freelist pointer corruption occurs if some slab flags are modified
> while CONFIG_SLAB_FREELIST_RANDOM is turned on.
>
> $ echo 0 > /sys/kernel/slab/zs_handle/store_user
> $ echo 0 > /sys/kernel/slab/zspage/store_user
> $ mkswap /dev/block/zram0
> $ swapon /dev/block/zram0 -p 32758
>
> =============================================================================
> BUG zs_handle (Not tainted): Freepointer corrupt
The problems here are actually larger than just the freelist pointers,
so this was actually solved by just making these parameters not writable
at runtime:
https://git.kernel.org/linus/ad38b5b1131e2a0e5c46724847da2e1eba31fb68
I wonder if perhaps the above patch needs to be explicitly sent to the
-stable trees?
-Kees
> -----------------------------------------------------------------------------
>
> Disabling lock debugging due to kernel taint
> INFO: Slab 0xffffffbf29603600 objects=102 used=102 fp=0x0000000000000000 flags=0x0200
> INFO: Object 0xffffffca580d8d78 @offset=3448 fp=0xffffffca580d8ed0
>
> Redzone 00000000f3cddd6c: bb bb bb bb bb bb bb bb ........
> Object 0000000082d5d74e: 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkk.
> Redzone 000000008fd80359: bb bb bb bb bb bb bb bb ........
> Padding 00000000c7f56047: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ
>
> In this example, an Android device tries to use zram as a swap and to
> turn off store_user in order to reduce the slub object size.
> When calculate_sizes is called in kmem_cache_open, size, order and
> objects for zs_handle is:
> size:360, order:0, objects:22
> However, if the SLAB_STORE_USER bit is cleared in store_user_store:
> size: 56, order:1, objects:73
>
> All the size, order, and objects is changed by calculate_sizes(), but
> the size of the random_seq array is still old value(22). As a result,
> out-of-bound array access can occur at shuffle_freelist() when slab
> allocation is requested.
>
> This patch fixes the problem by re-allocating the random_seq array
> with re-calculated correct objects value.
>
> Fixes: 210e7a43fa905 ("mm: SLUB freelist randomization")
> Reported-by: Ari-Pekka Verta <[email protected]>
> Reported-by: Timo Simola <[email protected]>
> Signed-off-by: Sahara <[email protected]>
> ---
> mm/slub.c | 23 ++++++++++++++++-------
> 1 file changed, 16 insertions(+), 7 deletions(-)
>
> diff --git a/mm/slub.c b/mm/slub.c
> index f226d66408ee..be1e4d6682b8 100644
> --- a/mm/slub.c
> +++ b/mm/slub.c
> @@ -3704,7 +3704,22 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)
> if (oo_objects(s->oo) > oo_objects(s->max))
> s->max = s->oo;
>
> - return !!oo_objects(s->oo);
> + if (!oo_objects(s->oo))
> + return 0;
> +
> + /*
> + * Initialize the pre-computed randomized freelist if slab is up.
> + * If the randomized freelist random_seq is already initialized,
> + * free and re-initialize it with re-calculated value.
> + */
> + if (slab_state >= UP) {
> + if (s->random_seq)
> + cache_random_seq_destroy(s);
> + if (init_cache_random_seq(s))
> + return 0;
> + }
> +
> + return 1;
> }
>
> static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags)
> @@ -3748,12 +3763,6 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags)
> s->remote_node_defrag_ratio = 1000;
> #endif
>
> - /* Initialize the pre-computed randomized freelist if slab is up */
> - if (slab_state >= UP) {
> - if (init_cache_random_seq(s))
> - goto error;
> - }
> -
> if (!init_kmem_cache_nodes(s))
> goto error;
>
> --
> 2.17.1
>
--
Kees Cook
I am not super familiar with the sysfs setup here but the random
sequence should work as expected. One minor comment below.
Reviewed-by: Thomas Garnier <[email protected]>
On Wed, Aug 19, 2020 at 1:26 PM Andrew Morton <[email protected]> wrote:
>
>
> (cc Thomas and linux-mm)
>
> On Sat, 8 Aug 2020 13:50:30 +0400 [email protected] wrote:
>
> > From: Sahara <[email protected]>
> >
> > Slab cache flags are exported to sysfs and are allowed to get modified
> > from userspace. Some of those may call calculate_sizes function because
> > the changed flag can take an effect on slab object size and layout,
> > which means kmem_cache may have different order and objects.
> > The freelist pointer corruption occurs if some slab flags are modified
> > while CONFIG_SLAB_FREELIST_RANDOM is turned on.
> >
> > $ echo 0 > /sys/kernel/slab/zs_handle/store_user
> > $ echo 0 > /sys/kernel/slab/zspage/store_user
> > $ mkswap /dev/block/zram0
> > $ swapon /dev/block/zram0 -p 32758
> >
> > =============================================================================
> > BUG zs_handle (Not tainted): Freepointer corrupt
> > -----------------------------------------------------------------------------
> >
> > Disabling lock debugging due to kernel taint
> > INFO: Slab 0xffffffbf29603600 objects=102 used=102 fp=0x0000000000000000 flags=0x0200
> > INFO: Object 0xffffffca580d8d78 @offset=3448 fp=0xffffffca580d8ed0
> >
> > Redzone 00000000f3cddd6c: bb bb bb bb bb bb bb bb ........
> > Object 0000000082d5d74e: 6b 6b 6b 6b 6b 6b 6b a5 kkkkkkk.
> > Redzone 000000008fd80359: bb bb bb bb bb bb bb bb ........
> > Padding 00000000c7f56047: 5a 5a 5a 5a 5a 5a 5a 5a ZZZZZZZZ
> >
> > In this example, an Android device tries to use zram as a swap and to
> > turn off store_user in order to reduce the slub object size.
> > When calculate_sizes is called in kmem_cache_open, size, order and
> > objects for zs_handle is:
> > size:360, order:0, objects:22
> > However, if the SLAB_STORE_USER bit is cleared in store_user_store:
> > size: 56, order:1, objects:73
> >
> > All the size, order, and objects is changed by calculate_sizes(), but
> > the size of the random_seq array is still old value(22). As a result,
> > out-of-bound array access can occur at shuffle_freelist() when slab
> > allocation is requested.
> >
> > This patch fixes the problem by re-allocating the random_seq array
> > with re-calculated correct objects value.
> >
> > Fixes: 210e7a43fa905 ("mm: SLUB freelist randomization")
> > Reported-by: Ari-Pekka Verta <[email protected]>
> > Reported-by: Timo Simola <[email protected]>
> > Signed-off-by: Sahara <[email protected]>
> > ---
> > mm/slub.c | 23 ++++++++++++++++-------
> > 1 file changed, 16 insertions(+), 7 deletions(-)
> >
> > diff --git a/mm/slub.c b/mm/slub.c
> > index f226d66408ee..be1e4d6682b8 100644
> > --- a/mm/slub.c
> > +++ b/mm/slub.c
> > @@ -3704,7 +3704,22 @@ static int calculate_sizes(struct kmem_cache *s, int forced_order)
> > if (oo_objects(s->oo) > oo_objects(s->max))
> > s->max = s->oo;
> >
> > - return !!oo_objects(s->oo);
> > + if (!oo_objects(s->oo))
> > + return 0;
> > +
> > + /*
> > + * Initialize the pre-computed randomized freelist if slab is up.
> > + * If the randomized freelist random_seq is already initialized,
> > + * free and re-initialize it with re-calculated value.
> > + */
> > + if (slab_state >= UP) {
> > + if (s->random_seq)
> > + cache_random_seq_destroy(s);
kfree(NULL) is a noop, so you don't need to check s->random_seq.
> > + if (init_cache_random_seq(s))
> > + return 0;
> > + }
> > +
> > + return 1;
> > }
> >
> > static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags)
> > @@ -3748,12 +3763,6 @@ static int kmem_cache_open(struct kmem_cache *s, slab_flags_t flags)
> > s->remote_node_defrag_ratio = 1000;
> > #endif
> >
> > - /* Initialize the pre-computed randomized freelist if slab is up */
> > - if (slab_state >= UP) {
> > - if (init_cache_random_seq(s))
> > - goto error;
> > - }
> > -
> > if (!init_kmem_cache_nodes(s))
> > goto error;
> >
> > --
> > 2.17.1
--
Thomas