=== Overview
Hardware tag-based KASAN mode [1] is intended to eventually be used in
production as a security mitigation. Therefore there's a need for finer
control over KASAN features and for an existence of a kill switch.
This patchset adds a few boot parameters for hardware tag-based KASAN that
allow to disable or otherwise control particular KASAN features.
There's another planned patchset what will further optimize hardware
tag-based KASAN, provide proper benchmarking and tests, and will fully
enable tag-based KASAN for production use.
Hardware tag-based KASAN relies on arm64 Memory Tagging Extension (MTE)
[2] to perform memory and pointer tagging. Please see [3] and [4] for
detailed analysis of how MTE helps to fight memory safety problems.
The features that can be controlled are:
1. Whether KASAN is enabled at all.
2. Whether KASAN collects and saves alloc/free stacks.
3. Whether KASAN panics on a detected bug or not.
The patch titled "kasan: add and integrate kasan boot parameters" of this
series adds a few new boot parameters.
kasan.mode allows to choose one of three main modes:
- kasan.mode=off - KASAN is disabled, no tag checks are performed
- kasan.mode=prod - only essential production features are enabled
- kasan.mode=full - all KASAN features are enabled
The chosen mode provides default control values for the features mentioned
above. However it's also possible to override the default values by
providing:
- kasan.stack=off/on - enable stacks collection
(default: on for mode=full, otherwise off)
- kasan.fault=report/panic - only report tag fault or also panic
(default: report)
If kasan.mode parameter is not provided, it defaults to full when
CONFIG_DEBUG_KERNEL is enabled, and to prod otherwise.
It is essential that switching between these modes doesn't require
rebuilding the kernel with different configs, as this is required by
the Android GKI (Generic Kernel Image) initiative.
=== Benchmarks
For now I've only performed a few simple benchmarks such as measuring
kernel boot time and slab memory usage after boot. There's an upcoming
patchset which will optimize KASAN further and include more detailed
benchmarking results.
The benchmarks were performed in QEMU and the results below exclude the
slowdown caused by QEMU memory tagging emulation (as it's different from
the slowdown that will be introduced by hardware and is therefore
irrelevant).
KASAN_HW_TAGS=y + kasan.mode=off introduces no performance or memory
impact compared to KASAN_HW_TAGS=n.
kasan.mode=prod (manually excluding tagging) introduces 3% of performance
and no memory impact (except memory used by hardware to store tags)
compared to kasan.mode=off.
kasan.mode=full has about 40% performance and 30% memory impact over
kasan.mode=prod. Both come from alloc/free stack collection.
=== Notes
This patchset is available here:
https://github.com/xairy/linux/tree/up-boot-mte-v1
and on Gerrit here:
https://linux-review.googlesource.com/c/linux/kernel/git/torvalds/linux/+/3707
This patchset is based on v8 of "kasan: add hardware tag-based mode for
arm64" patchset [1].
For testing in QEMU hardware tag-based KASAN requires:
1. QEMU built from master [6] (use "-machine virt,mte=on -cpu max" arguments
to run).
2. GCC version 10.
[1] https://lkml.org/lkml/2020/11/4/1208
[2] https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/enhancing-memory-safety
[3] https://arxiv.org/pdf/1802.09517.pdf
[4] https://github.com/microsoft/MSRC-Security-Research/blob/master/papers/2020/Security%20analysis%20of%20memory%20tagging.pdf
[5] https://source.android.com/devices/architecture/kernel/generic-kernel-image
[6] https://github.com/qemu/qemu
=== History
Changes RFC v2 -> v1:
- Rebrand the patchset from fully enabling production use to partially
addressing that; another optimization and testing patchset will be
required.
- Rebase onto v8 of KASAN_HW_TAGS series.
- Fix "ASYNC" -> "async" typo.
- Rework depends condition for VMAP_STACK and update config text.
- Remove unneeded reset_tag() macro, use kasan_reset_tag() instead.
- Rename kasan.stack to kasan.stacks to avoid confusion with stack
instrumentation.
- Introduce kasan_stack_collection_enabled() and kasan_is_enabled()
helpers.
- Simplify kasan_stack_collection_enabled() usage.
- Rework SLAB_KASAN flag and metadata allocation (see the corresponding
patch for details).
- Allow cache merging with KASAN_HW_TAGS when kasan.stacks is off.
- Use sync mode dy default for both prod and full KASAN modes.
- Drop kasan.trap=sync/async boot parameter, as async mode isn't supported
yet.
- Choose prod or full mode depending on CONFIG_DEBUG_KERNEL when no
kasan.mode boot parameter is provided.
- Drop krealloc optimization changes, those will be included in a separate
patchset.
- Update KASAN documentation to mention boot parameters.
Changes RFC v1 -> RFC v2:
- Rework boot parameters.
- Drop __init from empty kasan_init_tags() definition.
- Add cpu_supports_mte() helper that can be used during early boot and use
it in kasan_init_tags()
- Lots of new KASAN optimization commits.
Andrey Konovalov (20):
kasan: simplify quarantine_put call site
kasan: rename get_alloc/free_info
kasan: introduce set_alloc_info
kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK
kasan: allow VMAP_STACK for HW_TAGS mode
kasan: remove __kasan_unpoison_stack
kasan: inline kasan_reset_tag for tag-based modes
kasan: inline random_tag for HW_TAGS
kasan: inline kasan_poison_memory and check_invalid_free
kasan: inline and rename kasan_unpoison_memory
kasan: add and integrate kasan boot parameters
kasan, mm: check kasan_enabled in annotations
kasan: simplify kasan_poison_kfree
kasan, mm: rename kasan_poison_kfree
kasan: don't round_up too much
kasan: simplify assign_tag and set_tag calls
kasan: clarify comment in __kasan_kfree_large
kasan: clean up metadata allocation and usage
kasan, mm: allow cache merging with no metadata
kasan: update documentation
Documentation/dev-tools/kasan.rst | 180 ++++++++++++--------
arch/Kconfig | 8 +-
arch/arm64/kernel/sleep.S | 2 +-
arch/x86/kernel/acpi/wakeup_64.S | 2 +-
include/linux/kasan.h | 253 +++++++++++++++++++++------
include/linux/mm.h | 22 ++-
kernel/fork.c | 2 +-
mm/kasan/common.c | 274 ++++++++++++++++++------------
mm/kasan/generic.c | 27 +--
mm/kasan/hw_tags.c | 171 ++++++++++++++++---
mm/kasan/kasan.h | 113 ++++++++----
mm/kasan/quarantine.c | 13 +-
mm/kasan/report.c | 61 ++++---
mm/kasan/report_hw_tags.c | 2 +-
mm/kasan/report_sw_tags.c | 13 +-
mm/kasan/shadow.c | 5 +-
mm/kasan/sw_tags.c | 17 +-
mm/mempool.c | 2 +-
mm/slab_common.c | 13 +-
19 files changed, 816 insertions(+), 364 deletions(-)
--
2.29.1.341.ge80a0c044ae-goog
Using kasan_poison_memory() or check_invalid_free() currently results in
function calls. Move their definitions to mm/kasan/kasan.h and turn them
into static inline functions for hardware tag-based mode to avoid
unneeded function calls.
Signed-off-by: Andrey Konovalov <[email protected]>
Reviewed-by: Dmitry Vyukov <[email protected]>
Link: https://linux-review.googlesource.com/id/Ia9d8191024a12d1374675b3d27197f10193f50bb
---
mm/kasan/hw_tags.c | 15 ---------------
mm/kasan/kasan.h | 28 ++++++++++++++++++++++++----
2 files changed, 24 insertions(+), 19 deletions(-)
diff --git a/mm/kasan/hw_tags.c b/mm/kasan/hw_tags.c
index d5824530fd15..9d7b1f1a2553 100644
--- a/mm/kasan/hw_tags.c
+++ b/mm/kasan/hw_tags.c
@@ -26,27 +26,12 @@ void kasan_init_hw_tags(void)
pr_info("KernelAddressSanitizer initialized\n");
}
-void kasan_poison_memory(const void *address, size_t size, u8 value)
-{
- hw_set_mem_tag_range(kasan_reset_tag(address),
- round_up(size, KASAN_GRANULE_SIZE), value);
-}
-
void kasan_unpoison_memory(const void *address, size_t size)
{
hw_set_mem_tag_range(kasan_reset_tag(address),
round_up(size, KASAN_GRANULE_SIZE), get_tag(address));
}
-bool check_invalid_free(void *addr)
-{
- u8 ptr_tag = get_tag(addr);
- u8 mem_tag = hw_get_mem_tag(addr);
-
- return (mem_tag == KASAN_TAG_INVALID) ||
- (ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag);
-}
-
void kasan_set_free_info(struct kmem_cache *cache,
void *object, u8 tag)
{
diff --git a/mm/kasan/kasan.h b/mm/kasan/kasan.h
index d7a03eab5814..73364acf6ec8 100644
--- a/mm/kasan/kasan.h
+++ b/mm/kasan/kasan.h
@@ -153,8 +153,6 @@ struct kasan_alloc_meta *kasan_get_alloc_meta(struct kmem_cache *cache,
struct kasan_free_meta *kasan_get_free_meta(struct kmem_cache *cache,
const void *object);
-void kasan_poison_memory(const void *address, size_t size, u8 value);
-
#if defined(CONFIG_KASAN_GENERIC) || defined(CONFIG_KASAN_SW_TAGS)
static inline const void *kasan_shadow_to_mem(const void *shadow_addr)
@@ -194,8 +192,6 @@ void print_tags(u8 addr_tag, const void *addr);
static inline void print_tags(u8 addr_tag, const void *addr) { }
#endif
-bool check_invalid_free(void *addr);
-
void *find_first_bad_addr(void *addr, size_t size);
const char *get_bug_type(struct kasan_access_info *info);
void metadata_fetch_row(char *buffer, void *row);
@@ -275,6 +271,30 @@ static inline u8 random_tag(void)
}
#endif
+#ifdef CONFIG_KASAN_HW_TAGS
+
+static inline void kasan_poison_memory(const void *address, size_t size, u8 value)
+{
+ hw_set_mem_tag_range(kasan_reset_tag(address),
+ round_up(size, KASAN_GRANULE_SIZE), value);
+}
+
+static inline bool check_invalid_free(void *addr)
+{
+ u8 ptr_tag = get_tag(addr);
+ u8 mem_tag = hw_get_mem_tag(addr);
+
+ return (mem_tag == KASAN_TAG_INVALID) ||
+ (ptr_tag != KASAN_TAG_KERNEL && ptr_tag != mem_tag);
+}
+
+#else /* CONFIG_KASAN_HW_TAGS */
+
+void kasan_poison_memory(const void *address, size_t size, u8 value);
+bool check_invalid_free(void *addr);
+
+#endif /* CONFIG_KASAN_HW_TAGS */
+
/*
* Exported functions for interfaces called from assembly or from generated
* code. Declarations here to avoid warning about missing declarations.
--
2.29.1.341.ge80a0c044ae-goog
Add set_alloc_info() helper and move kasan_set_track() into it. This will
simplify the code for one of the upcoming changes.
No functional changes.
Signed-off-by: Andrey Konovalov <[email protected]>
Reviewed-by: Dmitry Vyukov <[email protected]>
Link: https://linux-review.googlesource.com/id/I0316193cbb4ecc9b87b7c2eee0dd79f8ec908c1a
---
mm/kasan/common.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/mm/kasan/common.c b/mm/kasan/common.c
index 8fd04415d8f4..a880e5a547ed 100644
--- a/mm/kasan/common.c
+++ b/mm/kasan/common.c
@@ -318,6 +318,11 @@ bool kasan_slab_free(struct kmem_cache *cache, void *object, unsigned long ip)
return __kasan_slab_free(cache, object, ip, true);
}
+static void set_alloc_info(struct kmem_cache *cache, void *object, gfp_t flags)
+{
+ kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags);
+}
+
static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object,
size_t size, gfp_t flags, bool keep_tag)
{
@@ -345,7 +350,7 @@ static void *__kasan_kmalloc(struct kmem_cache *cache, const void *object,
KASAN_KMALLOC_REDZONE);
if (cache->flags & SLAB_KASAN)
- kasan_set_track(&kasan_get_alloc_meta(cache, object)->alloc_track, flags);
+ set_alloc_info(cache, (void *)object, flags);
return set_tag(object, tag);
}
--
2.29.1.341.ge80a0c044ae-goog
On Wed, Nov 4, 2020 at 4:02 PM Andrey Konovalov <[email protected]> wrote:
>
> === Overview
>
> Hardware tag-based KASAN mode [1] is intended to eventually be used in
> production as a security mitigation. Therefore there's a need for finer
> control over KASAN features and for an existence of a kill switch.
>
> This patchset adds a few boot parameters for hardware tag-based KASAN that
> allow to disable or otherwise control particular KASAN features.
>
> There's another planned patchset what will further optimize hardware
> tag-based KASAN, provide proper benchmarking and tests, and will fully
> enable tag-based KASAN for production use.
>
> Hardware tag-based KASAN relies on arm64 Memory Tagging Extension (MTE)
> [2] to perform memory and pointer tagging. Please see [3] and [4] for
> detailed analysis of how MTE helps to fight memory safety problems.
>
> The features that can be controlled are:
>
> 1. Whether KASAN is enabled at all.
> 2. Whether KASAN collects and saves alloc/free stacks.
> 3. Whether KASAN panics on a detected bug or not.
>
> The patch titled "kasan: add and integrate kasan boot parameters" of this
> series adds a few new boot parameters.
>
> kasan.mode allows to choose one of three main modes:
>
> - kasan.mode=off - KASAN is disabled, no tag checks are performed
> - kasan.mode=prod - only essential production features are enabled
> - kasan.mode=full - all KASAN features are enabled
>
> The chosen mode provides default control values for the features mentioned
> above. However it's also possible to override the default values by
> providing:
>
> - kasan.stack=off/on - enable stacks collection
> (default: on for mode=full, otherwise off)
I think this was discussed before, but should this be kasan.stacktrace
or something like that?
In other places "kasan stack" refers to stack instrumentation, not
stack trace collection.
Ex.: CONFIG_KASAN_STACK
> - kasan.fault=report/panic - only report tag fault or also panic
> (default: report)
>
> If kasan.mode parameter is not provided, it defaults to full when
> CONFIG_DEBUG_KERNEL is enabled, and to prod otherwise.
>
> It is essential that switching between these modes doesn't require
> rebuilding the kernel with different configs, as this is required by
> the Android GKI (Generic Kernel Image) initiative.
>
> === Benchmarks
>
> For now I've only performed a few simple benchmarks such as measuring
> kernel boot time and slab memory usage after boot. There's an upcoming
> patchset which will optimize KASAN further and include more detailed
> benchmarking results.
>
> The benchmarks were performed in QEMU and the results below exclude the
> slowdown caused by QEMU memory tagging emulation (as it's different from
> the slowdown that will be introduced by hardware and is therefore
> irrelevant).
>
> KASAN_HW_TAGS=y + kasan.mode=off introduces no performance or memory
> impact compared to KASAN_HW_TAGS=n.
>
> kasan.mode=prod (manually excluding tagging) introduces 3% of performance
> and no memory impact (except memory used by hardware to store tags)
> compared to kasan.mode=off.
>
> kasan.mode=full has about 40% performance and 30% memory impact over
> kasan.mode=prod. Both come from alloc/free stack collection.
>
> === Notes
>
> This patchset is available here:
>
> https://github.com/xairy/linux/tree/up-boot-mte-v1
>
> and on Gerrit here:
>
> https://linux-review.googlesource.com/c/linux/kernel/git/torvalds/linux/+/3707
>
> This patchset is based on v8 of "kasan: add hardware tag-based mode for
> arm64" patchset [1].
>
> For testing in QEMU hardware tag-based KASAN requires:
>
> 1. QEMU built from master [6] (use "-machine virt,mte=on -cpu max" arguments
> to run).
> 2. GCC version 10.
>
> [1] https://lkml.org/lkml/2020/11/4/1208
> [2] https://community.arm.com/developer/ip-products/processors/b/processors-ip-blog/posts/enhancing-memory-safety
> [3] https://arxiv.org/pdf/1802.09517.pdf
> [4] https://github.com/microsoft/MSRC-Security-Research/blob/master/papers/2020/Security%20analysis%20of%20memory%20tagging.pdf
> [5] https://source.android.com/devices/architecture/kernel/generic-kernel-image
> [6] https://github.com/qemu/qemu
>
> === History
>
> Changes RFC v2 -> v1:
> - Rebrand the patchset from fully enabling production use to partially
> addressing that; another optimization and testing patchset will be
> required.
> - Rebase onto v8 of KASAN_HW_TAGS series.
> - Fix "ASYNC" -> "async" typo.
> - Rework depends condition for VMAP_STACK and update config text.
> - Remove unneeded reset_tag() macro, use kasan_reset_tag() instead.
> - Rename kasan.stack to kasan.stacks to avoid confusion with stack
> instrumentation.
> - Introduce kasan_stack_collection_enabled() and kasan_is_enabled()
> helpers.
> - Simplify kasan_stack_collection_enabled() usage.
> - Rework SLAB_KASAN flag and metadata allocation (see the corresponding
> patch for details).
> - Allow cache merging with KASAN_HW_TAGS when kasan.stacks is off.
> - Use sync mode dy default for both prod and full KASAN modes.
> - Drop kasan.trap=sync/async boot parameter, as async mode isn't supported
> yet.
> - Choose prod or full mode depending on CONFIG_DEBUG_KERNEL when no
> kasan.mode boot parameter is provided.
> - Drop krealloc optimization changes, those will be included in a separate
> patchset.
> - Update KASAN documentation to mention boot parameters.
>
> Changes RFC v1 -> RFC v2:
> - Rework boot parameters.
> - Drop __init from empty kasan_init_tags() definition.
> - Add cpu_supports_mte() helper that can be used during early boot and use
> it in kasan_init_tags()
> - Lots of new KASAN optimization commits.
>
> Andrey Konovalov (20):
> kasan: simplify quarantine_put call site
> kasan: rename get_alloc/free_info
> kasan: introduce set_alloc_info
> kasan, arm64: unpoison stack only with CONFIG_KASAN_STACK
> kasan: allow VMAP_STACK for HW_TAGS mode
> kasan: remove __kasan_unpoison_stack
> kasan: inline kasan_reset_tag for tag-based modes
> kasan: inline random_tag for HW_TAGS
> kasan: inline kasan_poison_memory and check_invalid_free
> kasan: inline and rename kasan_unpoison_memory
> kasan: add and integrate kasan boot parameters
> kasan, mm: check kasan_enabled in annotations
> kasan: simplify kasan_poison_kfree
> kasan, mm: rename kasan_poison_kfree
> kasan: don't round_up too much
> kasan: simplify assign_tag and set_tag calls
> kasan: clarify comment in __kasan_kfree_large
> kasan: clean up metadata allocation and usage
> kasan, mm: allow cache merging with no metadata
> kasan: update documentation
>
> Documentation/dev-tools/kasan.rst | 180 ++++++++++++--------
> arch/Kconfig | 8 +-
> arch/arm64/kernel/sleep.S | 2 +-
> arch/x86/kernel/acpi/wakeup_64.S | 2 +-
> include/linux/kasan.h | 253 +++++++++++++++++++++------
> include/linux/mm.h | 22 ++-
> kernel/fork.c | 2 +-
> mm/kasan/common.c | 274 ++++++++++++++++++------------
> mm/kasan/generic.c | 27 +--
> mm/kasan/hw_tags.c | 171 ++++++++++++++++---
> mm/kasan/kasan.h | 113 ++++++++----
> mm/kasan/quarantine.c | 13 +-
> mm/kasan/report.c | 61 ++++---
> mm/kasan/report_hw_tags.c | 2 +-
> mm/kasan/report_sw_tags.c | 13 +-
> mm/kasan/shadow.c | 5 +-
> mm/kasan/sw_tags.c | 17 +-
> mm/mempool.c | 2 +-
> mm/slab_common.c | 13 +-
> 19 files changed, 816 insertions(+), 364 deletions(-)
>
> --
> 2.29.1.341.ge80a0c044ae-goog
>
On Thu, Nov 5, 2020 at 9:49 PM Evgenii Stepanov <[email protected]> wrote:
>
> > The chosen mode provides default control values for the features mentioned
> > above. However it's also possible to override the default values by
> > providing:
> >
> > - kasan.stack=off/on - enable stacks collection
> > (default: on for mode=full, otherwise off)
>
> I think this was discussed before, but should this be kasan.stacktrace
> or something like that?
> In other places "kasan stack" refers to stack instrumentation, not
> stack trace collection.
> Ex.: CONFIG_KASAN_STACK
Forgot to update it here, but it's kasan.stacks now (with an s at the
end). kasan.stacktrace might be better, although it's somewhat long.
WDYT?
On Thu, Nov 5, 2020 at 12:55 PM Andrey Konovalov <[email protected]> wrote:
>
> On Thu, Nov 5, 2020 at 9:49 PM Evgenii Stepanov <[email protected]> wrote:
> >
> > > The chosen mode provides default control values for the features mentioned
> > > above. However it's also possible to override the default values by
> > > providing:
> > >
> > > - kasan.stack=off/on - enable stacks collection
> > > (default: on for mode=full, otherwise off)
> >
> > I think this was discussed before, but should this be kasan.stacktrace
> > or something like that?
> > In other places "kasan stack" refers to stack instrumentation, not
> > stack trace collection.
> > Ex.: CONFIG_KASAN_STACK
>
> Forgot to update it here, but it's kasan.stacks now (with an s at the
> end). kasan.stacktrace might be better, although it's somewhat long.
> WDYT?
I like kasan.stacktrace, but I would not insist.