2022-08-28 14:18:41

by Xianting Tian

[permalink] [raw]
Subject: [PATCH] RISC-V: Add STACKLEAK erasing the kernel stack at the end of syscalls

This adds support for the STACKLEAK gcc plugin to RISC-V and disables
the plugin in EFI stub code, which is out of scope for the protection.

For the benefits of STACKLEAK feature, please check the commit
afaef01c0015 ("x86/entry: Add STACKLEAK erasing the kernel stack at the end of syscalls")

Performance impact (tested on qemu env with 1 riscv64 hart, 1GB mem)
hackbench -s 512 -l 200 -g 15 -f 25 -P
2.0% slowdown

Signed-off-by: Xianting Tian <[email protected]>
---
arch/riscv/Kconfig | 1 +
arch/riscv/include/asm/processor.h | 4 ++++
arch/riscv/kernel/entry.S | 3 +++
drivers/firmware/efi/libstub/Makefile | 2 +-
4 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
index ed66c31e4655..61fd0dad4463 100644
--- a/arch/riscv/Kconfig
+++ b/arch/riscv/Kconfig
@@ -85,6 +85,7 @@ config RISCV
select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
select HAVE_ARCH_THREAD_STRUCT_WHITELIST
select HAVE_ARCH_VMAP_STACK if MMU && 64BIT
+ select HAVE_ARCH_STACKLEAK
select HAVE_ASM_MODVERSIONS
select HAVE_CONTEXT_TRACKING_USER
select HAVE_DEBUG_KMEMLEAK
diff --git a/arch/riscv/include/asm/processor.h b/arch/riscv/include/asm/processor.h
index 19eedd4af4cd..75620c467d25 100644
--- a/arch/riscv/include/asm/processor.h
+++ b/arch/riscv/include/asm/processor.h
@@ -85,6 +85,10 @@ int riscv_of_parent_hartid(struct device_node *node, unsigned long *hartid);
extern void riscv_fill_hwcap(void);
extern int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src);

+#define current_top_of_stack() ((unsigned long)current->stack + THREAD_SIZE)
+#define on_thread_stack() ((unsigned long)((current_top_of_stack() \
+ - current_stack_pointer) < THREAD_SIZE))
+
#endif /* __ASSEMBLY__ */

#endif /* _ASM_RISCV_PROCESSOR_H */
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index b9eda3fcbd6d..5f98660b46d4 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -233,6 +233,9 @@ check_syscall_nr:
ret_from_syscall:
/* Set user a0 to kernel a0 */
REG_S a0, PT_A0(sp)
+#ifdef CONFIG_GCC_PLUGIN_STACKLEAK
+ call stackleak_erase_on_task_stack
+#endif
/*
* We didn't execute the actual syscall.
* Seccomp already set return value for the current task pt_regs.
diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
index d0537573501e..5e1fc4f82883 100644
--- a/drivers/firmware/efi/libstub/Makefile
+++ b/drivers/firmware/efi/libstub/Makefile
@@ -25,7 +25,7 @@ cflags-$(CONFIG_ARM) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
-fno-builtin -fpic \
$(call cc-option,-mno-single-pic-base)
cflags-$(CONFIG_RISCV) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
- -fpic
+ -fpic $(DISABLE_STACKLEAK_PLUGIN)

cflags-$(CONFIG_EFI_GENERIC_STUB) += -I$(srctree)/scripts/dtc/libfdt

--
2.17.1


2022-09-06 18:34:23

by Conor Dooley

[permalink] [raw]
Subject: Re: [PATCH] RISC-V: Add STACKLEAK erasing the kernel stack at the end of syscalls

On 03/09/2022 17:23, [email protected] wrote:
> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
>
> From: Xianting Tian <[email protected]>
>
> This adds support for the STACKLEAK gcc plugin to RISC-V and disables
> the plugin in EFI stub code, which is out of scope for the protection.
>
> For the benefits of STACKLEAK feature, please check the commit
> afaef01c0015 ("x86/entry: Add STACKLEAK erasing the kernel stack at the end of syscalls")
>
> Performance impact (tested on qemu env with 1 riscv64 hart, 1GB mem)
> hackbench -s 512 -l 200 -g 15 -f 25 -P
> 2.0% slowdown
>
> Signed-off-by: Xianting Tian <[email protected]>

What changed since Xianting posted it himself a week ago:
https://lore.kernel.org/linux-riscv/[email protected]/

There's an older patch from Du Lao adding STACKLEAK too:
https://lore.kernel.org/linux-riscv/[email protected]/

But since there's been no activity there since June...

> ---
> arch/riscv/Kconfig | 1 +
> arch/riscv/include/asm/processor.h | 4 ++++
> arch/riscv/kernel/entry.S | 3 +++
> drivers/firmware/efi/libstub/Makefile | 2 +-
> 4 files changed, 9 insertions(+), 1 deletion(-)
>
> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> index ed66c31e4655..61fd0dad4463 100644
> --- a/arch/riscv/Kconfig
> +++ b/arch/riscv/Kconfig
> @@ -85,6 +85,7 @@ config RISCV
> select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
> select HAVE_ARCH_THREAD_STRUCT_WHITELIST
> select HAVE_ARCH_VMAP_STACK if MMU && 64BIT
> + select HAVE_ARCH_STACKLEAK
> select HAVE_ASM_MODVERSIONS
> select HAVE_CONTEXT_TRACKING_USER
> select HAVE_DEBUG_KMEMLEAK
> diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
> index d0537573501e..5e1fc4f82883 100644
> --- a/drivers/firmware/efi/libstub/Makefile
> +++ b/drivers/firmware/efi/libstub/Makefile
> @@ -25,7 +25,7 @@ cflags-$(CONFIG_ARM) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
> -fno-builtin -fpic \
> $(call cc-option,-mno-single-pic-base)
> cflags-$(CONFIG_RISCV) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
> - -fpic
> + -fpic $(DISABLE_STACKLEAK_PLUGIN)
>
> cflags-$(CONFIG_EFI_GENERIC_STUB) += -I$(srctree)/scripts/dtc/libfdt
>
> --
> 2.17.1
>
>
> _______________________________________________
> linux-riscv mailing list
> [email protected]
> http://lists.infradead.org/mailman/listinfo/linux-riscv

2022-09-07 02:23:21

by Guo Ren

[permalink] [raw]
Subject: Re: [PATCH] RISC-V: Add STACKLEAK erasing the kernel stack at the end of syscalls

Hi all,

How about the generic_entry version:

https://lore.kernel.org/lkml/[email protected]/

On Wed, Sep 7, 2022 at 1:35 AM <[email protected]> wrote:
>
> On 03/09/2022 17:23, [email protected] wrote:
> > EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> >
> > From: Xianting Tian <[email protected]>
> >
> > This adds support for the STACKLEAK gcc plugin to RISC-V and disables
> > the plugin in EFI stub code, which is out of scope for the protection.
> >
> > For the benefits of STACKLEAK feature, please check the commit
> > afaef01c0015 ("x86/entry: Add STACKLEAK erasing the kernel stack at the end of syscalls")
> >
> > Performance impact (tested on qemu env with 1 riscv64 hart, 1GB mem)
> > hackbench -s 512 -l 200 -g 15 -f 25 -P
> > 2.0% slowdown
> >
> > Signed-off-by: Xianting Tian <[email protected]>
>
> What changed since Xianting posted it himself a week ago:
> https://lore.kernel.org/linux-riscv/[email protected]/
>
> There's an older patch from Du Lao adding STACKLEAK too:
> https://lore.kernel.org/linux-riscv/[email protected]/
>
> But since there's been no activity there since June...
>
> > ---
> > arch/riscv/Kconfig | 1 +
> > arch/riscv/include/asm/processor.h | 4 ++++
> > arch/riscv/kernel/entry.S | 3 +++
> > drivers/firmware/efi/libstub/Makefile | 2 +-
> > 4 files changed, 9 insertions(+), 1 deletion(-)
> >
> > diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> > index ed66c31e4655..61fd0dad4463 100644
> > --- a/arch/riscv/Kconfig
> > +++ b/arch/riscv/Kconfig
> > @@ -85,6 +85,7 @@ config RISCV
> > select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
> > select HAVE_ARCH_THREAD_STRUCT_WHITELIST
> > select HAVE_ARCH_VMAP_STACK if MMU && 64BIT
> > + select HAVE_ARCH_STACKLEAK
> > select HAVE_ASM_MODVERSIONS
> > select HAVE_CONTEXT_TRACKING_USER
> > select HAVE_DEBUG_KMEMLEAK
> > diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
> > index d0537573501e..5e1fc4f82883 100644
> > --- a/drivers/firmware/efi/libstub/Makefile
> > +++ b/drivers/firmware/efi/libstub/Makefile
> > @@ -25,7 +25,7 @@ cflags-$(CONFIG_ARM) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
> > -fno-builtin -fpic \
> > $(call cc-option,-mno-single-pic-base)
> > cflags-$(CONFIG_RISCV) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
> > - -fpic
> > + -fpic $(DISABLE_STACKLEAK_PLUGIN)
> >
> > cflags-$(CONFIG_EFI_GENERIC_STUB) += -I$(srctree)/scripts/dtc/libfdt
> >
> > --
> > 2.17.1
> >
> >
> > _______________________________________________
> > linux-riscv mailing list
> > [email protected]
> > http://lists.infradead.org/mailman/listinfo/linux-riscv
>


--
Best Regards
Guo Ren

2022-10-07 03:01:52

by Palmer Dabbelt

[permalink] [raw]
Subject: Re: [PATCH] RISC-V: Add STACKLEAK erasing the kernel stack at the end of syscalls

On Tue, 06 Sep 2022 10:35:10 PDT (-0700), [email protected] wrote:
> On 03/09/2022 17:23, [email protected] wrote:
>> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
>>
>> From: Xianting Tian <[email protected]>
>>
>> This adds support for the STACKLEAK gcc plugin to RISC-V and disables
>> the plugin in EFI stub code, which is out of scope for the protection.
>>
>> For the benefits of STACKLEAK feature, please check the commit
>> afaef01c0015 ("x86/entry: Add STACKLEAK erasing the kernel stack at the end of syscalls")
>>
>> Performance impact (tested on qemu env with 1 riscv64 hart, 1GB mem)
>> hackbench -s 512 -l 200 -g 15 -f 25 -P
>> 2.0% slowdown
>>
>> Signed-off-by: Xianting Tian <[email protected]>
>
> What changed since Xianting posted it himself a week ago:
> https://lore.kernel.org/linux-riscv/[email protected]/
>
> There's an older patch from Du Lao adding STACKLEAK too:
> https://lore.kernel.org/linux-riscv/[email protected]/
>
> But since there's been no activity there since June...

Looks like the only issues were some commit log wording stuff, and that
there's a test suite that should be run. It's not clear from the
commits that anyone has done that, I'm fine with the patch if it passes
the tests but don't really know how to run them.

Has anyone run the tests?

>
>> ---
>> arch/riscv/Kconfig | 1 +
>> arch/riscv/include/asm/processor.h | 4 ++++
>> arch/riscv/kernel/entry.S | 3 +++
>> drivers/firmware/efi/libstub/Makefile | 2 +-
>> 4 files changed, 9 insertions(+), 1 deletion(-)
>>
>> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
>> index ed66c31e4655..61fd0dad4463 100644
>> --- a/arch/riscv/Kconfig
>> +++ b/arch/riscv/Kconfig
>> @@ -85,6 +85,7 @@ config RISCV
>> select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
>> select HAVE_ARCH_THREAD_STRUCT_WHITELIST
>> select HAVE_ARCH_VMAP_STACK if MMU && 64BIT
>> + select HAVE_ARCH_STACKLEAK
>> select HAVE_ASM_MODVERSIONS
>> select HAVE_CONTEXT_TRACKING_USER
>> select HAVE_DEBUG_KMEMLEAK
>> diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
>> index d0537573501e..5e1fc4f82883 100644
>> --- a/drivers/firmware/efi/libstub/Makefile
>> +++ b/drivers/firmware/efi/libstub/Makefile
>> @@ -25,7 +25,7 @@ cflags-$(CONFIG_ARM) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
>> -fno-builtin -fpic \
>> $(call cc-option,-mno-single-pic-base)
>> cflags-$(CONFIG_RISCV) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
>> - -fpic
>> + -fpic $(DISABLE_STACKLEAK_PLUGIN)
>>
>> cflags-$(CONFIG_EFI_GENERIC_STUB) += -I$(srctree)/scripts/dtc/libfdt
>>
>> --
>> 2.17.1
>>
>>
>> _______________________________________________
>> linux-riscv mailing list
>> [email protected]
>> http://lists.infradead.org/mailman/listinfo/linux-riscv
>

2022-10-07 11:48:33

by Mark Rutland

[permalink] [raw]
Subject: Re: [PATCH] RISC-V: Add STACKLEAK erasing the kernel stack at the end of syscalls

On Thu, Oct 06, 2022 at 07:31:01PM -0700, Palmer Dabbelt wrote:
> On Tue, 06 Sep 2022 10:35:10 PDT (-0700), [email protected] wrote:
> > On 03/09/2022 17:23, [email protected] wrote:
> > > EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> > >
> > > From: Xianting Tian <[email protected]>
> > >
> > > This adds support for the STACKLEAK gcc plugin to RISC-V and disables
> > > the plugin in EFI stub code, which is out of scope for the protection.
> > >
> > > For the benefits of STACKLEAK feature, please check the commit
> > > afaef01c0015 ("x86/entry: Add STACKLEAK erasing the kernel stack at the end of syscalls")
> > >
> > > Performance impact (tested on qemu env with 1 riscv64 hart, 1GB mem)
> > > hackbench -s 512 -l 200 -g 15 -f 25 -P
> > > 2.0% slowdown
> > >
> > > Signed-off-by: Xianting Tian <[email protected]>
> >
> > What changed since Xianting posted it himself a week ago:
> > https://lore.kernel.org/linux-riscv/[email protected]/
> >
> > There's an older patch from Du Lao adding STACKLEAK too:
> > https://lore.kernel.org/linux-riscv/[email protected]/
> >
> > But since there's been no activity there since June...
>
> Looks like the only issues were some commit log wording stuff, and that
> there's a test suite that should be run. It's not clear from the commits
> that anyone has done that, I'm fine with the patch if it passes the tests
> but don't really know how to run them.

Enable CONFIG_LKDTM, and do:

echo STACKLEAK_ERASING > /sys/kernel/debug/provoke-crash/DIRECT

Example GOOD/BAD output below, taken from:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/commit/drivers/misc/lkdtm/stackleak.c?id=72b61896f2b47fa4b98e86184bc0e6ddbd1a8db1

GOOD result on x86_64:

| # echo STACKLEAK_ERASING > /sys/kernel/debug/provoke-crash/DIRECT
| lkdtm: Performing direct entry STACKLEAK_ERASING
| lkdtm: stackleak stack usage:
| high offset: 168 bytes
| current: 336 bytes
| lowest: 656 bytes
| tracked: 656 bytes
| untracked: 400 bytes
| poisoned: 15152 bytes
| low offset: 8 bytes
| lkdtm: OK: the rest of the thread stack is properly erased

GOOD result on arm64:

| # echo STACKLEAK_ERASING > /sys/kernel/debug/provoke-crash/DIRECT
| lkdtm: Performing direct entry STACKLEAK_ERASING
| lkdtm: stackleak stack usage:
| high offset: 336 bytes
| current: 656 bytes
| lowest: 1232 bytes
| tracked: 1232 bytes
| untracked: 672 bytes
| poisoned: 14136 bytes
| low offset: 8 bytes
| lkdtm: OK: the rest of the thread stack is properly erased

BAD result on arm64:

| # echo STACKLEAK_ERASING > /sys/kernel/debug/provoke-crash/DIRECT
| lkdtm: Performing direct entry STACKLEAK_ERASING
| lkdtm: FAIL: non-poison value 24 bytes below poison boundary: 0x0
| lkdtm: FAIL: non-poison value 32 bytes below poison boundary: 0xffff8000083dbc00
...
| lkdtm: FAIL: non-poison value 1912 bytes below poison boundary: 0x78b4b9999e8cb15
| lkdtm: FAIL: non-poison value 1920 bytes below poison boundary: 0xffff8000083db400
| lkdtm: stackleak stack usage:
| high offset: 336 bytes
| current: 688 bytes
| lowest: 1232 bytes
| tracked: 576 bytes
| untracked: 288 bytes
| poisoned: 15176 bytes
| low offset: 8 bytes
| lkdtm: FAIL: the thread stack is NOT properly erased!
| lkdtm: Unexpected! This kernel (5.18.0-rc1-00013-g1f7b1f1e29e0-dirty aarch64) was built with CONFIG_GCC_PLUGIN_STACKLEAK=y

Mark.

2022-10-08 00:58:12

by Guo Ren

[permalink] [raw]
Subject: Re: [PATCH] RISC-V: Add STACKLEAK erasing the kernel stack at the end of syscalls

On Fri, Oct 7, 2022 at 10:31 AM Palmer Dabbelt <[email protected]> wrote:
>
> On Tue, 06 Sep 2022 10:35:10 PDT (-0700), [email protected] wrote:
> > On 03/09/2022 17:23, [email protected] wrote:
> >> EXTERNAL EMAIL: Do not click links or open attachments unless you know the content is safe
> >>
> >> From: Xianting Tian <[email protected]>
> >>
> >> This adds support for the STACKLEAK gcc plugin to RISC-V and disables
> >> the plugin in EFI stub code, which is out of scope for the protection.
> >>
> >> For the benefits of STACKLEAK feature, please check the commit
> >> afaef01c0015 ("x86/entry: Add STACKLEAK erasing the kernel stack at the end of syscalls")
> >>
> >> Performance impact (tested on qemu env with 1 riscv64 hart, 1GB mem)
> >> hackbench -s 512 -l 200 -g 15 -f 25 -P
> >> 2.0% slowdown
> >>
> >> Signed-off-by: Xianting Tian <[email protected]>
> >
> > What changed since Xianting posted it himself a week ago:
> > https://lore.kernel.org/linux-riscv/[email protected]/
> >
> > There's an older patch from Du Lao adding STACKLEAK too:
> > https://lore.kernel.org/linux-riscv/[email protected]/
> >
> > But since there's been no activity there since June...
>
> Looks like the only issues were some commit log wording stuff, and that
> there's a test suite that should be run. It's not clear from the
> commits that anyone has done that, I'm fine with the patch if it passes
> the tests but don't really know how to run them.
>
> Has anyone run the tests?
I'm trying to do that with genric_entry.
https://lore.kernel.org/linux-riscv/[email protected]/

Mark Rutland has found an issue, and I'm solving it.

>
> >
> >> ---
> >> arch/riscv/Kconfig | 1 +
> >> arch/riscv/include/asm/processor.h | 4 ++++
> >> arch/riscv/kernel/entry.S | 3 +++
> >> drivers/firmware/efi/libstub/Makefile | 2 +-
> >> 4 files changed, 9 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig
> >> index ed66c31e4655..61fd0dad4463 100644
> >> --- a/arch/riscv/Kconfig
> >> +++ b/arch/riscv/Kconfig
> >> @@ -85,6 +85,7 @@ config RISCV
> >> select ARCH_ENABLE_THP_MIGRATION if TRANSPARENT_HUGEPAGE
> >> select HAVE_ARCH_THREAD_STRUCT_WHITELIST
> >> select HAVE_ARCH_VMAP_STACK if MMU && 64BIT
> >> + select HAVE_ARCH_STACKLEAK
> >> select HAVE_ASM_MODVERSIONS
> >> select HAVE_CONTEXT_TRACKING_USER
> >> select HAVE_DEBUG_KMEMLEAK
> >> diff --git a/drivers/firmware/efi/libstub/Makefile b/drivers/firmware/efi/libstub/Makefile
> >> index d0537573501e..5e1fc4f82883 100644
> >> --- a/drivers/firmware/efi/libstub/Makefile
> >> +++ b/drivers/firmware/efi/libstub/Makefile
> >> @@ -25,7 +25,7 @@ cflags-$(CONFIG_ARM) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
> >> -fno-builtin -fpic \
> >> $(call cc-option,-mno-single-pic-base)
> >> cflags-$(CONFIG_RISCV) := $(subst $(CC_FLAGS_FTRACE),,$(KBUILD_CFLAGS)) \
> >> - -fpic
> >> + -fpic $(DISABLE_STACKLEAK_PLUGIN)
> >>
> >> cflags-$(CONFIG_EFI_GENERIC_STUB) += -I$(srctree)/scripts/dtc/libfdt
> >>
> >> --
> >> 2.17.1
> >>
> >>
> >> _______________________________________________
> >> linux-riscv mailing list
> >> [email protected]
> >> http://lists.infradead.org/mailman/listinfo/linux-riscv
> >



--
Best Regards
Guo Ren