On bootup of the latest kernel my init segfaults. Debugging it, I found
that vread_tsc (a vsyscall) increments some strange kernel memory:
0000000000000000 <vread_tsc>:
0: 55 push %rbp
1: 48 ff 05 00 00 00 00 incq 0(%rip) # 8 <vread_tsc+0x8>
4: R_X86_64_PC32 .bss+0x3c
8: 48 89 e5 mov %rsp,%rbp
b: 66 66 90 xchg %ax,%ax
e: 48 ff 05 00 00 00 00 incq 0(%rip) # 15 <vread_tsc+0x15>
11: R_X86_64_PC32 .bss+0x44
15: 66 66 90 xchg %ax,%ax
18: 48 ff 05 00 00 00 00 incq 0(%rip) # 1f <vread_tsc+0x1f>
1b: R_X86_64_PC32 .bss+0x4c
1f: 0f 31 rdtsc
Those "incq" is very bad to happen in vsyscall memory, since userspace can
not modify it. You need to make something prevent profiling of vsyscall
memory (like I do with ftrace).
-- Steve
Steven Rostedt wrote:
> On bootup of the latest kernel my init segfaults. Debugging it, I found
> that vread_tsc (a vsyscall) increments some strange kernel memory:
>
> 0000000000000000 <vread_tsc>:
> 0: 55 push %rbp
> 1: 48 ff 05 00 00 00 00 incq 0(%rip) # 8 <vread_tsc+0x8>
> 4: R_X86_64_PC32 .bss+0x3c
> 8: 48 89 e5 mov %rsp,%rbp
> b: 66 66 90 xchg %ax,%ax
> e: 48 ff 05 00 00 00 00 incq 0(%rip) # 15 <vread_tsc+0x15>
> 11: R_X86_64_PC32 .bss+0x44
> 15: 66 66 90 xchg %ax,%ax
> 18: 48 ff 05 00 00 00 00 incq 0(%rip) # 1f <vread_tsc+0x1f>
> 1b: R_X86_64_PC32 .bss+0x4c
> 1f: 0f 31 rdtsc
>
>
> Those "incq" is very bad to happen in vsyscall memory, since userspace can
> not modify it. You need to make something prevent profiling of vsyscall
> memory (like I do with ftrace).
>
> -- Steve
You're right, I missed that file. This should be fixed with the patch
below. As the problem didn't occur on my test machine, please retest
with the patch applied. Thanks!
Also seeing as function tracer and gcov work on a similar basis and
require similar files to be excluded from profiling, it would be nice
if we wouldn't need to mark those files separately. Instead it would
be great if the Makefile could be used to specify that a certain
object file has a certain property (e.g. PROPERTY_USERPACE_file.o := y)
and the mechanism (e.g. function tracer) would only need to specify
that the extra gcc options should not be applied when that property is
set. What do you think?
=================
Subject: [PATCH] gcov: exclude code operating in userspace from profiling
From: Peter Oberparleiter <[email protected]>
Fix for this issue on x86_64:
[email protected] wrote:
> On bootup of the latest kernel my init segfaults. Debugging it,
> I found that vread_tsc (a vsyscall) increments some strange
> kernel memory:
>
> 0000000000000000 <vread_tsc>:
> 0: 55 push %rbp
> 1: 48 ff 05 00 00 00 00 incq 0(%rip)
> # 8 <vread_tsc+0x8>
> 4: R_X86_64_PC32 .bss+0x3c
> 8: 48 89 e5 mov %rsp,%rbp
> b: 66 66 90 xchg %ax,%ax
> e: 48 ff 05 00 00 00 00 incq 0(%rip)
> # 15 <vread_tsc+0x15>
> 11: R_X86_64_PC32 .bss+0x44
> 15: 66 66 90 xchg %ax,%ax
> 18: 48 ff 05 00 00 00 00 incq 0(%rip)
> # 1f <vread_tsc+0x1f>
> 1b: R_X86_64_PC32 .bss+0x4c
> 1f: 0f 31 rdtsc
>
>
> Those "incq" is very bad to happen in vsyscall memory, since
> userspace can not modify it. You need to make something prevent
> profiling of vsyscall memory (like I do with ftrace).
Signed-off-by: Peter Oberparleiter <[email protected]>
---
arch/x86/kernel/Makefile | 2 ++
1 file changed, 2 insertions(+)
Index: linux-2.6.31-rc1/arch/x86/kernel/Makefile
===================================================================
--- linux-2.6.31-rc1.orig/arch/x86/kernel/Makefile
+++ linux-2.6.31-rc1/arch/x86/kernel/Makefile
@@ -26,6 +26,8 @@ CFLAGS_tsc.o := $(nostackp)
CFLAGS_paravirt.o := $(nostackp)
GCOV_PROFILE_vsyscall_64.o := n
GCOV_PROFILE_hpet.o := n
+GCOV_PROFILE_tsc.o := n
+GCOV_PROFILE_paravirt.o := n
obj-y := process_$(BITS).o signal.o entry_$(BITS).o
obj-y += traps.o irq.o irq_$(BITS).o dumpstack_$(BITS).o
On Thu, 2 Jul 2009, Peter Oberparleiter wrote:
> You're right, I missed that file. This should be fixed with the patch
> below. As the problem didn't occur on my test machine, please retest
> with the patch applied. Thanks!
>
> Also seeing as function tracer and gcov work on a similar basis and
> require similar files to be excluded from profiling, it would be nice
> if we wouldn't need to mark those files separately. Instead it would
> be great if the Makefile could be used to specify that a certain
> object file has a certain property (e.g. PROPERTY_USERPACE_file.o := y)
> and the mechanism (e.g. function tracer) would only need to specify
> that the extra gcc options should not be applied when that property is
> set. What do you think?
I think a property for vsyscall would be nice. Then you can add your
restrictions there. But ftrace also has to deal with other issues as well,
not just vsyscall.
>
> =================
> Subject: [PATCH] gcov: exclude code operating in userspace from profiling
>
> From: Peter Oberparleiter <[email protected]>
>
> Fix for this issue on x86_64:
>
> [email protected] wrote:
> > On bootup of the latest kernel my init segfaults. Debugging it,
> > I found that vread_tsc (a vsyscall) increments some strange
> > kernel memory:
> >
> > 0000000000000000 <vread_tsc>:
> > 0: 55 push %rbp
> > 1: 48 ff 05 00 00 00 00 incq 0(%rip)
> > # 8 <vread_tsc+0x8>
> > 4: R_X86_64_PC32 .bss+0x3c
> > 8: 48 89 e5 mov %rsp,%rbp
> > b: 66 66 90 xchg %ax,%ax
> > e: 48 ff 05 00 00 00 00 incq 0(%rip)
> > # 15 <vread_tsc+0x15>
> > 11: R_X86_64_PC32 .bss+0x44
> > 15: 66 66 90 xchg %ax,%ax
> > 18: 48 ff 05 00 00 00 00 incq 0(%rip)
> > # 1f <vread_tsc+0x1f>
> > 1b: R_X86_64_PC32 .bss+0x4c
> > 1f: 0f 31 rdtsc
> >
> >
> > Those "incq" is very bad to happen in vsyscall memory, since
> > userspace can not modify it. You need to make something prevent
> > profiling of vsyscall memory (like I do with ftrace).
>
Reported-and-tested-by: Steven Rostedt <[email protected]>
Thanks!
-- Steve
> Signed-off-by: Peter Oberparleiter <[email protected]>
> ---
> arch/x86/kernel/Makefile | 2 ++
> 1 file changed, 2 insertions(+)
>
> Index: linux-2.6.31-rc1/arch/x86/kernel/Makefile
> ===================================================================
> --- linux-2.6.31-rc1.orig/arch/x86/kernel/Makefile
> +++ linux-2.6.31-rc1/arch/x86/kernel/Makefile
> @@ -26,6 +26,8 @@ CFLAGS_tsc.o := $(nostackp)
> CFLAGS_paravirt.o := $(nostackp)
> GCOV_PROFILE_vsyscall_64.o := n
> GCOV_PROFILE_hpet.o := n
> +GCOV_PROFILE_tsc.o := n
> +GCOV_PROFILE_paravirt.o := n
>
> obj-y := process_$(BITS).o signal.o entry_$(BITS).o
> obj-y += traps.o irq.o irq_$(BITS).o dumpstack_$(BITS).o
>
>
* Peter Oberparleiter <[email protected]> wrote:
> Also seeing as function tracer and gcov work on a similar basis
> and require similar files to be excluded from profiling, it would
> be nice if we wouldn't need to mark those files separately. [...]
One of the general things i pointed out during review is that
there's a fair amount of overlap between tracing/instrumentation and
the gcov support patches and now for a change Andrew has merged
incomplete stuff upstream prematurely ;-) Andrew, could you please
make sure this all gets fixed and the review suggestions get
implemented?
Ingo
Ingo Molnar wrote:
> * Peter Oberparleiter <[email protected]> wrote:
>
>> Also seeing as function tracer and gcov work on a similar basis
>> and require similar files to be excluded from profiling, it would
>> be nice if we wouldn't need to mark those files separately. [...]
>
> One of the general things i pointed out during review is that
> there's a fair amount of overlap between tracing/instrumentation and
> the gcov support patches and now for a change Andrew has merged
> incomplete stuff upstream prematurely ;-)
Yes there is overlap between ftrace and gcov and I will help in finding
and merging those bits, but as I also pointed out during review, there
are also fundamental differences.
By including the gcov infrastructure upstream now, we can incrementally
improve its integration with any similar mechanism _while_ people are
improving their test cases and identifying more bugs based on code
coverage measurements.