2017-12-15 15:36:53

by Thomas-Mich Richter

[permalink] [raw]
Subject: [PATCH] trace/uprobes: fix output issue with address randomization

Commit ad67b74d2469 ("printk: hash addresses printed with %p")
changed %p to hash pointers in order to avoid leaking
kernel addresses.

This breaks the tool perf probe.

To set a uprobe on a function named inet_pton in libc library,
obtain the address of the symbol inet_pton using command nm and
then use the following command to set the uprobe:

# echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
> /sys/kernel/debug/tracing/uprobe_events

However the output shows a randomized address:
# cat /sys/kernel/debug/tracing/uprobe_events
p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x000000002d0f8952

The displayed address 0x000000002d0f8952 is incorrect and breaks
tools post processing it:

# linux/tools/perf/perf probe -l
Failed to find debug information for address 2d0f8952
probe_libc:inet_pton (on 0x2d0f8952 in /usr/lib64/libc-2.26.so)

Using the %px printk format string fixes this issue for root
and shows the correct address allowing the perf probe tool
to resolve the address to the symbol:

# echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
> /sys/kernel/debug/tracing/uprobe_events
# cat /sys/kernel/debug/tracing/uprobe_events
p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x0000000000142060
# linux/tools/perf/perf probe -l
probe_libc:inet_pton (on __inet_pton@resolv/inet_pton.c
in /usr/lib64/libc-2.26.so)

Signed-off-by: Thomas Richter <[email protected]>
Suggested-by: Peter Zijlstra <[email protected]>
---
kernel/trace/trace_uprobe.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c
index 40592e7b3568..268029ae1be6 100644
--- a/kernel/trace/trace_uprobe.c
+++ b/kernel/trace/trace_uprobe.c
@@ -608,7 +608,7 @@ static int probes_seq_show(struct seq_file *m, void *v)

/* Don't print "0x (null)" when offset is 0 */
if (tu->offset) {
- seq_printf(m, "0x%p", (void *)tu->offset);
+ seq_printf(m, "0x%px", (void *)tu->offset);
} else {
switch (sizeof(void *)) {
case 4:
--
2.13.4


2017-12-15 17:32:30

by Peter Zijlstra

[permalink] [raw]
Subject: Re: [PATCH] trace/uprobes: fix output issue with address randomization


I'm not in fact the maintainer covering this file..

On Fri, Dec 15, 2017 at 04:36:25PM +0100, Thomas Richter wrote:
> Commit ad67b74d2469 ("printk: hash addresses printed with %p")
> changed %p to hash pointers in order to avoid leaking
> kernel addresses.
>
> This breaks the tool perf probe.
>
> To set a uprobe on a function named inet_pton in libc library,
> obtain the address of the symbol inet_pton using command nm and
> then use the following command to set the uprobe:
>
> # echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
> > /sys/kernel/debug/tracing/uprobe_events
>
> However the output shows a randomized address:
> # cat /sys/kernel/debug/tracing/uprobe_events
> p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x000000002d0f8952
>
> The displayed address 0x000000002d0f8952 is incorrect and breaks
> tools post processing it:
>
> # linux/tools/perf/perf probe -l
> Failed to find debug information for address 2d0f8952
> probe_libc:inet_pton (on 0x2d0f8952 in /usr/lib64/libc-2.26.so)
>
> Using the %px printk format string fixes this issue for root
> and shows the correct address allowing the perf probe tool
> to resolve the address to the symbol:
>
> # echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
> > /sys/kernel/debug/tracing/uprobe_events
> # cat /sys/kernel/debug/tracing/uprobe_events
> p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x0000000000142060
> # linux/tools/perf/perf probe -l
> probe_libc:inet_pton (on __inet_pton@resolv/inet_pton.c
> in /usr/lib64/libc-2.26.so)
>
> Signed-off-by: Thomas Richter <[email protected]>
> Suggested-by: Peter Zijlstra <[email protected]>
> ---
> kernel/trace/trace_uprobe.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c
> index 40592e7b3568..268029ae1be6 100644
> --- a/kernel/trace/trace_uprobe.c
> +++ b/kernel/trace/trace_uprobe.c
> @@ -608,7 +608,7 @@ static int probes_seq_show(struct seq_file *m, void *v)
>
> /* Don't print "0x (null)" when offset is 0 */
> if (tu->offset) {
> - seq_printf(m, "0x%p", (void *)tu->offset);
> + seq_printf(m, "0x%px", (void *)tu->offset);
> } else {
> switch (sizeof(void *)) {
> case 4:

So I had a wee peek and afaict this ends up at:

trace_create_file("uprobe_events", 0644, d_tracer,
NULL, &uprobe_events_ops);

which is a world readable file. Doesn't that leak the kaslr offset?

2017-12-15 17:41:03

by Steven Rostedt

[permalink] [raw]
Subject: Re: [PATCH] trace/uprobes: fix output issue with address randomization

On Fri, 15 Dec 2017 18:31:56 +0100
Peter Zijlstra <[email protected]> wrote:

> > --- a/kernel/trace/trace_uprobe.c
> > +++ b/kernel/trace/trace_uprobe.c
> > @@ -608,7 +608,7 @@ static int probes_seq_show(struct seq_file *m, void *v)
> >
> > /* Don't print "0x (null)" when offset is 0 */
> > if (tu->offset) {
> > - seq_printf(m, "0x%p", (void *)tu->offset);
> > + seq_printf(m, "0x%px", (void *)tu->offset);
> > } else {
> > switch (sizeof(void *)) {
> > case 4:
>
> So I had a wee peek and afaict this ends up at:
>
> trace_create_file("uprobe_events", 0644, d_tracer,
> NULL, &uprobe_events_ops);
>
> which is a world readable file. Doesn't that leak the kaslr offset?

Usually the tracefs directory is mounted and is not world readable.

I just checked, and sure enough, after mounting tracefs, I can't read
it from non-root:

# mount -t tracefs nodev /sys/kernel/tracing/
# ls /sys/kernel/tracing/
available_events max_graph_depth stack_max_size
available_filter_functions options stack_trace
available_tracers per_cpu stack_trace_filter
buffer_size_kb printk_formats trace
buffer_total_size_kb README trace_clock
current_tracer saved_cmdlines trace_marker
dyn_ftrace_total_info saved_cmdlines_size trace_marker_raw
enabled_functions saved_tgids trace_options
eval_map set_event trace_pipe
events set_event_pid trace_stat
free_buffer set_ftrace_filter tracing_cpumask
function_profile_enabled set_ftrace_notrace tracing_max_latency
hwlat_detector set_ftrace_pid tracing_on
instances set_graph_function tracing_thresh
kprobe_events set_graph_notrace uprobe_events
kprobe_profile snapshot uprobe_profile

# exit
$ ls /sys/kernel/tracing/
ls: cannot open directory '/sys/kernel/tracing/': Permission denied

-- Steve

2017-12-19 00:25:59

by Masami Hiramatsu

[permalink] [raw]
Subject: Re: [PATCH] trace/uprobes: fix output issue with address randomization

On Fri, 15 Dec 2017 18:31:56 +0100
Peter Zijlstra <[email protected]> wrote:

>
> I'm not in fact the maintainer covering this file..
>
> On Fri, Dec 15, 2017 at 04:36:25PM +0100, Thomas Richter wrote:
> > Commit ad67b74d2469 ("printk: hash addresses printed with %p")
> > changed %p to hash pointers in order to avoid leaking
> > kernel addresses.
> >
> > This breaks the tool perf probe.
> >
> > To set a uprobe on a function named inet_pton in libc library,
> > obtain the address of the symbol inet_pton using command nm and
> > then use the following command to set the uprobe:
> >
> > # echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
> > > /sys/kernel/debug/tracing/uprobe_events
> >
> > However the output shows a randomized address:
> > # cat /sys/kernel/debug/tracing/uprobe_events
> > p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x000000002d0f8952
> >
> > The displayed address 0x000000002d0f8952 is incorrect and breaks
> > tools post processing it:
> >
> > # linux/tools/perf/perf probe -l
> > Failed to find debug information for address 2d0f8952
> > probe_libc:inet_pton (on 0x2d0f8952 in /usr/lib64/libc-2.26.so)
> >
> > Using the %px printk format string fixes this issue for root
> > and shows the correct address allowing the perf probe tool
> > to resolve the address to the symbol:
> >
> > # echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
> > > /sys/kernel/debug/tracing/uprobe_events
> > # cat /sys/kernel/debug/tracing/uprobe_events
> > p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x0000000000142060
> > # linux/tools/perf/perf probe -l
> > probe_libc:inet_pton (on __inet_pton@resolv/inet_pton.c
> > in /usr/lib64/libc-2.26.so)
> >
> > Signed-off-by: Thomas Richter <[email protected]>
> > Suggested-by: Peter Zijlstra <[email protected]>
> > ---
> > kernel/trace/trace_uprobe.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/kernel/trace/trace_uprobe.c b/kernel/trace/trace_uprobe.c
> > index 40592e7b3568..268029ae1be6 100644
> > --- a/kernel/trace/trace_uprobe.c
> > +++ b/kernel/trace/trace_uprobe.c
> > @@ -608,7 +608,7 @@ static int probes_seq_show(struct seq_file *m, void *v)
> >
> > /* Don't print "0x (null)" when offset is 0 */
> > if (tu->offset) {
> > - seq_printf(m, "0x%p", (void *)tu->offset);
> > + seq_printf(m, "0x%px", (void *)tu->offset);
> > } else {
> > switch (sizeof(void *)) {
> > case 4:
>
> So I had a wee peek and afaict this ends up at:
>
> trace_create_file("uprobe_events", 0644, d_tracer,
> NULL, &uprobe_events_ops);
>
> which is a world readable file. Doesn't that leak the kaslr offset?

As I correctly understand, this fix shows the correct offset in the file
of the probe point, this means we fixes that leakage of the aslr offset,
doesn't it?

Thank you,

--
Masami Hiramatsu <[email protected]>