During debugging of perf probe tool I discovered an issue with
uprobes and address randomization.
To set a uprobe on a function named inet_pton in libc library, you
obtain the address of the symbol inet_pton using command nm and
then use the following command to set the uprobe:
# echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
> /sys/kernel/debug/tracing/uprobe_events
0x142060 is the address of inet_pton on my system.
This works nicely and the uprobe is usable.
The issue is with the output:
# cat /sys/kernel/debug/tracing/uprobe_events
p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x000000002d0f8952
#
The displayed address 0x000000002d0f8952 is wrong, probably
randomized and post processing of this output with the perf
probe tool fails due to this random address:
# linux/tools/perf/perf probe -l
Failed to find debug information for address 2d0f8952
probe_libc:inet_pton (on 0x2d0f8952 in /usr/lib64/libc-2.26.so)
#
So how to fix this (if at all)?
Is replacing %p by %llx in line 612 of file kernel/trace/trace_uprobe.c
seq_printf(m, "0x%p", (void *)tu->offset)
an option?
Or is this broken by design and intention?
Thanks for hints and suggestions.
--
Thomas Richter, Dept 3303, IBM LTC Boeblingen Germany
--
Vorsitzende des Aufsichtsrats: Martina Koederitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294
On Fri, Dec 15, 2017 at 03:14:37PM +0100, Thomas-Mich Richter wrote:
> During debugging of perf probe tool I discovered an issue with
> uprobes and address randomization.
>
> To set a uprobe on a function named inet_pton in libc library, you
> obtain the address of the symbol inet_pton using command nm and
> then use the following command to set the uprobe:
>
> # echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
> > /sys/kernel/debug/tracing/uprobe_events
>
> 0x142060 is the address of inet_pton on my system.
> This works nicely and the uprobe is usable.
>
> The issue is with the output:
> # cat /sys/kernel/debug/tracing/uprobe_events
> p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x000000002d0f8952
> #
>
> The displayed address 0x000000002d0f8952 is wrong, probably
> randomized and post processing of this output with the perf
> probe tool fails due to this random address:
>
> # linux/tools/perf/perf probe -l
> Failed to find debug information for address 2d0f8952
> probe_libc:inet_pton (on 0x2d0f8952 in /usr/lib64/libc-2.26.so)
> #
>
> So how to fix this (if at all)?
> Is replacing %p by %llx in line 612 of file kernel/trace/trace_uprobe.c
> seq_printf(m, "0x%p", (void *)tu->offset)
> an option?
> Or is this broken by design and intention?
So recently %p got changed to hash pointers in order to avoid leaking
kernel addresses.
ad67b74d2469 ("printk: hash addresses printed with %p")
I'm not sure what privilidges are required for reading that kprobe
state, but I suspect its root only, so changing this to %px might be
what is needed.
On 12/15/2017 03:21 PM, Peter Zijlstra wrote:
> On Fri, Dec 15, 2017 at 03:14:37PM +0100, Thomas-Mich Richter wrote:
>> During debugging of perf probe tool I discovered an issue with
>> uprobes and address randomization.
>>
>> To set a uprobe on a function named inet_pton in libc library, you
>> obtain the address of the symbol inet_pton using command nm and
>> then use the following command to set the uprobe:
>>
>> # echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
>> > /sys/kernel/debug/tracing/uprobe_events
>>
>> 0x142060 is the address of inet_pton on my system.
>> This works nicely and the uprobe is usable.
>>
>> The issue is with the output:
>> # cat /sys/kernel/debug/tracing/uprobe_events
>> p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x000000002d0f8952
>> #
>>
>> The displayed address 0x000000002d0f8952 is wrong, probably
>> randomized and post processing of this output with the perf
>> probe tool fails due to this random address:
>>
>> # linux/tools/perf/perf probe -l
>> Failed to find debug information for address 2d0f8952
>> probe_libc:inet_pton (on 0x2d0f8952 in /usr/lib64/libc-2.26.so)
>> #
>>
>> So how to fix this (if at all)?
>> Is replacing %p by %llx in line 612 of file kernel/trace/trace_uprobe.c
>> seq_printf(m, "0x%p", (void *)tu->offset)
>> an option?
>> Or is this broken by design and intention?
>
> So recently %p got changed to hash pointers in order to avoid leaking
> kernel addresses.
>
> ad67b74d2469 ("printk: hash addresses printed with %p")
>
> I'm not sure what privilidges are required for reading that kprobe
> state, but I suspect its root only, so changing this to %px might be
> what is needed.
>
Thanks for the hint. The %px printk format string fixes the issue:
# echo "p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x142060"
> /sys/kernel/debug/tracing/uprobe_events
# cat /sys/kernel/debug/tracing/uprobe_events
p:probe_libc/inet_pton /usr/lib64/libc-2.26.so:0x0000000000142060
# ./tools/perf/perf probe -l
probe_libc:inet_pton (on __inet_pton@resolv/inet_pton.c in /usr/lib64/libc-2.26.so)
#
I will send a patch to fix this.
--
Thomas Richter, Dept 3303, IBM LTC Boeblingen Germany
--
Vorsitzende des Aufsichtsrats: Martina Koederitz
Geschäftsführung: Dirk Wittkopp
Sitz der Gesellschaft: Böblingen / Registergericht: Amtsgericht Stuttgart, HRB 243294