2012-10-31 07:04:45

by Pekka Enberg

[permalink] [raw]
Subject: Re: [tip:perf/core] trace: Move trace event enable from fs_initcall to core_initcall

Hi Ezequiel,

On Thu, Sep 27, 2012 at 8:49 AM, tip-bot for Ezequiel Garcia
<[email protected]> wrote:
> Commit-ID: 8781915ad2716adcd8cd5cc52cee791fc8b00fdf
> Gitweb: http://git.kernel.org/tip/8781915ad2716adcd8cd5cc52cee791fc8b00fdf
> Author: Ezequiel Garcia <[email protected]>
> AuthorDate: Wed, 12 Sep 2012 11:47:57 -0300
> Committer: Steven Rostedt <[email protected]>
> CommitDate: Mon, 24 Sep 2012 14:13:02 -0400
>
> trace: Move trace event enable from fs_initcall to core_initcall
>
> This patch splits trace event initialization in two stages:
> * ftrace enable
> * sysfs event entry creation
>
> This allows to capture trace events from an earlier point
> by using 'trace_event' kernel parameter and is important
> to trace boot-up allocations.
>
> Note that, in order to enable events at core_initcall,
> it's necessary to move init_ftrace_syscalls() from
> core_initcall to early_initcall.
>
> Link: http://lkml.kernel.org/r/[email protected]
>
> Signed-off-by: Ezequiel Garcia <[email protected]>
> Signed-off-by: Steven Rostedt <[email protected]>

So what's still needed for perf to be able to trace early boot slab
allocations like kmemtrace used to be able to do?

Pekka


2012-10-31 11:20:00

by Ezequiel Garcia

[permalink] [raw]
Subject: Re: [tip:perf/core] trace: Move trace event enable from fs_initcall to core_initcall

Hi Pekka,

On Wed, Oct 31, 2012 at 4:04 AM, Pekka Enberg <[email protected]> wrote:
> Hi Ezequiel,
>
> On Thu, Sep 27, 2012 at 8:49 AM, tip-bot for Ezequiel Garcia
> <[email protected]> wrote:
>> Commit-ID: 8781915ad2716adcd8cd5cc52cee791fc8b00fdf
>> Gitweb: http://git.kernel.org/tip/8781915ad2716adcd8cd5cc52cee791fc8b00fdf
>> Author: Ezequiel Garcia <[email protected]>
>> AuthorDate: Wed, 12 Sep 2012 11:47:57 -0300
>> Committer: Steven Rostedt <[email protected]>
>> CommitDate: Mon, 24 Sep 2012 14:13:02 -0400
>>
>> trace: Move trace event enable from fs_initcall to core_initcall
>>
>> This patch splits trace event initialization in two stages:
>> * ftrace enable
>> * sysfs event entry creation
>>
>> This allows to capture trace events from an earlier point
>> by using 'trace_event' kernel parameter and is important
>> to trace boot-up allocations.
>>
>> Note that, in order to enable events at core_initcall,
>> it's necessary to move init_ftrace_syscalls() from
>> core_initcall to early_initcall.
>>
>> Link: http://lkml.kernel.org/r/[email protected]
>>
>> Signed-off-by: Ezequiel Garcia <[email protected]>
>> Signed-off-by: Steven Rostedt <[email protected]>
>
> So what's still needed for perf to be able to trace early boot slab
> allocations like kmemtrace used to be able to do?
>

Steven suggested [1] it would be possible to create some sort
of static ring buffer, and use it to trace early events.
Later on, this static buffer should be copied into the "real" one.

Since this patch, ftrace is enabled at core_initcall.
It's an improvement, but we still loose events.
For instance, in a probe kernel we loose 800k bytes,
and 20k allocations, compared to accounting from boot up.

To be able to use ftrace (and therefore perf) for long run tracing,
you need to consider ring buffer overrun.
In my tests, I had to set a bigger buffer,
which severely impacts performance.

IMHO, this makes ftrace unsuitable for very long run tests
with "verbose" events; unless you put an user daemon
or something popping events, preventing overrun.

Ezequiel

[1] https://lkml.org/lkml/2012/8/17/253