Dear Steven,
I want to add a new function to ftrace subsystem. Sometimes, we will face such a problem: system do not response to the input on time one to two times everyday. It is not easy to capture because it rarely happens. So I want to add a function to the kernel. If I have such problem, I insert a kernel module, who add a hook to the position that receive the input and another to the position that response to the input (with a session id if necessary). And I can compare the time between them and if the period is longer then a pre-set threshold, I can give a signal to a user helper application (maybe a script waiting on the file), which then can save the trace event to a file for later inspection.
The user helper script may look like this:
#/bin/sh
echo ??sched:*?? > /sys/kernel/debug/tracing/set_event
modprobe delay_inspector.ko threshold=500
cat /sys/kernel/debug/tracing/waiter #wait for signal
cp /sys/kernel/debug/tracing/trace /var/log/delay_infomation
It looks like a standalone function. But I don??t have place to put it. Do you think I can implement it in ftrace? And do you think if there are better solution?
Thank you.
Kenneth Lee-
On Wed, 30 Oct 2013 15:39:50 -0700
Kenneth Lee <[email protected]> wrote:
> Dear Steven,
>
> I want to add a new function to ftrace subsystem. Sometimes, we will face such a problem: system do not response to the input on time one to two times everyday. It is not easy to capture because it rarely happens. So I want to add a function to the kernel. If I have such problem, I insert a kernel module, who add a hook to the position that receive the input and another to the position that response to the input (with a session id if necessary). And I can compare the time between them and if the period is longer then a pre-set threshold, I can give a signal to a user helper application (maybe a script waiting on the file), which then can save the trace event to a file for later inspection.
I'm a little confused in what you want.
>
>
>
> The user helper script may look like this:
>
>
>
> #/bin/sh
>
>
>
> echo ‘sched:*’ > /sys/kernel/debug/tracing/set_event
>
> modprobe delay_inspector.ko threshold=500
>
> cat /sys/kernel/debug/tracing/waiter #wait for signal
>
> cp /sys/kernel/debug/tracing/trace /var/log/delay_infomation
>
>
>
>
>
> It looks like a standalone function. But I don’t have place to put it. Do you think I can implement it in ftrace? And do you think if there are better solution?
>
You want something to wake up if it takes too long before an event
happens?
If so, why not just use a select() on the trace_pipe and if it times
out, then dump the trace. You can even set up a separate instance.
(this is waiting for a schedule switch to pid 1)
cd /sys/kernel/debug/tracing
mkdir instances/mine
echo 'next_pid == 1' > instances/mine/events/sched/sched_switch/filter
echo 1 > instances/mine/events/sched/sched_switch/enable
The in a userspace program, I open "instances/mine/events/trace_pipe"
and run a select() on that file descriptor with a given timeout. If the
event does not happen within the expected time frame, the select
returns zero, and this userspace program can deal with it.
Is that the functionality you are trying to achieve?
-- Steve
Thank you, Steve,
Yes, with a separated instance, I can measure the latency for a stimulation while capture the other schedule events which I am interesting in. This is a better solution. I don??t know this ??instance?? stuff before. I don??t need to create another axe. I am sorry for my ignorance.
Thanks and regards.
--Kenneth
?? 2013??10??31?գ?????1:50??Steven Rostedt <[email protected]> д????
> On Wed, 30 Oct 2013 15:39:50 -0700
> Kenneth Lee <[email protected]> wrote:
>
>> Dear Steven,
>>
>> I want to add a new function to ftrace subsystem. Sometimes, we will face such a problem: system do not response to the input on time one to two times everyday. It is not easy to capture because it rarely happens. So I want to add a function to the kernel. If I have such problem, I insert a kernel module, who add a hook to the position that receive the input and another to the position that response to the input (with a session id if necessary). And I can compare the time between them and if the period is longer then a pre-set threshold, I can give a signal to a user helper application (maybe a script waiting on the file), which then can save the trace event to a file for later inspection.
>
> I'm a little confused in what you want.
>
>
>>
>>
>>
>> The user helper script may look like this:
>>
>>
>>
>> #/bin/sh
>>
>>
>>
>> echo ??sched:*?? > /sys/kernel/debug/tracing/set_event
>>
>> modprobe delay_inspector.ko threshold=500
>>
>> cat /sys/kernel/debug/tracing/waiter #wait for signal
>>
>> cp /sys/kernel/debug/tracing/trace /var/log/delay_infomation
>>
>>
>>
>>
>>
>> It looks like a standalone function. But I don??t have place to put it. Do you think I can implement it in ftrace? And do you think if there are better solution?
>>
>
> You want something to wake up if it takes too long before an event
> happens?
>
> If so, why not just use a select() on the trace_pipe and if it times
> out, then dump the trace. You can even set up a separate instance.
>
> (this is waiting for a schedule switch to pid 1)
>
> cd /sys/kernel/debug/tracing
> mkdir instances/mine
> echo 'next_pid == 1' > instances/mine/events/sched/sched_switch/filter
> echo 1 > instances/mine/events/sched/sched_switch/enable
>
>
> The in a userspace program, I open "instances/mine/events/trace_pipe"
> and run a select() on that file descriptor with a given timeout. If the
> event does not happen within the expected time frame, the select
> returns zero, and this userspace program can deal with it.
>
> Is that the functionality you are trying to achieve?
>
> -- Steve