Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756465AbcK2MJ2 (ORCPT ); Tue, 29 Nov 2016 07:09:28 -0500 Received: from Galois.linutronix.de ([146.0.238.70]:50800 "EHLO Galois.linutronix.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755028AbcK2MJT (ORCPT ); Tue, 29 Nov 2016 07:09:19 -0500 Date: Tue, 29 Nov 2016 13:06:37 +0100 (CET) From: Thomas Gleixner To: John Stultz cc: lkml , Baolin Wang , Richard Cochran , Prarit Bhargava , Ingo Molnar , Steven Rostedt Subject: Re: [PATCH 4/7] time: alarmtimer: Add the tracepoints for alarmtimer In-Reply-To: <1480372524-15181-5-git-send-email-john.stultz@linaro.org> Message-ID: References: <1480372524-15181-1-git-send-email-john.stultz@linaro.org> <1480372524-15181-5-git-send-email-john.stultz@linaro.org> User-Agent: Alpine 2.20 (DEB 67 2015-01-07) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 11980 Lines: 410 On Mon, 28 Nov 2016, John Stultz wrote: > From: Baolin Wang > > For system debugging, we sometimes want to know who sets one > alarm timer, the time of the timer, when the timer started and > fired and so on. Thus adding tracepoints can help us trace the > alarmtimer information. > > For example, when we debug the system supend/resume, if the > system is always resumed by RTC alarm, we can find out which > process set the alarm timer to resume system by below trace log: > > ...... > > Binder:3292_2-3304 [000] d..2 149.981123: alarmtimer_cancel: > alarmtimer:ffffffc1319a7800 type:REALTIME > expires:1325463120000000000 now:1325376810370370245 > > Binder:3292_2-3304 [000] d..2 149.981136: alarmtimer_start: > alarmtimer:ffffffc1319a7800 type:REALTIME > expires:1325376840000000000 now:1325376810370384591 > > Binder:3292_9-3953 [000] d..2 150.212991: alarmtimer_cancel: > alarmtimer:ffffffc1319a5a00 type:BOOTTIME > expires:179552000000 now:150154008122 > > Binder:3292_9-3953 [000] d..2 150.213006: alarmtimer_start: > alarmtimer:ffffffc1319a5a00 type:BOOTTIME > expires:179551000000 now:150154025622 > > ...... > > system_server-3000 [002] ...1 162.701940: alarmtimer_suspend: > alarmtimer type:REALTIME expires:1325376839802714584 > ...... > > >From the trace log, we can find out the 'Binder:3292_2' process > set one alarm timer which resumes the system. To be honest. I have a hard time to figure out that Binder:3292_2 armed the timer. suspend: expires:1325376839802714584 Binder start: expires:1325376840000000000 That does not make any sense at all. And this is a result of how that suspend tracepoint got glued into alarmtimer_suspend(): > @@ -222,7 +226,7 @@ static int alarmtimer_suspend(struct device *dev) > ktime_t min, now; > unsigned long flags; > struct rtc_device *rtc; > - int i; > + int i, min_type = -1; > int ret; > > spin_lock_irqsave(&freezer_delta_lock, flags); > @@ -247,8 +251,10 @@ static int alarmtimer_suspend(struct device *dev) > if (!next) > continue; > delta = ktime_sub(next->expires, base->gettime()); > - if (!min.tv64 || (delta.tv64 < min.tv64)) > + if (!min.tv64 || (delta.tv64 < min.tv64)) { > min = delta; > + min_type = i; > + } > } > if (min.tv64 == 0) > return 0; > @@ -264,6 +270,8 @@ static int alarmtimer_suspend(struct device *dev) > now = rtc_tm_to_ktime(tm); > now = ktime_add(now, min); > > + trace_alarmtimer_suspend(now, min_type); So first we calculate the relative expiry time from now, add it to the RTC and then print out the sum. That's utter crap especially if the first expiring timer is based on CLOCK_BOOTTIME. Even with CLOCK_REALTIME you do not get matching expiry time as you can see above. Now looking at the other possibility to arm the resume time: freezer_delta. That's even more undebuggable because the freezer time is stored as relative timeout without any type information. How is one supposed to map that back to an alarm timer start? Find below a revamped version (untested) of this including a proper written changelog. Thanks, tglx 8<-------------- Subject: time/alarmtimer: Add tracepoints for alarmtimer From: John Stultz Date: Mon, 28 Nov 2016 14:35:21 -0800 From: Baolin Wang Alarm timers are one of the mechanisms to wake up a system from suspend, but there exist no tracepoints to analyse which process/thread armed an alarmtimer. Add tracepoints for start/cancel/expire of individual alarm timers and one for tracing the suspend time decision when to resume the system. The following trace excerpt illustrates the new mechanism: Binder:3292_2-3304 [000] d..2 149.981123: alarmtimer_cancel: alarmtimer:ffffffc1319a7800 type:REALTIME expires:1325463120000000000 now:1325376810370370245 Binder:3292_2-3304 [000] d..2 149.981136: alarmtimer_start: alarmtimer:ffffffc1319a7800 type:REALTIME expires:1325376840000000000 now:1325376810370384591 Binder:3292_9-3953 [000] d..2 150.212991: alarmtimer_cancel: alarmtimer:ffffffc1319a5a00 type:BOOTTIME expires:179552000000 now:150154008122 Binder:3292_9-3953 [000] d..2 150.213006: alarmtimer_start: alarmtimer:ffffffc1319a5a00 type:BOOTTIME expires:179551000000 now:150154025622 system_server-3000 [002] ...1 162.701940: alarmtimer_suspend: alarmtimer type:REALTIME expires:1325376840000000000 The wakeup time which is selected at suspend time allows to map it back to the task arming the timer: Binder:3292_2. [ tglx: Store real alarm timer expiry time instead of some useless RTC relative information, add proper type information for wakeups which are handled via the clock_nanosleep/freezer and massage the changelog. ] Signed-off-by: Baolin Wang Signed-off-by: John Stultz Acked-by: Steven Rostedt Cc: Prarit Bhargava Cc: Richard Cochran Link: http://lkml.kernel.org/r/1480372524-15181-5-git-send-email-john.stultz@linaro.org Signed-off-by: Thomas Gleixner --- Changes since v4: - Initialize 'type' to -1 and rename it in alarmtimer_suspend(). - Fix typo in subject line. Changes since v3: - Remove the "ALARM_" prefix in the string. - Add the ACK by Steven Rostedt. Changes since v2: - Save time as s64 type. - Remove 'process_name' parameter and add 'now' parameter. - Rename the trace event name. - Remove restart trace event. - Other optimization. --- include/linux/alarmtimer.h | 5 + include/trace/events/alarmtimer.h | 96 ++++++++++++++++++++++++++++++++++++++ kernel/time/alarmtimer.c | 53 +++++++++++++++++--- 3 files changed, 144 insertions(+), 10 deletions(-) create mode 100644 include/trace/events/alarmtimer.h --- a/include/linux/alarmtimer.h +++ b/include/linux/alarmtimer.h @@ -10,7 +10,12 @@ enum alarmtimer_type { ALARM_REALTIME, ALARM_BOOTTIME, + /* Supported types end here */ ALARM_NUMTYPE, + + /* Used for tracing information. No usable types. */ + ALARM_REALTIME_FREEZER, + ALARM_BOOTTIME_FREEZER, }; enum alarmtimer_restart { --- /dev/null +++ b/include/trace/events/alarmtimer.h @@ -0,0 +1,96 @@ +#undef TRACE_SYSTEM +#define TRACE_SYSTEM alarmtimer + +#if !defined(_TRACE_ALARMTIMER_H) || defined(TRACE_HEADER_MULTI_READ) +#define _TRACE_ALARMTIMER_H + +#include +#include +#include + +TRACE_DEFINE_ENUM(ALARM_REALTIME); +TRACE_DEFINE_ENUM(ALARM_BOOTTIME); +TRACE_DEFINE_ENUM(ALARM_REALTIME_FREEZER); +TRACE_DEFINE_ENUM(ALARM_BOOTTIME_FREEZER); + +#define show_alarm_type(type) __print_flags(type, " | ", \ + { 1 << ALARM_REALTIME, "REALTIME" }, \ + { 1 << ALARM_BOOTTIME, "BOOTTIME" }, \ + { 1 << ALARM_REALTIME_FREEZER, "REALTIME Freezer" }, \ + { 1 << ALARM_BOOTTIME_FREEZER, "BOOTTIME Freezer" }) + +TRACE_EVENT(alarmtimer_suspend, + + TP_PROTO(ktime_t expires, int flag), + + TP_ARGS(expires, flag), + + TP_STRUCT__entry( + __field(s64, expires) + __field(unsigned char, alarm_type) + ), + + TP_fast_assign( + __entry->expires = expires.tv64; + __entry->alarm_type = flag; + ), + + TP_printk("alarmtimer type:%s expires:%llu", + show_alarm_type((1 << __entry->alarm_type)), + __entry->expires + ) +); + +DECLARE_EVENT_CLASS(alarm_class, + + TP_PROTO(struct alarm *alarm, ktime_t now), + + TP_ARGS(alarm, now), + + TP_STRUCT__entry( + __field(void *, alarm) + __field(unsigned char, alarm_type) + __field(s64, expires) + __field(s64, now) + ), + + TP_fast_assign( + __entry->alarm = alarm; + __entry->alarm_type = alarm->type; + __entry->expires = alarm->node.expires.tv64; + __entry->now = now.tv64; + ), + + TP_printk("alarmtimer:%p type:%s expires:%llu now:%llu", + __entry->alarm, + show_alarm_type((1 << __entry->alarm_type)), + __entry->expires, + __entry->now + ) +); + +DEFINE_EVENT(alarm_class, alarmtimer_fired, + + TP_PROTO(struct alarm *alarm, ktime_t now), + + TP_ARGS(alarm, now) +); + +DEFINE_EVENT(alarm_class, alarmtimer_start, + + TP_PROTO(struct alarm *alarm, ktime_t now), + + TP_ARGS(alarm, now) +); + +DEFINE_EVENT(alarm_class, alarmtimer_cancel, + + TP_PROTO(struct alarm *alarm, ktime_t now), + + TP_ARGS(alarm, now) +); + +#endif /* _TRACE_ALARMTIMER_H */ + +/* This part must be outside protection */ +#include --- a/kernel/time/alarmtimer.c +++ b/kernel/time/alarmtimer.c @@ -26,6 +26,9 @@ #include #include +#define CREATE_TRACE_POINTS +#include + /** * struct alarm_base - Alarm timer bases * @lock: Lock for syncrhonized access to the base @@ -40,7 +43,9 @@ static struct alarm_base { clockid_t base_clockid; } alarm_bases[ALARM_NUMTYPE]; -/* freezer delta & lock used to handle clock_nanosleep triggered wakeups */ +/* freezer information to handle clock_nanosleep triggered wakeups */ +static enum alarmtimer_type freezer_alarmtype; +static ktime_t freezer_expires; static ktime_t freezer_delta; static DEFINE_SPINLOCK(freezer_delta_lock); @@ -194,6 +199,7 @@ static enum hrtimer_restart alarmtimer_f } spin_unlock_irqrestore(&base->lock, flags); + trace_alarmtimer_fired(alarm, base->gettime()); return ret; } @@ -218,15 +224,16 @@ EXPORT_SYMBOL_GPL(alarm_expires_remainin */ static int alarmtimer_suspend(struct device *dev) { - struct rtc_time tm; - ktime_t min, now; - unsigned long flags; + ktime_t min, now, expires; + int i, ret, type; struct rtc_device *rtc; - int i; - int ret; + unsigned long flags; + struct rtc_time tm; spin_lock_irqsave(&freezer_delta_lock, flags); min = freezer_delta; + expires = freezer_expires; + type = freezer_alarmtype; freezer_delta = ktime_set(0, 0); spin_unlock_irqrestore(&freezer_delta_lock, flags); @@ -247,8 +254,11 @@ static int alarmtimer_suspend(struct dev if (!next) continue; delta = ktime_sub(next->expires, base->gettime()); - if (!min.tv64 || (delta.tv64 < min.tv64)) + if (!min.tv64 || (delta.tv64 < min.tv64)) { + expires = next->expires; min = delta; + type = i; + } } if (min.tv64 == 0) return 0; @@ -258,6 +268,8 @@ static int alarmtimer_suspend(struct dev return -EBUSY; } + trace_alarmtimer_suspend(expires, type); + /* Setup an rtc timer to fire that far in the future */ rtc_timer_cancel(rtc, &rtctimer); rtc_read_time(rtc, &tm); @@ -295,15 +307,32 @@ static int alarmtimer_resume(struct devi static void alarmtimer_freezerset(ktime_t absexp, enum alarmtimer_type type) { - ktime_t delta; + struct alarm_base *base; unsigned long flags; - struct alarm_base *base = &alarm_bases[type]; + ktime_t delta; + + switch(type) { + case ALARM_REALTIME: + base = &alarm_bases[ALARM_REALTIME]; + type = ALARM_REALTIME_FREEZER; + break; + case ALARM_BOOTTIME: + base = &alarm_bases[ALARM_BOOTTIME]; + type = ALARM_BOOTTIME_FREEZER; + break; + default: + WARN_ONCE(1, "Invalid alarm type: %d\n", type); + return; + } delta = ktime_sub(absexp, base->gettime()); spin_lock_irqsave(&freezer_delta_lock, flags); - if (!freezer_delta.tv64 || (delta.tv64 < freezer_delta.tv64)) + if (!freezer_delta.tv64 || (delta.tv64 < freezer_delta.tv64)) { freezer_delta = delta; + freezer_expires = absexp; + freezer_alarmtype = type; + } spin_unlock_irqrestore(&freezer_delta_lock, flags); } @@ -342,6 +371,8 @@ void alarm_start(struct alarm *alarm, kt alarmtimer_enqueue(base, alarm); hrtimer_start(&alarm->timer, alarm->node.expires, HRTIMER_MODE_ABS); spin_unlock_irqrestore(&base->lock, flags); + + trace_alarmtimer_start(alarm, base->gettime()); } EXPORT_SYMBOL_GPL(alarm_start); @@ -390,6 +421,8 @@ int alarm_try_to_cancel(struct alarm *al if (ret >= 0) alarmtimer_dequeue(base, alarm); spin_unlock_irqrestore(&base->lock, flags); + + trace_alarmtimer_cancel(alarm, base->gettime()); return ret; } EXPORT_SYMBOL_GPL(alarm_try_to_cancel);