All,
The following patchset applies against 2.6.15-rc3-mm1 and provides a
generic timekeeping subsystem that is independent of the timer
interrupt. This allows for robust and correct behavior in cases of late
or lost ticks, avoids interpolation errors, reduces duplication in arch
specific code, and allows or assists future changes such as high-res
timers, dynamic ticks, or realtime preemption. Additionally, it provides
finer nanosecond resolution values to the clock_gettime functions.
The patch set provides the minimal NTP changes, the clocksource
abstraction, the core timekeeping code as well as the code to convert
the i386 and x86-64 archs. I have started on converting more arches, but
for now I'm focusing on i386 and x86-64.
New in this release:
o Merged Ingo's whitespace/style fixups (*Many* thanks for this help!)
o Addressed Ingo's TODO items.
o Merged cpufreq SMP fix
o Dropped tsc-interp clocksource (wasn't being used)
o Improved NUMAQ bits
Still on the TODO list:
o Submit to -mm
I'd like to thank the following people who have contributed ideas,
criticism, testing and code that has helped shape this work:
George Anzinger, Nish Aravamudan, Max Asbock, Dominik Brodowski, Thomas
Gleixner, Darren Hart, Christoph Lameter, Matt Mackal, Keith Mannthey,
Ingo Molnar, Martin Schwidefsky, Frank Sorenson, Ulrich Windl, Darrick
Wong, Roman Zippel and any others whom I've accidentally forgotten.
thanks
-john
All,
With Roman's suggestions, I've been working on reducing the footprint
of my timeofday patches. This is the first of two patches that reworks
some of the interrupt time NTP adjustments so that it could be re-used
with the timeofday patches. The motivation of the change is to logically
separate the code which adjusts xtime and the code that decides, based
on the NTP state variables, how much per tick to adjust xtime.
Thus this patch should not affect the existing behavior, but just
separate the logical functionality so it can be re-used.
thanks
-john
Signed-off-by: John Stultz <[email protected]>
timer.c | 123 ++++++++++++++++++++++++++++++++++++++++++++--------------------
1 files changed, 85 insertions(+), 38 deletions(-)
linux-2.6.15-rc3-mm1_timeofday-ntp-part1_B12.patch
==========================
diff -ruN tod-mm1/kernel/timer.c tod-mm2/kernel/timer.c
--- tod-mm1/kernel/timer.c 2005-12-01 18:13:40.000000000 -0800
+++ tod-mm2/kernel/timer.c 2005-12-01 18:18:22.000000000 -0800
@@ -595,6 +595,7 @@
long time_reftime; /* time at last adjustment (s) */
long time_adjust;
long time_next_adjust;
+long time_adjust_step; /* per tick time_adjust step */
/*
* this routine handles the overflow of the microsecond field
@@ -722,45 +723,86 @@
#endif
}
-/* in the NTP reference this is called "hardclock()" */
-static void update_wall_time_one_tick(void)
+/**
+ * ntp_advance - increments the NTP state machine
+ * @interval_ns: interval, in nanoseconds
+ *
+ * Must be holding the xtime writelock when calling.
+ */
+static void ntp_advance(unsigned long interval_ns)
{
- long time_adjust_step, delta_nsec;
+ static unsigned long interval_sum;
- if ((time_adjust_step = time_adjust) != 0 ) {
- /*
- * We are doing an adjtime thing. Prepare time_adjust_step to
- * be within bounds. Note that a positive time_adjust means we
- * want the clock to run faster.
- *
- * Limit the amount of the step to be in the range
- * -tickadj .. +tickadj
- */
- time_adjust_step = min(time_adjust_step, (long)tickadj);
- time_adjust_step = max(time_adjust_step, (long)-tickadj);
+ /* increment the interval sum: */
+ interval_sum += interval_ns;
+
+ /* calculate the per tick singleshot adjtime adjustment step: */
+ while (interval_ns >= tick_nsec) {
+ time_adjust_step = time_adjust;
+ if (time_adjust_step) {
+ /*
+ * We are doing an adjtime thing.
+ *
+ * Prepare time_adjust_step to be within bounds.
+ * Note that a positive time_adjust means we want
+ * the clock to run faster.
+ *
+ * Limit the amount of the step to be in the range
+ * -tickadj .. +tickadj:
+ */
+ time_adjust_step = min(time_adjust_step, (long)tickadj);
+ time_adjust_step = max(time_adjust_step,
+ (long)-tickadj);
- /* Reduce by this step the amount of time left */
- time_adjust -= time_adjust_step;
- }
- delta_nsec = tick_nsec + time_adjust_step * 1000;
- /*
- * Advance the phase, once it gets to one microsecond, then
- * advance the tick more.
- */
- time_phase += time_adj;
- if ((time_phase >= FINENSEC) || (time_phase <= -FINENSEC)) {
- long ltemp = shift_right(time_phase, (SHIFT_SCALE - 10));
- time_phase -= ltemp << (SHIFT_SCALE - 10);
- delta_nsec += ltemp;
+ /* Reduce by this step the amount of time left: */
+ time_adjust -= time_adjust_step;
+ }
+ interval_ns -= tick_nsec;
}
- xtime.tv_nsec += delta_nsec;
- time_interpolator_update(delta_nsec);
/* Changes by adjtime() do not take effect till next tick. */
if (time_next_adjust != 0) {
time_adjust = time_next_adjust;
time_next_adjust = 0;
}
+
+ while (interval_sum >= NSEC_PER_SEC) {
+ interval_sum -= NSEC_PER_SEC;
+ second_overflow();
+ }
+}
+
+/**
+ * phase_advance - advance the phase
+ *
+ * advance the phase, once it gets to one nanosecond advance the tick more.
+ */
+static inline long phase_advance(void)
+{
+ long delta = 0;
+
+ time_phase += time_adj;
+
+ if ((time_phase >= FINENSEC) || (time_phase <= -FINENSEC)) {
+ delta = shift_right(time_phase, (SHIFT_SCALE - 10));
+ time_phase -= delta << (SHIFT_SCALE - 10);
+ }
+
+ return delta;
+}
+
+/**
+ * xtime_advance - advance xtime
+ * @delta_nsec: adjustment in nsecs
+ */
+static inline void xtime_advance(long delta_nsec)
+{
+ xtime.tv_nsec += delta_nsec;
+ if (likely(xtime.tv_nsec < NSEC_PER_SEC))
+ return;
+
+ xtime.tv_nsec -= NSEC_PER_SEC;
+ xtime.tv_sec++;
}
/*
@@ -768,19 +810,24 @@
* usually just one (we shouldn't be losing ticks,
* we're doing this this way mainly for interrupt
* latency reasons, not because we think we'll
- * have lots of lost timer ticks
+ * have lots of lost timer ticks)
*/
static void update_wall_time(unsigned long ticks)
{
do {
- ticks--;
- update_wall_time_one_tick();
- if (xtime.tv_nsec >= 1000000000) {
- xtime.tv_nsec -= 1000000000;
- xtime.tv_sec++;
- second_overflow();
- }
- } while (ticks);
+ /*
+ * Calculate the nsec delta using the precomputed NTP
+ * adjustments:
+ * tick_nsec, time_adjust_step, time_adj
+ */
+ long delta_nsec = tick_nsec + time_adjust_step * 1000;
+ delta_nsec += phase_advance();
+
+ xtime_advance(delta_nsec);
+ ntp_advance(tick_nsec);
+ time_interpolator_update(delta_nsec);
+
+ } while (--ticks);
}
/*
All,
The conversion of i386 to use the generic timeofday subsystem has been
split into 6 parts. This patch, the first of six, is just a simple
cleanup for the i386 arch in preparation of moving to the generic
timeofday infrastructure. It simply moves some code from timer_pit.c to
i8253.c.
It applies on top of my timeofday-core patch. This patch is part the
timeofday-arch-i386 patchset, so without the following parts it is not
expected to compile (although just this one should).
thanks
-john
Signed-off-by: John Stultz <[email protected]>
Makefile | 2 -
i8253.c | 59 +++++++++++++++++++++++++++++++++++++++++++++++++++++
time.c | 6 -----
timers/timer_pit.c | 13 -----------
4 files changed, 60 insertions(+), 20 deletions(-)
linux-2.6.15-rc3-mm1_timeofday-arch-i386-part1_B12.patch
==========================
diff -ruN tod-mm1/arch/i386/kernel/i8253.c tod-mm2/arch/i386/kernel/i8253.c
--- tod-mm1/arch/i386/kernel/i8253.c 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/arch/i386/kernel/i8253.c 2005-12-01 18:23:24.000000000 -0800
@@ -0,0 +1,59 @@
+/*
+ * i8253.c 8253/PIT functions
+ *
+ */
+#include <linux/spinlock.h>
+#include <linux/jiffies.h>
+#include <linux/sysdev.h>
+#include <linux/module.h>
+#include <linux/init.h>
+
+#include <asm/smp.h>
+#include <asm/delay.h>
+#include <asm/i8253.h>
+#include <asm/io.h>
+
+#include "io_ports.h"
+
+DEFINE_SPINLOCK(i8253_lock);
+EXPORT_SYMBOL(i8253_lock);
+
+void setup_pit_timer(void)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&i8253_lock, flags);
+ outb_p(0x34,PIT_MODE); /* binary, mode 2, LSB/MSB, ch 0 */
+ udelay(10);
+ outb_p(LATCH & 0xff , PIT_CH0); /* LSB */
+ udelay(10);
+ outb(LATCH >> 8 , PIT_CH0); /* MSB */
+ spin_unlock_irqrestore(&i8253_lock, flags);
+}
+
+static int timer_resume(struct sys_device *dev)
+{
+ setup_pit_timer();
+
+ return 0;
+}
+
+static struct sysdev_class timer_sysclass = {
+ set_kset_name("timer_pit"),
+ .resume = timer_resume,
+};
+
+static struct sys_device device_timer = {
+ .id = 0,
+ .cls = &timer_sysclass,
+};
+
+static int __init init_timer_sysfs(void)
+{
+ int error = sysdev_class_register(&timer_sysclass);
+ if (!error)
+ error = sysdev_register(&device_timer);
+ return error;
+}
+
+device_initcall(init_timer_sysfs);
diff -ruN tod-mm1/arch/i386/kernel/Makefile tod-mm2/arch/i386/kernel/Makefile
--- tod-mm1/arch/i386/kernel/Makefile 2005-12-01 18:12:56.000000000 -0800
+++ tod-mm2/arch/i386/kernel/Makefile 2005-12-01 18:23:57.000000000 -0800
@@ -7,7 +7,7 @@
obj-y := process.o semaphore.o signal.o entry.o traps.o irq.o vm86.o \
ptrace.o time.o ioport.o ldt.o setup.o i8259.o sys_i386.o \
pci-dma.o i386_ksyms.o i387.o dmi_scan.o bootflag.o \
- quirks.o i8237.o
+ quirks.o i8237.o i8253.o
obj-y += cpu/
obj-y += timers/
diff -ruN tod-mm1/arch/i386/kernel/time.c tod-mm2/arch/i386/kernel/time.c
--- tod-mm1/arch/i386/kernel/time.c 2005-12-01 18:12:56.000000000 -0800
+++ tod-mm2/arch/i386/kernel/time.c 2005-12-01 18:23:24.000000000 -0800
@@ -82,11 +82,6 @@
DEFINE_SPINLOCK(rtc_lock);
EXPORT_SYMBOL(rtc_lock);
-#include <asm/i8253.h>
-
-DEFINE_SPINLOCK(i8253_lock);
-EXPORT_SYMBOL(i8253_lock);
-
struct timer_opts *cur_timer __read_mostly = &timer_none;
/*
@@ -400,7 +395,6 @@
if (is_hpet_enabled())
hpet_reenable();
#endif
- setup_pit_timer();
sec = get_cmos_time() + clock_cmos_diff;
sleep_length = (get_cmos_time() - sleep_start) * HZ;
write_seqlock_irqsave(&xtime_lock, flags);
diff -ruN tod-mm1/arch/i386/kernel/timers/timer_pit.c tod-mm2/arch/i386/kernel/timers/timer_pit.c
--- tod-mm1/arch/i386/kernel/timers/timer_pit.c 2005-12-01 18:12:56.000000000 -0800
+++ tod-mm2/arch/i386/kernel/timers/timer_pit.c 2005-12-01 18:23:24.000000000 -0800
@@ -162,16 +162,3 @@
.init = init_pit,
.opts = &timer_pit,
};
-
-void setup_pit_timer(void)
-{
- unsigned long flags;
-
- spin_lock_irqsave(&i8253_lock, flags);
- outb_p(0x34,PIT_MODE); /* binary, mode 2, LSB/MSB, ch 0 */
- udelay(10);
- outb_p(LATCH & 0xff , PIT_CH0); /* LSB */
- udelay(10);
- outb(LATCH >> 8 , PIT_CH0); /* MSB */
- spin_unlock_irqrestore(&i8253_lock, flags);
-}
All,
This patch introduces the clocksource management infrastructure. A
clocksource is a driver-like architecture generic abstraction of a
freerunning counter. This patch defines the clocksource structure, and
provides management code for registering, selecting, accessing and
scaling clocksources. The clocksource structure is influenced by the
time_interpolator code, although I feel it has a cleaner interface and
avoids preserving system state in the clocksource structure.
Additionally, this patch includes the trivial jiffies clocksource, a
lowest common denominator clocksource, provided mainly for use as an
example.
This patch applies on top of my ntp cleanup patchset.
Since this patch provides the groundwork for the generic timeofday core,
it will not function without the generic timeofday patches to follow.
thanks
-john
Signed-off-by: John Stultz <[email protected]>
Documentation/kernel-parameters.txt | 14 +
include/linux/clocksource.h | 304 ++++++++++++++++++++++++++++++++++
kernel/Makefile | 1
kernel/time/Makefile | 1
kernel/time/clocksource.c | 313 ++++++++++++++++++++++++++++++++++++
kernel/time/jiffies.c | 75 ++++++++
6 files changed, 704 insertions(+), 4 deletions(-)
linux-2.6.15-rc3-mm1_timeofday-clocksource-core_B12.patch
============================================
diff -ruN tod-mm1/Documentation/kernel-parameters.txt tod-mm2/Documentation/kernel-parameters.txt
--- tod-mm1/Documentation/kernel-parameters.txt 2005-12-01 18:54:24.000000000 -0800
+++ tod-mm2/Documentation/kernel-parameters.txt 2005-12-01 18:58:45.000000000 -0800
@@ -52,6 +52,7 @@
MTD MTD support is enabled.
NET Appropriate network support is enabled.
NUMA NUMA support is enabled.
+ GENERIC_TIME The generic timeofday code is enabled.
NFS Appropriate NFS support is enabled.
OSS OSS sound support is enabled.
PARIDE The ParIDE subsystem is enabled.
@@ -329,10 +330,11 @@
Value can be changed at runtime via
/selinux/checkreqprot.
- clock= [BUGS=IA-32,HW] gettimeofday timesource override.
- Forces specified timesource (if avaliable) to be used
- when calculating gettimeofday(). If specicified
- timesource is not avalible, it defaults to PIT.
+ clock= [BUGS=IA-32, HW] gettimeofday clocksource override.
+ [Deprecated]
+ Forces specified clocksource (if avaliable) to be used
+ when calculating gettimeofday(). If specified
+ clocksource is not avalible, it defaults to PIT.
Format: { pit | tsc | cyclone | pmtmr }
hpet= [IA-32,HPET] option to disable HPET and use PIT.
@@ -1478,6 +1480,10 @@
time Show timing data prefixed to each printk message line
+ clocksource= [GENERIC_TIME] Override the default clocksource
+ Override the default clocksource and use the clocksource
+ with the name specified.
+
tipar.timeout= [HW,PPT]
Set communications timeout in tenths of a second
(default 15).
diff -ruN tod-mm1/include/linux/clocksource.h tod-mm2/include/linux/clocksource.h
--- tod-mm1/include/linux/clocksource.h 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/include/linux/clocksource.h 2005-12-01 18:58:45.000000000 -0800
@@ -0,0 +1,304 @@
+/* linux/include/linux/clocksource.h
+ *
+ * This file contains the structure definitions for clocksources.
+ *
+ * If you are not a clocksource, or the time of day code, you should
+ * not be including this file!
+ */
+#ifndef _LINUX_CLOCKSOURCE_H
+#define _LINUX_CLOCKSOURCE_H
+
+#include <linux/types.h>
+#include <linux/timex.h>
+#include <linux/time.h>
+#include <linux/list.h>
+#include <asm/div64.h>
+#include <asm/io.h>
+
+/**
+ * struct clocksource - hardware abstraction for a free running counter
+ * Provides mostly state-free accessors to the underlying hardware.
+ *
+ * @name: ptr to clocksource name
+ * @list: list head for registration
+ * @rating: rating value for selection (higher is better)
+ * To avoid rating inflation the following
+ * list should give you a guide as to how
+ * to assign your clocksource a rating
+ * 1-99: Unfit for real use
+ * Only available for bootup and testing purposes.
+ * 100-199: Base level usability.
+ * Functional for real use, but not desired.
+ * 200-299: Good.
+ * A correct and usable clocksource.
+ * 300-399: Desired.
+ * A reasonably fast and accurate clocksource.
+ * 400-499: Perfect
+ * The ideal clocksource. A must-use where
+ * available.
+ * @read: returns a cycle value
+ * @mask: bitmask for two's complement
+ * subtraction of non 64 bit counters
+ * @mult: cycle to nanosecond multiplier
+ * @shift: cycle to nanosecond divisor (power of two)
+ * @update_callback: called when safe to alter clocksource values
+ * @is_continuous: defines if clocksource is free-running.
+ * @vread: vsyscall read function
+ * @vdata: vsyscall data value passed to read function
+ */
+struct clocksource {
+ char *name;
+ struct list_head list;
+ int rating;
+ cycle_t (*read)(void);
+ cycle_t mask;
+ u32 mult;
+ u32 shift;
+ int (*update_callback)(void);
+ int is_continuous;
+ cycle_t (*vread)(void *);
+ void *vdata;
+};
+
+
+/**
+ * clocksource_khz2mult - calculates mult from khz and shift
+ * @khz: Clocksource frequency in KHz
+ * @shift_constant: Clocksource shift factor
+ *
+ * Helper functions that converts a khz counter frequency to a timsource
+ * multiplier, given the clocksource shift value
+ */
+static inline u32 clocksource_khz2mult(u32 khz, u32 shift_constant)
+{
+ /* khz = cyc/(Million ns)
+ * mult/2^shift = ns/cyc
+ * mult = ns/cyc * 2^shift
+ * mult = 1Million/khz * 2^shift
+ * mult = 1000000 * 2^shift / khz
+ * mult = (1000000<<shift) / khz
+ */
+ u64 tmp = ((u64)1000000) << shift_constant;
+
+ tmp += khz/2; /* round for do_div */
+ do_div(tmp, khz);
+
+ return (u32)tmp;
+}
+
+/**
+ * clocksource_hz2mult - calculates mult from hz and shift
+ * @hz: Clocksource frequency in Hz
+ * @shift_constant: Clocksource shift factor
+ *
+ * Helper functions that converts a hz counter
+ * frequency to a timsource multiplier, given the
+ * clocksource shift value
+ */
+static inline u32 clocksource_hz2mult(u32 hz, u32 shift_constant)
+{
+ /* hz = cyc/(Billion ns)
+ * mult/2^shift = ns/cyc
+ * mult = ns/cyc * 2^shift
+ * mult = 1Billion/hz * 2^shift
+ * mult = 1000000000 * 2^shift / hz
+ * mult = (1000000000<<shift) / hz
+ */
+ u64 tmp = ((u64)1000000000) << shift_constant;
+
+ tmp += hz/2; /* round for do_div */
+ do_div(tmp, hz);
+
+ return (u32)tmp;
+}
+
+/**
+ * read_clocksource: - Access the clocksource's current cycle value
+ * @cs: pointer to clocksource being read
+ *
+ * Uses the clocksource to return the current cycle_t value
+ */
+static inline cycle_t read_clocksource(struct clocksource *cs)
+{
+ return cs->read();
+}
+
+/**
+ * ppm_to_mult_adj - Converts shifted ppm values to mult adjustment
+ * @cs: Pointer to clocksource
+ * @ppm: Shifted PPM value
+ *
+ * Helper which converts a shifted ppm value to clocksource mult_adj value.
+ *
+ * XXX - this could use some optimization
+ */
+static inline int ppm_to_mult_adj(struct clocksource *cs, int ppm)
+{
+ u64 mult_adj;
+ int ret_adj;
+
+ /* The basic math is as follows:
+ * cyc * mult/2^shift * (1 + ppm/MILL) = scaled ns
+ * We want to precalculate the ppm factor so it can be added
+ * to the multiplyer saving the extra multiplication step.
+ * cyc * (mult/2^shift + (mult/2^shift) * (ppm/MILL)) =
+ * cyc * (mult/2^shift + (mult*ppm/MILL)/2^shift) =
+ * cyc * (mult + (mult*ppm/MILL))/2^shift =
+ * Thus we want to calculate the value of:
+ * mult*ppm/MILL
+ */
+ mult_adj = abs(ppm);
+ mult_adj = (mult_adj * cs->mult)>>SHIFT_USEC;
+ mult_adj += 1000000/2; /* round for div*/
+ do_div(mult_adj, 1000000);
+ if (ppm < 0)
+ ret_adj = -(int)mult_adj;
+ else
+ ret_adj = (int)mult_adj;
+
+ return ret_adj;
+}
+
+/**
+ * cyc2ns - converts clocksource cycles to nanoseconds
+ * @cs: Pointer to clocksource
+ * @ntp_adj: Multiplier adjustment value
+ * @cycles: Cycles
+ *
+ * Uses the clocksource and ntp ajdustment to convert cycle_ts to nanoseconds.
+ *
+ * XXX - This could use some mult_lxl_ll() asm optimization
+ */
+static inline nsec_t cyc2ns(struct clocksource *cs, int ntp_adj, cycle_t cycles)
+{
+ u64 ret = (u64)cycles;
+
+ ret *= (cs->mult + ntp_adj);
+ ret >>= cs->shift;
+
+ return (nsec_t)ret;
+}
+
+/**
+ * cyc2ns_rem - converts clocksource cycles to nanoseconds w/ remainder
+ * @cs: Pointer to clocksource
+ * @ntp_adj: Multiplier adjustment value
+ * @cycles: Cycles
+ * @rem: Remainder
+ *
+ * Uses the clocksource and ntp ajdustment interval to convert cycle_t to
+ * nanoseconds. Add in remainder portion which is stored in (ns<<cs->shift)
+ * units and save the new remainder off.
+ *
+ * XXX - This could use some mult_lxl_ll() asm optimization.
+ */
+static inline nsec_t cyc2ns_rem(struct clocksource *cs, int ntp_adj,
+ cycle_t cycles, u64* rem)
+{
+ u64 ret = (u64)cycles;
+
+ ret *= (cs->mult + ntp_adj);
+ if (rem) {
+ ret += *rem;
+ *rem = ret & ((1<<cs->shift)-1);
+ }
+ ret >>= cs->shift;
+
+ return (nsec_t)ret;
+}
+
+
+/**
+ * struct clocksource_interval - Fixed interval conversion structure
+ *
+ * @cycles: A specified number of cycles
+ * @nsecs: The number of nanoseconds equivalent to the cycles value
+ * @remainder: Non-integer nanosecond remainder stored in (ns<<cs->shift) units
+ * @remainder_ns_overflow: Value at which the remainder is equal to
+ * one second
+ *
+ * This is a optimization structure used by cyc2ns_fixed_rem() to avoid the
+ * multiply in cyc2ns().
+ *
+ * Unless you're the timeofday_periodic_hook, you should not be using this!
+ */
+struct clocksource_interval {
+ cycle_t cycles;
+ nsec_t nsecs;
+ u64 remainder;
+ u64 remainder_ns_overflow;
+};
+
+/**
+ * calculate_clocksource_interval - Calculates a clocksource interval struct
+ *
+ * @c: Pointer to clocksource.
+ * @adj: Multiplyer adjustment.
+ * @length_nsec: Desired interval length in nanoseconds.
+ *
+ * Calculates a fixed cycle/nsec interval for a given clocksource/adjustment
+ * pair and interval request.
+ *
+ * Unless you're the timeofday_periodic_hook, you should not be using this!
+ */
+static inline struct clocksource_interval
+calculate_clocksource_interval(struct clocksource *c, long adj,
+ unsigned long length_nsec)
+{
+ struct clocksource_interval ret;
+ u64 tmp;
+
+ /* XXX - All of this could use a whole lot of optimization */
+ tmp = length_nsec;
+ tmp <<= c->shift;
+ do_div(tmp, c->mult+adj);
+
+ ret.cycles = (cycle_t)tmp;
+ if(ret.cycles == 0)
+ ret.cycles = 1;
+
+ ret.remainder = 0;
+ ret.remainder_ns_overflow = 1 << c->shift;
+ ret.nsecs = cyc2ns_rem(c, adj, ret.cycles, &ret.remainder);
+
+ return ret;
+}
+
+/**
+ * cyc2ns_fixed_rem -
+ * converts clocksource cycles to nanoseconds using fixed intervals
+ *
+ * @interval: precalculated clocksource_interval structure
+ * @cycles: Number of clocksource cycles
+ * @rem: Remainder
+ *
+ * Uses a precalculated fixed cycle/nsec interval to convert cycles to
+ * nanoseconds. Returns the unaccumulated cycles in the cycles pointer as
+ * well as uses and updates the value at the remainder pointer
+ *
+ * Unless you're the timeofday_periodic_hook, you should not be using this!
+ */
+static inline nsec_t cyc2ns_fixed_rem(struct clocksource_interval interval,
+ cycle_t *cycles, u64* rem)
+{
+ nsec_t delta_nsec = 0;
+
+ while (*cycles > interval.cycles) {
+ delta_nsec += interval.nsecs;
+ *cycles -= interval.cycles;
+ *rem += interval.remainder;
+ while(*rem > interval.remainder_ns_overflow) {
+ *rem -= interval.remainder_ns_overflow;
+ delta_nsec += 1;
+ }
+ }
+
+ return delta_nsec;
+}
+
+/* used to install a new clocksource */
+void register_clocksource(struct clocksource*);
+void reselect_clocksource(void);
+struct clocksource* get_next_clocksource(void);
+
+#endif /* _LINUX_CLOCKSOURCE_H */
diff -ruN tod-mm1/kernel/Makefile tod-mm2/kernel/Makefile
--- tod-mm1/kernel/Makefile 2005-12-01 18:55:14.000000000 -0800
+++ tod-mm2/kernel/Makefile 2005-12-01 18:59:29.000000000 -0800
@@ -11,6 +11,7 @@
ktimers.o
obj-$(CONFIG_OBSOLETE_INTERMODULE) += intermodule.o
+obj-$(CONFIG_GENERIC_TIME) += time/
obj-$(CONFIG_FUTEX) += futex.o
obj-$(CONFIG_GENERIC_ISA_DMA) += dma.o
obj-$(CONFIG_SMP) += cpu.o spinlock.o
diff -ruN tod-mm1/kernel/time/clocksource.c tod-mm2/kernel/time/clocksource.c
--- tod-mm1/kernel/time/clocksource.c 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/kernel/time/clocksource.c 2005-12-01 18:58:45.000000000 -0800
@@ -0,0 +1,313 @@
+/*
+ * linux/kernel/time/clocksource.c
+ *
+ * This file contains the functions which manage clocksource drivers.
+ *
+ * Copyright (C) 2004, 2005 IBM, John Stultz ([email protected])
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ *
+ * TODO WishList:
+ * o Allow clocksource drivers to be unregistered
+ * o get rid of clocksource_jiffies extern
+ */
+
+#include <linux/clocksource.h>
+#include <linux/sysdev.h>
+#include <linux/init.h>
+#include <linux/module.h>
+
+/* XXX - Would like a better way for initializing curr_clocksource */
+extern struct clocksource clocksource_jiffies;
+
+/*[Clocksource internal variables]---------
+ * curr_clocksource:
+ * currently selected clocksource. Initialized to clocksource_jiffies.
+ * next_clocksource:
+ * pending next selected clocksource.
+ * clocksource_list:
+ * linked list with the registered clocksources
+ * clocksource_lock:
+ * protects manipulations to curr_clocksource and next_clocksource
+ * and the clocksource_list
+ * override_name:
+ * Name of the user-specified clocksource.
+ */
+static struct clocksource *curr_clocksource = &clocksource_jiffies;
+static struct clocksource *next_clocksource;
+static LIST_HEAD(clocksource_list);
+static DEFINE_SPINLOCK(clocksource_lock);
+static char override_name[32];
+
+/**
+ * get_next_clocksource - Returns the selected clocksource
+ */
+struct clocksource *get_next_clocksource(void)
+{
+ spin_lock(&clocksource_lock);
+ if (next_clocksource) {
+ curr_clocksource = next_clocksource;
+ next_clocksource = NULL;
+ }
+ spin_unlock(&clocksource_lock);
+
+ return curr_clocksource;
+}
+
+/**
+ * select_clocksource - Finds the best registered clocksource.
+ *
+ * Private function. Must hold clocksource_lock when called.
+ */
+static struct clocksource *select_clocksource(void)
+{
+ struct clocksource *best = NULL;
+ struct list_head *tmp;
+
+ list_for_each(tmp, &clocksource_list) {
+ struct clocksource *src;
+
+ src = list_entry(tmp, struct clocksource, list);
+ if (!best)
+ best = src;
+
+ /* check for override: */
+ if (strlen(src->name) == strlen(override_name) &&
+ !strcmp(src->name, override_name)) {
+ best = src;
+ break;
+ }
+ /* pick the highest rating: */
+ if (src->rating > best->rating)
+ best = src;
+ }
+
+ return best;
+}
+
+/**
+ * is_registered_source - Checks if clocksource is registered
+ * @c: pointer to a clocksource
+ *
+ * Private helper function. Must hold clocksource_lock when called.
+ *
+ * Returns one if the clocksource is already registered, zero otherwise.
+ */
+static inline int is_registered_source(struct clocksource *c)
+{
+ int len = strlen(c->name);
+ struct list_head *tmp;
+
+ list_for_each(tmp, &clocksource_list) {
+ struct clocksource *src;
+
+ src = list_entry(tmp, struct clocksource, list);
+ if (strlen(src->name) == len && !strcmp(src->name, c->name))
+ return 1;
+ }
+
+ return 0;
+}
+
+/**
+ * register_clocksource - Used to install new clocksources
+ * @t: clocksource to be registered
+ */
+void register_clocksource(struct clocksource *c)
+{
+ spin_lock(&clocksource_lock);
+
+ /* check if clocksource is already registered */
+ if (is_registered_source(c)) {
+ printk("register_clocksource: Cannot register %s. Already registered!",
+ c->name);
+ } else {
+ list_add(&c->list, &clocksource_list);
+ /* select next clocksource */
+ next_clocksource = select_clocksource();
+ }
+ spin_unlock(&clocksource_lock);
+}
+
+EXPORT_SYMBOL(register_clocksource);
+
+/**
+ * reselect_clocksource - Rescan list for next clocksource
+ *
+ * A quick helper function to be used if a clocksource changes its
+ * rating. Forces the clocksource list to be re-scaned for the best
+ * clocksource.
+ */
+void reselect_clocksource(void)
+{
+ spin_lock(&clocksource_lock);
+ next_clocksource = select_clocksource();
+ spin_unlock(&clocksource_lock);
+}
+
+/**
+ * sysfs_show_current_clocksources - sysfs interface for current clocksource
+ * @dev: unused
+ * @buf: char buffer to be filled with clocksource list
+ *
+ * Provides sysfs interface for listing current clocksource.
+ */
+static ssize_t
+sysfs_show_current_clocksources(struct sys_device *dev, char *buf)
+{
+ char *curr = buf;
+
+ spin_lock(&clocksource_lock);
+ curr += sprintf(curr, "%s ", curr_clocksource->name);
+ spin_unlock(&clocksource_lock);
+
+ curr += sprintf(curr, "\n");
+
+ return curr - buf;
+}
+
+/**
+ * sysfs_override_clocksource - interface for manually overriding clocksource
+ * @dev: unused
+ * @buf: name of override clocksource
+ * @count: length of buffer
+ *
+ * Takes input from sysfs interface for manually overriding the default
+ * clocksource selction.
+ */
+static ssize_t sysfs_override_clocksource(struct sys_device *dev,
+ const char *buf, size_t count)
+{
+ /* strings from sysfs write are not 0 terminated! */
+ if (count >= sizeof(override_name))
+ return -EINVAL;
+
+ /* strip of \n: */
+ if (buf[count-1] == '\n')
+ count--;
+ if (count < 1)
+ return -EINVAL;
+
+ spin_lock(&clocksource_lock);
+
+ /* copy the name given: */
+ memcpy(override_name, buf, count);
+ override_name[count] = 0;
+
+ /* try to select it: */
+ next_clocksource = select_clocksource();
+
+ spin_unlock(&clocksource_lock);
+
+ return count;
+}
+
+/**
+ * sysfs_show_available_clocksources - sysfs interface for listing clocksource
+ * @dev: unused
+ * @buf: char buffer to be filled with clocksource list
+ *
+ * Provides sysfs interface for listing registered clocksources
+ */
+static ssize_t
+sysfs_show_available_clocksources(struct sys_device *dev, char *buf)
+{
+ struct list_head *tmp;
+ char *curr = buf;
+
+ spin_lock(&clocksource_lock);
+ list_for_each(tmp, &clocksource_list) {
+ struct clocksource *src;
+
+ src = list_entry(tmp, struct clocksource, list);
+ curr += sprintf(curr, "%s ", src->name);
+ }
+ spin_unlock(&clocksource_lock);
+
+ curr += sprintf(curr, "\n");
+
+ return curr - buf;
+}
+
+/*
+ * Sysfs setup bits:
+ */
+static SYSDEV_ATTR(current_clocksource, 0600, sysfs_show_current_clocksources,
+ sysfs_override_clocksource);
+
+static SYSDEV_ATTR(available_clocksource, 0600,
+ sysfs_show_available_clocksources, NULL);
+
+static struct sysdev_class clocksource_sysclass = {
+ set_kset_name("clocksource"),
+};
+
+static struct sys_device device_clocksource = {
+ .id = 0,
+ .cls = &clocksource_sysclass,
+};
+
+static int init_clocksource_sysfs(void)
+{
+ int error = sysdev_class_register(&clocksource_sysclass);
+
+ if (!error)
+ error = sysdev_register(&device_clocksource);
+ if (!error)
+ error = sysdev_create_file(
+ &device_clocksource,
+ &attr_current_clocksource);
+ if (!error)
+ error = sysdev_create_file(
+ &device_clocksource,
+ &attr_available_clocksource);
+ return error;
+}
+
+device_initcall(init_clocksource_sysfs);
+
+/**
+ * boot_override_clocksource - boot clock override
+ * @str: override name
+ *
+ * Takes a clocksource= boot argument and uses it
+ * as the clocksource override name.
+ */
+static int __init boot_override_clocksource(char* str)
+{
+ spin_lock(&clocksource_lock);
+ if (str)
+ strlcpy(override_name, str, sizeof(override_name));
+ spin_unlock(&clocksource_lock);
+ return 1;
+}
+
+__setup("clocksource=", boot_override_clocksource);
+
+/**
+ * boot_override_clock - Compatibility layer for deprecated boot option
+ * @str: override name
+ *
+ * DEPRECATED! Takes a clock= boot argument and uses it
+ * as the clocksource override name
+ */
+static int __init boot_override_clock(char* str)
+{
+ printk("Warning! clock= boot option is deprecated.\n");
+
+ return boot_override_clocksource(str);
+}
+
+__setup("clock=", boot_override_clock);
diff -ruN tod-mm1/kernel/time/jiffies.c tod-mm2/kernel/time/jiffies.c
--- tod-mm1/kernel/time/jiffies.c 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/kernel/time/jiffies.c 2005-12-01 18:58:45.000000000 -0800
@@ -0,0 +1,75 @@
+/***********************************************************************
+* linux/kernel/time/jiffies.c
+*
+* This file contains the jiffies based clocksource.
+*
+* Copyright (C) 2004, 2005 IBM, John Stultz ([email protected])
+*
+* This program is free software; you can redistribute it and/or modify
+* it under the terms of the GNU General Public License as published by
+* the Free Software Foundation; either version 2 of the License, or
+* (at your option) any later version.
+*
+* This program is distributed in the hope that it will be useful,
+* but WITHOUT ANY WARRANTY; without even the implied warranty of
+* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+* GNU General Public License for more details.
+*
+* You should have received a copy of the GNU General Public License
+* along with this program; if not, write to the Free Software
+* Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+*
+************************************************************************/
+#include <linux/clocksource.h>
+#include <linux/jiffies.h>
+#include <linux/init.h>
+
+/* The Jiffies based clocksource is the lowest common
+ * denominator clock source which should function on
+ * all systems. It has the same coarse resolution as
+ * the timer interrupt frequency HZ and it suffers
+ * inaccuracies caused by missed or lost timer
+ * interrupts and the inability for the timer
+ * interrupt hardware to accuratly tick at the
+ * requested HZ value. It is also not reccomended
+ * for "tick-less" systems.
+ */
+#define NSEC_PER_JIFFY ((u32)((((u64)NSEC_PER_SEC)<<8)/ACTHZ))
+
+/* Since jiffies uses a simple NSEC_PER_JIFFY multiplier
+ * conversion, the .shift value could be zero. However
+ * this would make NTP adjustments impossible as they are
+ * in units of 1/2^.shift. Thus we use JIFFIES_SHIFT to
+ * shift both the nominator and denominator the same
+ * amount, and give ntp adjustments in units of 1/2^8
+ *
+ * The value 8 is somewhat carefully chosen, as anything
+ * larger can result in overflows. NSEC_PER_JIFFY grows as
+ * HZ shrinks, so values greater then 8 overflow 32bits when
+ * HZ=100.
+ */
+#define JIFFIES_SHIFT 8
+
+static cycle_t jiffies_read(void)
+{
+ return (cycle_t) get_jiffies_64();
+}
+
+struct clocksource clocksource_jiffies = {
+ .name = "jiffies",
+ .rating = 0, /* lowest rating*/
+ .read = jiffies_read,
+ .mask = (cycle_t)-1,
+ .mult = NSEC_PER_JIFFY << JIFFIES_SHIFT, /* details above */
+ .shift = JIFFIES_SHIFT,
+ .is_continuous = 0, /* tick based, not free running */
+};
+
+static int __init init_jiffies_clocksource(void)
+{
+ register_clocksource(&clocksource_jiffies);
+
+ return 0;
+}
+
+module_init(init_jiffies_clocksource);
diff -ruN tod-mm1/kernel/time/Makefile tod-mm2/kernel/time/Makefile
--- tod-mm1/kernel/time/Makefile 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/kernel/time/Makefile 2005-12-01 18:58:45.000000000 -0800
@@ -0,0 +1 @@
+obj-y += clocksource.o jiffies.o
All,
Here is the second of two patches which try to minimize my ntp rework
patches.
This patch further changes the interrupt time NTP code, breaking out the
leapsecond processing and introduces an accessor to a shifted ppm
adjustment value. For correctness, I've also introduced a new lock, the
ntp_lock, which protects the NTP state machine when accessing it from my
timekeeping code (which does not use the xtime_lock).
Again, this patch should not affect the existing behavior, but just
separate the logical functionality so it can be re-used by my timeofday
patches.
thanks
-john
Signed-off-by: John Stultz <[email protected]>
include/linux/timex.h | 23 +++++++
kernel/time.c | 8 +-
kernel/timer.c | 144 ++++++++++++++++++++++++++++++++++++++++++++++++--
3 files changed, 165 insertions(+), 10 deletions(-)
linux-2.6.15-rc3-mm1_timeofday-ntp-part2_B12.patch
==========================
diff -ruN tod-mm1/include/linux/timex.h tod-mm2/include/linux/timex.h
--- tod-mm1/include/linux/timex.h 2005-12-01 18:13:36.000000000 -0800
+++ tod-mm2/include/linux/timex.h 2005-12-01 18:20:41.000000000 -0800
@@ -260,6 +260,8 @@
extern long pps_errcnt; /* calibration errors */
extern long pps_stbcnt; /* stability limit exceeded */
+extern seqlock_t ntp_lock;
+
/**
* ntp_clear - Clears the NTP state variables
*
@@ -267,21 +269,40 @@
*/
static inline void ntp_clear(void)
{
+ unsigned long flags;
+
+ write_seqlock_irqsave(&ntp_lock, flags);
time_adjust = 0; /* stop active adjtime() */
time_status |= STA_UNSYNC;
time_maxerror = NTP_PHASE_LIMIT;
time_esterror = NTP_PHASE_LIMIT;
+ write_sequnlock_irqrestore(&ntp_lock, flags);
}
/**
* ntp_synced - Returns 1 if the NTP status is not UNSYNC
- *
*/
static inline int ntp_synced(void)
{
return !(time_status & STA_UNSYNC);
}
+/**
+ * ntp_get_ppm_adjustment - Returns Shifted PPM adjustment
+ */
+extern long ntp_get_ppm_adjustment(void);
+
+/**
+ * ntp_advance - Advances the NTP state machine by interval_ns
+ */
+extern void ntp_advance(unsigned long interval_ns);
+
+/**
+ * ntp_leapsecond - NTP leapsecond processing code.
+ */
+extern int ntp_leapsecond(struct timespec now);
+
+
/* Required to safely shift negative values */
#define shift_right(x, s) ({ \
__typeof__(x) __x = (x); \
diff -ruN tod-mm1/kernel/time.c tod-mm2/kernel/time.c
--- tod-mm1/kernel/time.c 2005-12-01 18:13:40.000000000 -0800
+++ tod-mm2/kernel/time.c 2005-12-01 18:20:41.000000000 -0800
@@ -255,6 +255,8 @@
return -EINVAL;
write_seqlock_irq(&xtime_lock);
+ write_seqlock(&ntp_lock);
+
result = time_state; /* mostly `TIME_OK' */
/* Save for later - semantics of adjtime is to return old value */
@@ -392,6 +394,7 @@
txc->calcnt = pps_calcnt;
txc->errcnt = pps_errcnt;
txc->stbcnt = pps_stbcnt;
+ write_sequnlock(&ntp_lock);
write_sequnlock_irq(&xtime_lock);
do_gettimeofday(&txc->time);
notify_arch_cmos_timer();
@@ -509,10 +512,7 @@
set_normalized_timespec(&xtime, sec, nsec);
set_normalized_timespec(&wall_to_monotonic, wtm_sec, wtm_nsec);
- time_adjust = 0; /* stop active adjtime() */
- time_status |= STA_UNSYNC;
- time_maxerror = NTP_PHASE_LIMIT;
- time_esterror = NTP_PHASE_LIMIT;
+ ntp_clear();
time_interpolator_reset();
}
write_sequnlock_irq(&xtime_lock);
diff -ruN tod-mm1/kernel/timer.c tod-mm2/kernel/timer.c
--- tod-mm1/kernel/timer.c 2005-12-01 18:20:20.000000000 -0800
+++ tod-mm2/kernel/timer.c 2005-12-01 18:20:41.000000000 -0800
@@ -588,7 +588,6 @@
long time_precision = 1; /* clock precision (us) */
long time_maxerror = NTP_PHASE_LIMIT; /* maximum error (us) */
long time_esterror = NTP_PHASE_LIMIT; /* estimated error (us) */
-static long time_phase; /* phase offset (scaled us) */
long time_freq = (((NSEC_PER_SEC + HZ/2) % HZ - HZ/2) << SHIFT_USEC) / NSEC_PER_USEC;
/* frequency offset (scaled ppm)*/
static long time_adj; /* tick adjust (scaled 1 / HZ) */
@@ -597,8 +596,91 @@
long time_next_adjust;
long time_adjust_step; /* per tick time_adjust step */
+long total_sppm; /* shifted ppm sum of all adjustments */
+long offset_adj_ppm;
+long tick_adj_ppm;
+long singleshot_adj_ppm;
+
+#define MAX_SINGLESHOT_ADJ 500 /* (ppm) */
+#define SEC_PER_DAY 86400
+#define END_OF_DAY(x) ((x) + SEC_PER_DAY - ((x) % SEC_PER_DAY) - 1)
+
+/* NTP lock, protects NTP state machine */
+seqlock_t ntp_lock = SEQLOCK_UNLOCKED;
+
+/**
+ * ntp_leapsecond - NTP leapsecond processing code.
+ * now: the current time
+ *
+ * Returns the number of seconds (-1, 0, or 1) that
+ * should be added to the current time to properly
+ * adjust for leapseconds.
+ */
+int ntp_leapsecond(struct timespec now)
+{
+ /*
+ * Leap second processing. If in leap-insert state at
+ * the end of the day, the system clock is set back one
+ * second; if in leap-delete state, the system clock is
+ * set ahead one second.
+ */
+ static time_t leaptime = 0;
+
+ unsigned long flags;
+ int ret = 0;
+
+ write_seqlock_irqsave(&ntp_lock, flags);
+
+ switch (time_state) {
+
+ case TIME_OK:
+ if (time_status & STA_INS) {
+ time_state = TIME_INS;
+ leaptime = END_OF_DAY(now.tv_sec);
+ } else if (time_status & STA_DEL) {
+ time_state = TIME_DEL;
+ leaptime = END_OF_DAY(now.tv_sec);
+ }
+ break;
+
+ case TIME_INS:
+ /* Once we are at (or past) leaptime, insert the second */
+ if (now.tv_sec >= leaptime) {
+ time_state = TIME_OOP;
+ printk(KERN_NOTICE "Clock: inserting leap second 23:59:60 UTC\n");
+ ret = -1;
+ }
+ break;
+
+ case TIME_DEL:
+ /* Once we are at (or past) leaptime, delete the second */
+ if (now.tv_sec >= leaptime) {
+ time_state = TIME_WAIT;
+ printk(KERN_NOTICE "Clock: deleting leap second 23:59:59 UTC\n");
+ ret = 1;
+ }
+ break;
+
+ case TIME_OOP:
+ /* Wait for the end of the leap second */
+ if (now.tv_sec > (leaptime + 1))
+ time_state = TIME_WAIT;
+ time_state = TIME_WAIT;
+ break;
+
+ case TIME_WAIT:
+ if (!(time_status & (STA_INS | STA_DEL)))
+ time_state = TIME_OK;
+ break;
+ }
+
+ write_sequnlock_irqrestore(&ntp_lock, flags);
+
+ return ret;
+}
+
/*
- * this routine handles the overflow of the microsecond field
+ * this routine handles the overflow of the nanosecond field
*
* The tricky bits of code to handle the accurate clock support
* were provided by Dave Mills ([email protected]) of NTP fame.
@@ -669,6 +751,13 @@
time_state = TIME_OK;
}
+ /* Bump the maxerror field */
+ time_maxerror += time_tolerance >> SHIFT_USEC;
+ if ( time_maxerror > NTP_PHASE_LIMIT ) {
+ time_maxerror = NTP_PHASE_LIMIT;
+ time_status |= STA_UNSYNC;
+ }
+
/*
* Compute the phase adjustment for the next second. In PLL mode, the
* offset is reduced by a fixed factor times the time constant. In FLL
@@ -684,6 +773,13 @@
time_offset -= ltemp;
time_adj = ltemp << (SHIFT_SCALE - SHIFT_HZ - SHIFT_UPDATE);
+ offset_adj_ppm = shift_right(ltemp, SHIFT_UPDATE); /* ppm */
+
+ /* first calculate usec/user_tick offset: */
+ tick_adj_ppm = ((USEC_PER_SEC + USER_HZ/2)/USER_HZ) - tick_usec;
+ /* multiply by user_hz to get usec/sec => ppm: */
+ tick_adj_ppm *= USER_HZ;
+
/*
* Compute the frequency estimate and additional phase adjustment due
* to frequency error for the next second. When the PPS signal is
@@ -724,15 +820,25 @@
}
/**
+ * ntp_get_ppm_adjustment - return shifted PPM adjustment
+ */
+long ntp_get_ppm_adjustment(void)
+{
+ return total_sppm;
+}
+
+/**
* ntp_advance - increments the NTP state machine
* @interval_ns: interval, in nanoseconds
- *
- * Must be holding the xtime writelock when calling.
*/
-static void ntp_advance(unsigned long interval_ns)
+void ntp_advance(unsigned long interval_ns)
{
static unsigned long interval_sum;
+ unsigned long flags;
+
+ write_seqlock_irqsave(&ntp_lock, flags);
+
/* increment the interval sum: */
interval_sum += interval_ns;
@@ -759,6 +865,8 @@
}
interval_ns -= tick_nsec;
}
+ /* usec/tick => ppm: */
+ singleshot_adj_ppm = time_adjust_step*(1000000/HZ);
/* Changes by adjtime() do not take effect till next tick. */
if (time_next_adjust != 0) {
@@ -770,6 +878,14 @@
interval_sum -= NSEC_PER_SEC;
second_overflow();
}
+
+ /* calculate the total continuous ppm adjustment: */
+ total_sppm = time_freq; /* already shifted by SHIFT_USEC */
+ total_sppm += offset_adj_ppm << SHIFT_USEC;
+ total_sppm += tick_adj_ppm << SHIFT_USEC;
+ total_sppm += singleshot_adj_ppm << SHIFT_USEC;
+
+ write_sequnlock_irqrestore(&ntp_lock, flags);
}
/**
@@ -779,6 +895,8 @@
*/
static inline long phase_advance(void)
{
+ static long time_phase; /* phase offset (scaled us) */
+
long delta = 0;
time_phase += time_adj;
@@ -797,12 +915,28 @@
*/
static inline void xtime_advance(long delta_nsec)
{
+ int leapsecond;
+
xtime.tv_nsec += delta_nsec;
if (likely(xtime.tv_nsec < NSEC_PER_SEC))
return;
xtime.tv_nsec -= NSEC_PER_SEC;
xtime.tv_sec++;
+
+ /* process leapsecond: */
+ leapsecond = ntp_leapsecond(xtime);
+ if (likely(!leapsecond))
+ return;
+
+ xtime.tv_sec += leapsecond;
+ wall_to_monotonic.tv_sec -= leapsecond;
+ /*
+ * Use of time interpolator for a gradual
+ * change of time:
+ */
+ time_interpolator_update(leapsecond*NSEC_PER_SEC);
+ clock_was_set();
}
/*
All,
This patch implements my generic timeofday framework. This common
infrastructure is intended to be used by any arch to reduce code
duplication, improve robustness in the face of late or lost ticks, and
enable finer granularity timekeeping.
The major change with this code is that it allows timekeeping to be
independent of the timer interrupt. This provides a linear mapping
(ignoring ntp adjustments) between a hardware clocksource counter and
the time of day. This allows for lost ticks or other software delays to
not affect timekeeping.
Included below is timeofday.c (which includes all the time of day
management and accessor functions), and minimal hooks into arch
independent code. This patch does not remove the current timekeeping
code, allowing architectures to move over when they are ready.
This patch applies on top of my clocksource managment patch.
The patch does nothing without at least minimal architecture specific
hooks (i386, x86-64 and other architecture examples to follow), and it
should be able to be applied to a tree without affecting the existing
code.
thanks
-john
Signed-off-by: John Stultz <[email protected]>
Documentation/timekeeping.txt | 246 ++++++++++++++
drivers/char/hangcheck-timer.c | 5
include/asm-generic/timeofday.h | 30 +
include/linux/time.h | 25 +
include/linux/timeofday.h | 44 ++
include/linux/timex.h | 2
include/sound/timer.h | 1
init/main.c | 2
kernel/ktimers.c | 1
kernel/posix-timers.c | 2
kernel/time.c | 17
kernel/time/Makefile | 2
kernel/time/timeofday.c | 681 ++++++++++++++++++++++++++++++++++++++++
kernel/timer.c | 7
14 files changed, 1059 insertions(+), 6 deletions(-)
linux-2.6.15-rc3-mm1_timeofday-core_B12.patch
==========================
diff -ruN tod-mm1/Documentation/timekeeping.txt tod-mm2/Documentation/timekeeping.txt
--- tod-mm1/Documentation/timekeeping.txt 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/Documentation/timekeeping.txt 2005-12-01 18:22:50.000000000 -0800
@@ -0,0 +1,246 @@
+How timekeeping works with CONFIG_GENERIC_TIME
+========================================================================
+
+The generic timekeeping code maintains and allows access to the systems understanding of how much time has passed from a certain point. However, in order to measure the passing of time, the generic timekeeping code relies on the clocksource abstraction. A clocksource abstracts a free running counter who's value increases at a known frequency.
+
+In the generic timekeeping code, we use a pointer to a selected clocksource to measure the passing of time.
+
+struct clocksource *clock
+
+The clocksource has some limitations however. Since its likely of fixed width, it will not increment forever and will overflow. In order to still properly keep time, we must occasionally accumulate an interval of time. In the generic timekeeping code, we accumulate the amount of time system the system booted into the value system_time, which keeps nanosecond resolution in a ktime_t storage.
+
+ktime_t system_time
+
+Since its likely your system has not been running continually since midnight on the 1st of January in 1970, we must provide an offset from that time in accordance with conventions. This only occasionally changed (via settimeofday()) offset is the wall_time_offset value, which is also stored as a ktime_t.
+
+ktime_t wall_time_offset
+
+
+Since we accumulate time in intervals, we need a base cycle value that we can use to generate an offset from the time value kept in system_time. We store this value in cycle_last.
+
+cycle_t cycle_last;
+
+
+Further since all clocks drift somewhat from each other, we use the adjustment values provided via adjtimex() to correct our clocksource frequency for each interval. This frequency adjustment value is stored in ntp_adj.
+
+long ntp_adj;
+
+Now that we've covered the core global variables for timekeeping, lets look at how we maintain these values.
+
+As stated above, we want to avoid the clocksource from overflowing on us, so we accumulate a time interval periodically. This periodic accumulation function is called timeofday_periodic_hook(). In simplified pseudo code, it logically is presented as:
+
+timeofday_periodic_hook():
+ cycle_now = read_clocksource(clock)
+ cycle_delta = (cycle_now - cycle_last) & clock->mask
+ nsec = cyc2ns(clock, cycle_delta, ntp_adj)
+ system_time += nsec
+ cycle_last = cycle_now
+
+ /* do other stuff */
+
+You can see we read the cycle value from the clocksource, calculate a cycle delta for the interval since we last called timeofday_periodic_hook(), convert that cycle delta to a nanosecond interval (for now ignore ntp_adj), add it to the system time and finally set our cycle_last value to cycle_now for the next interval. Using this simple algorithm we can correctly measure and record the passing of time.
+
+But just storing this info isn't very useful, we also want to make it available to be used elsewhere. So how do we provide a notion of how much time has passed inbetween calls to timeofday_periodic_hook()?
+
+First, lets create a function that calculates the time since the last call to timeofday_peridoic_hook().
+
+get_nsec_offset():
+ cycle_now = read_clocksource(clock)
+ cycle_delta = (cycle_now - cycle_last) & clock->mask
+ nsec = cyc2ns(clock, cycle_delta, ntp_adj)
+ return nsec
+
+Here you can see, we read the clocksource, calculate a cycle interval, and convert that to a nanosecond interval. Just like how it is done in timeofday_periodic_hook!
+
+Now lets use this function to provide the number of nanoseconds that the system has been running:
+
+do_monotonic_clock():
+ return system_time + get_nsec_offset()
+
+Here we trivially add the nanosecond offset since the last timeofday_periodic_hook() to the value of system_time which was stored at the last timeofday_periodic_hook().
+
+Note that since we use the same method to calculate time intervals, assuming each function is atomic and the clocksource functions as it should, time cannot go backward!
+
+Now to get the time of day using the standard convention:
+
+do_gettimeofday():
+ return do_monotonic_clock() + wall_time_offset
+
+We simply add the wall_time_offset, and we have the number of nanoseconds since 1970 began!
+
+
+Of course, in real life, things are not so static. We have to handle a number of dynamic values that may change and affect timekeeping. In order to do these safely, we must only change values in-between intervals. This means the periodic_hook call must handle these changes.
+
+Since clocksources can be changed while the system is running, we need to check for and possibly switch to using new clocksources in the periodic_hook call. Further, clocksources may change their frequency. Since this must be done only at a safe point, we use the update_callback function pointer (for more details, see "How to write a clocksource driver" below), this too must be done in-between intervals in the periodic_hook call. Finally, since the ntp adjustment made in the cyc2ns conversion is not static, we need to update the ntp state machine and get a calculate a new adjustment value.
+
+This adds some extra pseudo code to the timeofday_periodic_hook function:
+
+timeofday_periodic_hook():
+ cycle_now = read_clocksource(clock)
+ cycle_delta = (cycle_now - cycle_last) & clock->mask
+ nsec = cyc2ns(clock, cycle_delta, ntp_adj)
+ system_time += nsec
+ cycle_last = cycle_now
+
+ next = get_next_clocksource()
+ if(next != clock):
+ cycle_last = read_clocksource(next)
+ clock = next
+
+ if(clock->update_callback):
+ clock->update_callback()
+
+ ntp_advance(nsec)
+ ppm = ntp_get_ppm_adjustment()
+ ntp_adj = ppm_to_mult_adj(clock, ppm)
+
+
+Unfortunately, the actual timeofday_periodic_hook code is not as simple as this pseudo code. For performance concerns, much has been done to pre-calculate values and use them repeatedly. Thus be aware that the code in timeofday.c is more complex, however the functional logic is the same.
+
+
+How to port an architecture to GENERIC_TIME
+========================================================================
+Porting an architecture to the GENERIC_TIME timekeeping code consists of moving a little bit of code around then deleting a fair amount. It is my hope that this will reduce the arch specific maintenance work around timekeeping.
+
+Porting an arch usually requires the following steps.
+
+1. Define CONFIG_GENERIC_TIME in the arches Kconfig
+2. Implmenting the following functions
+ nsec_t read_persistent_clock(void)
+ void sync_persistent_clock(struct timespec ts)
+3. Removing all of the arch specific timekeeping code
+ do_gettimeofday()
+ do_settimeofday()
+ etc
+4. Implementing clocksource drivers
+ See "How to write a clocksource driver" for more details
+
+The exeptions to the above are:
+
+5. If the arch is has no continuous clocksource
+ A) Implement 1-3 in the above list.
+ B) Define CONFIG_IS_TICK_BASED in arches Kconfig
+ C) Implement the "long arch_getoffset(void)" function
+
+6. If the arch supports vsyscall gettimeofday (see x86_64 for reference)
+ A) Implement 1-4 in the above list
+ B) Define GENERIC_TIME_VSYSCALL
+ C) Implement arch_update_vsyscall_gtod()
+ D) Implement vsyscall gettimeofday (similar to __get_realtime_clock_ts)
+ E) Implement vread functions for supported clocksources
+
+
+
+How to write a clocksource driver.
+========================================================================
+First, a quick summary of what a clocksource driver provides.
+
+Simply put, a clocksource is a abstraction of a free running increasing counter. The abstraction provides the minimal amount of info for that counter to be usable for timekeeping. Those required values are:
+ 1. It's name
+ 2. A rating value for selection priority
+ 3. A read function pointer
+ 4. A mask value for correct twos-complement subtraction
+ 5. A mult and shift pair that aproximate the counter frequency
+ mult/(2^shift) ~= nanoseconds per cycle
+
+Additionally, there are other optionally set values that allow for advanced functinoality. Those values are:
+ 6. The update_callback function.
+ 7. The is_continuous flag.
+ 8. The vread function pointer
+ 9. The vdata pointer value
+
+
+Now lets go over these values in detail.
+
+1. Name.
+ The clocksource's name should be unique since it is used for both identification as well as for manually overriding the default clocksource selection. The name length must be shorter then 32 characters in order for it to be properly overrided.
+
+2. Rating value
+ This rating value is used as a priority value for clocksource selection. It has no direct connection to quality or physical properties of the clocksource, but is to be set and manipulated to guarantee that the best (by no specific metric) clocksource that will provide correct timekeeping is automatically selected. Rating suggestions can be found in include/linux/clocksource.h
+
+3. Read function pointer
+ This pointer should point to a function that returns an unsigned increasing cycle value from the clocksource. The value should have a coverage from zero to the maximum cycle value the clocksource can provide. This does not have to be direct hardware value and can also be a software counter. An example of a software counter is the jiffies clocksource.
+
+4. The mask value
+ This value should be the largest power of two that is smaller then the maximum cycle value. This allows twos complement subtraction to work on overflow boundary conditions if the max value is less then (cycle_t)-1. So for example, if we have a 16 bit counter (ie: one that loops to zero after 0x0000FFFF), the mask would be 0xFFFF. So then when finding the cycle difference around a overflow, where now = 0x0013 and then = 0xFFEE, we can compute the cycle delta properly using the equation:
+ delta = (now - then)&mask
+ delta = (0x0013 - 0xFFEE) & 0xFFFF
+ delta = 0xFFFF0025 & 0xFFFF /* note the unmasked negative value */
+ delta = 0x25
+
+5. The mult and shift pair
+ These 32bit values approximate the nanosecond per cycle frequency of the clocksource using the equation: mult/(2^shift). If you have a khz or hz frequency value, the mult value for a given shift value can be easily calculated using the clocksource_hz2mult() and clocksource_khz2mult() helper functions. When selecting a shift value, it is important to be careful. Larger shift values give a finer precision in the cycle to nanosecond conversion and allows for more exact NTP adjustments. However if you select too large a shift value, the resulting mult value might overflow a cycle_t * mult computation.
+
+
+So if you have a simple hardware counter that does not change frequency, filling in the above should be sufficient for a functional clocksource. But read on for details on implementing a more complex clocksource.
+
+6. The update_callback function pointer.
+ If this function pointer is non-NULL, it will be called every periodic hook when it is safe for the clocksource to change its state. This would be necessary in the case where the counter frequency changes, for example. One user of this function pointer is the TSC clocksource. When the TSC frequency changes (which may occur if the cpu changes frequency) we need to notify the clocksource at a safe point where that state may change. Thus, if the TSC has changed frequency we set the new mult/shift values in the update_callback function.
+
+7. The is_continuous flag.
+ This flag variable (0 if false, 1 if true) denotes that the clocksource is continuous. This means that it is a purely hardware driven clocksource and is not dependent on any software code to run for it to increment properly. This denotation will be useful in the future when timer ticks may be disabled for long periods of time. Doing so using software clocksources, like the jiffies clocksource, would cause timekeeping problems.
+
+8. The vread function pointer.
+ This function pointer points to a user-space accessible function that reads the clocksource. This is used in userspace gettimeofday implementations to improve performance. See the x86-64 TSC clocksource implementation for an example.
+
+8. The vdata pointer.
+ This pointer is passed to the vread function pointer in a userspace gettimeofday implementation. Its usage is dependent on the vread implementation, but if the pointer points to data, that data must be readable from userspace.
+
+
+Now lets write a quick clocksource for an imaginary bit of hardware. Here are the specs:
+
+ A 32bit counter can be found at the MMIO address 0xFEEDF000. It runs at 100Mhz. To enable it, the the low bit of the address 0xFEEDF0F0 must be set to one.
+
+So lets start out an empty cool-counter.c file, and define the clocksource.
+
+#include <linux/clocksource.h>
+#include <linux/init.h>
+#include <asm/io.h>
+
+#define COOL_READ_PTR 0xFEEDF000
+#define COOL_START_PTR 0xFEEDF0F0
+
+static __iomem *cool_ptr = COOL_READ_PTR;
+
+struct clocksource clocksource_cool
+{
+ .name = "cool",
+ .rating = 200, /* its a pretty decent clock */
+ .mask = 0xFFFFFFFF, /* 32 bits */
+ .mult = 0, /*to be computed */
+ .shift = 10,
+}
+
+
+Now let's write the read function:
+
+cycle_t cool_counter_read(void)
+{
+ cycle_t ret = readl(cool_ptr);
+ return ret;
+}
+
+Finally, lets write the init function:
+
+void cool_counter_init(void)
+{
+ __iomem *ptr = COOL_START_PTR;
+ u32 val;
+
+ /* start the counter */
+ val = readl(ptr);
+ val |= 0x1;
+ writel(val, ptr);
+
+ /* finish initializing the clocksource */
+ clocksource_cool.read = cool_counter_read;
+ clocksource_cool.mult = clocksource_khz2mult(100000,
+ clocksource_cool.shift);
+
+ /* register the clocksource */
+ register_clocksource(&clocksource_cool);
+}
+module_init(cool_counter_init);
+
+
+Now wasn't that easy!
diff -ruN tod-mm1/drivers/char/hangcheck-timer.c tod-mm2/drivers/char/hangcheck-timer.c
--- tod-mm1/drivers/char/hangcheck-timer.c 2005-12-01 18:13:06.000000000 -0800
+++ tod-mm2/drivers/char/hangcheck-timer.c 2005-12-01 18:22:50.000000000 -0800
@@ -49,6 +49,7 @@
#include <linux/delay.h>
#include <asm/uaccess.h>
#include <linux/sysrq.h>
+#include <linux/timeofday.h>
#define VERSION_STR "0.9.0"
@@ -130,8 +131,12 @@
#endif
#ifdef HAVE_MONOTONIC
+#ifndef CONFIG_GENERIC_TIME
extern unsigned long long monotonic_clock(void);
#else
+#define monotonic_clock() ktime_to_ns(get_monotonic_clock())
+#endif
+#else
static inline unsigned long long monotonic_clock(void)
{
# ifdef __s390__
diff -ruN tod-mm1/include/asm-generic/timeofday.h tod-mm2/include/asm-generic/timeofday.h
--- tod-mm1/include/asm-generic/timeofday.h 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/include/asm-generic/timeofday.h 2005-12-01 18:22:50.000000000 -0800
@@ -0,0 +1,30 @@
+/* linux/include/asm-generic/timeofday.h
+ *
+ * This file contains the asm-generic interface
+ * to the arch specific calls used by the time of day subsystem
+ */
+#ifndef _ASM_GENERIC_TIMEOFDAY_H
+#define _ASM_GENERIC_TIMEOFDAY_H
+#include <linux/types.h>
+#include <linux/time.h>
+#include <linux/timex.h>
+#include <linux/timeofday.h>
+#include <linux/clocksource.h>
+
+#include <asm/div64.h>
+
+#ifdef CONFIG_GENERIC_TIME
+/* Required externs */
+extern nsec_t read_persistent_clock(void);
+extern void sync_persistent_clock(struct timespec ts);
+
+#ifdef CONFIG_GENERIC_TIME_VSYSCALL
+extern void arch_update_vsyscall_gtod(struct timespec wall_time,
+ cycle_t offset_base, struct clocksource* clock,
+ int ntp_adj);
+#else
+# define arch_update_vsyscall_gtod(x,y,z,w) do { } while(0)
+#endif /* CONFIG_GENERIC_TIME_VSYSCALL */
+
+#endif /* CONFIG_GENERIC_TIME */
+#endif
diff -ruN tod-mm1/include/linux/time.h tod-mm2/include/linux/time.h
--- tod-mm1/include/linux/time.h 2005-12-01 18:13:36.000000000 -0800
+++ tod-mm2/include/linux/time.h 2005-12-01 18:22:50.000000000 -0800
@@ -28,6 +28,10 @@
#ifdef __KERNEL__
+/* timeofday base types */
+typedef s64 nsec_t;
+typedef u64 cycle_t;
+
/* Parameters used to convert the timespec values */
#define MSEC_PER_SEC (1000L)
#define USEC_PER_SEC (1000000L)
@@ -42,8 +46,6 @@
#define timespec_valid(ts) \
(((ts)->tv_sec >= 0) && (((unsigned) (ts)->tv_nsec) < NSEC_PER_SEC))
-typedef s64 nsec_t;
-
extern unsigned long
mktime (unsigned int year, unsigned int mon,
unsigned int day, unsigned int hour,
@@ -73,7 +75,6 @@
struct itimerval;
extern int do_setitimer(int which, struct itimerval *value, struct itimerval *ovalue);
extern int do_getitimer(int which, struct itimerval *value);
-extern void getnstimeofday (struct timespec *tv);
extern struct timespec timespec_trunc(struct timespec t, unsigned gran);
@@ -108,6 +109,24 @@
extern void ns_to_timespec(struct timespec *ts, nsec_t nsec);
extern void ns_to_timeval(struct timeval *tv, nsec_t nsec);
+static inline void normalize_timespec(struct timespec *ts)
+{
+ while (unlikely((unsigned long)ts->tv_nsec >= NSEC_PER_SEC)) {
+ ts->tv_nsec -= NSEC_PER_SEC;
+ ts->tv_sec++;
+ }
+}
+
+static inline void timespec_add_ns(struct timespec *a, nsec_t ns)
+{
+ while(unlikely(ns >= NSEC_PER_SEC)) {
+ ns -= NSEC_PER_SEC;
+ a->tv_sec++;
+ }
+ a->tv_nsec += ns;
+ normalize_timespec(a);
+}
+
#endif /* __KERNEL__ */
#define NFDBITS __NFDBITS
diff -ruN tod-mm1/include/linux/timeofday.h tod-mm2/include/linux/timeofday.h
--- tod-mm1/include/linux/timeofday.h 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/include/linux/timeofday.h 2005-12-01 18:22:50.000000000 -0800
@@ -0,0 +1,44 @@
+/* linux/include/linux/timeofday.h
+ *
+ * This file contains the interface to the time of day subsystem
+ */
+#ifndef _LINUX_TIMEOFDAY_H
+#define _LINUX_TIMEOFDAY_H
+#include <linux/calc64.h>
+#include <linux/types.h>
+#include <linux/ktime.h>
+#include <linux/time.h>
+#include <linux/timex.h>
+
+#ifdef CONFIG_GENERIC_TIME
+
+/* Kernel internal interfaces */
+extern ktime_t get_monotonic_clock(void);
+extern ktime_t get_realtime_clock(void);
+extern ktime_t get_realtime_offset(void);
+
+/* Timepsec based interfaces for user space functionality */
+extern void get_realtime_clock_ts(struct timespec *ts);
+extern void get_monotonic_clock_ts(struct timespec *ts);
+
+/* legacy timeofday interfaces*/
+#define getnstimeofday(ts) get_realtime_clock_ts(ts)
+extern void getnstimeofday(struct timespec *ts);
+extern void do_gettimeofday(struct timeval *tv);
+extern int do_settimeofday(struct timespec *ts);
+
+/* Internal functions */
+extern int timeofday_is_continuous(void);
+extern void timeofday_init(void);
+
+#ifndef CONFIG_IS_TICK_BASED
+#define arch_getoffset() (0)
+#else
+extern unsigned long arch_getoffset(void);
+#endif
+
+#else /* CONFIG_GENERIC_TIME */
+#define timeofday_init()
+extern void getnstimeofday(struct timespec *ts);
+#endif /* CONFIG_GENERIC_TIME */
+#endif /* _LINUX_TIMEOFDAY_H */
diff -ruN tod-mm1/include/linux/timex.h tod-mm2/include/linux/timex.h
--- tod-mm1/include/linux/timex.h 2005-12-01 18:21:24.000000000 -0800
+++ tod-mm2/include/linux/timex.h 2005-12-01 18:22:50.000000000 -0800
@@ -310,6 +310,7 @@
__x < 0 ? -(-__x >> __s) : __x >> __s; \
})
+#ifndef CONFIG_GENERIC_TIME
#ifdef CONFIG_TIME_INTERPOLATION
@@ -365,6 +366,7 @@
}
#endif /* !CONFIG_TIME_INTERPOLATION */
+#endif /* !CONFIG_GENERIC_TIME */
#endif /* KERNEL */
diff -ruN tod-mm1/include/sound/timer.h tod-mm2/include/sound/timer.h
--- tod-mm1/include/sound/timer.h 2005-12-01 18:13:37.000000000 -0800
+++ tod-mm2/include/sound/timer.h 2005-12-01 18:22:50.000000000 -0800
@@ -25,6 +25,7 @@
#include <sound/asound.h>
#include <linux/interrupt.h>
+#include <linux/timeofday.h>
#define snd_timer_chip(timer) ((timer)->private_data)
diff -ruN tod-mm1/init/main.c tod-mm2/init/main.c
--- tod-mm1/init/main.c 2005-12-01 18:13:39.000000000 -0800
+++ tod-mm2/init/main.c 2005-12-01 18:22:50.000000000 -0800
@@ -47,6 +47,7 @@
#include <linux/rmap.h>
#include <linux/mempolicy.h>
#include <linux/key.h>
+#include <linux/timeofday.h>
#include <net/sock.h>
#include <asm/io.h>
@@ -492,6 +493,7 @@
init_timers();
ktimers_init();
softirq_init();
+ timeofday_init();
time_init();
/*
diff -ruN tod-mm1/kernel/ktimers.c tod-mm2/kernel/ktimers.c
--- tod-mm1/kernel/ktimers.c 2005-12-01 18:13:40.000000000 -0800
+++ tod-mm2/kernel/ktimers.c 2005-12-01 18:22:50.000000000 -0800
@@ -30,6 +30,7 @@
#include <linux/percpu.h>
#include <linux/notifier.h>
#include <linux/syscalls.h>
+#include <linux/timeofday.h>
#include <linux/interrupt.h>
#include <asm/uaccess.h>
diff -ruN tod-mm1/kernel/posix-timers.c tod-mm2/kernel/posix-timers.c
--- tod-mm1/kernel/posix-timers.c 2005-12-01 18:13:40.000000000 -0800
+++ tod-mm2/kernel/posix-timers.c 2005-12-01 18:22:50.000000000 -0800
@@ -34,7 +34,7 @@
#include <linux/smp_lock.h>
#include <linux/interrupt.h>
#include <linux/slab.h>
-#include <linux/time.h>
+#include <linux/timeofday.h>
#include <asm/uaccess.h>
#include <asm/semaphore.h>
diff -ruN tod-mm1/kernel/time/Makefile tod-mm2/kernel/time/Makefile
--- tod-mm1/kernel/time/Makefile 2005-12-01 18:22:41.000000000 -0800
+++ tod-mm2/kernel/time/Makefile 2005-12-01 18:22:50.000000000 -0800
@@ -1 +1 @@
-obj-y += clocksource.o jiffies.o
+obj-y += clocksource.o jiffies.o timeofday.o
diff -ruN tod-mm1/kernel/time/timeofday.c tod-mm2/kernel/time/timeofday.c
--- tod-mm1/kernel/time/timeofday.c 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/kernel/time/timeofday.c 2005-12-01 18:22:50.000000000 -0800
@@ -0,0 +1,681 @@
+/*
+ * linux/kernel/time/timeofday.c
+ *
+ * This file contains the functions which access and manage
+ * the system's time of day functionality.
+ *
+ * Copyright (C) 2003, 2004, 2005 IBM, John Stultz ([email protected])
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
+ *
+ * TODO WishList:
+ * o See XXX's below.
+ */
+
+#include <linux/clocksource.h>
+#include <linux/timeofday.h>
+#include <linux/jiffies.h>
+#include <linux/sysdev.h>
+#include <linux/ktimer.h>
+#include <linux/module.h>
+#include <linux/ktime.h>
+#include <linux/timex.h>
+#include <linux/sched.h>
+
+#include <asm/timeofday.h>
+
+/* Periodic hook interval */
+#define PERIODIC_INTERVAL_MS 50
+
+/* [ktime_t based variables]
+ * system_time:
+ * Monotonically increasing counter of the number of nanoseconds
+ * since boot.
+ * wall_time_offset:
+ * Offset added to system_time to provide accurate time-of-day
+ */
+static ktime_t system_time;
+static ktime_t wall_time_offset;
+
+/* [timespec based variables]
+ * These variables mirror teh ktime_t based variables to avoid
+ * performance issues in the userspace syscall paths.
+ *
+ * wall_time_ts:
+ * timespec holding the current wall time.
+ * mono_time_ts:
+ * timespec holding the current monotonic time.
+ * monotonic_time_offset_ts:
+ * timespec holding the difference between wall and monotonic time.
+ */
+static struct timespec wall_time_ts;
+static struct timespec mono_time_ts;
+static struct timespec monotonic_time_offset_ts;
+
+/* [cycle based variables]
+ * cycle_last:
+ * Value of the clocksource at the last timeofday_periodic_hook()
+ * (adjusted only minorly to account for rounded off cycles)
+ */
+static cycle_t cycle_last;
+
+/* [clocksource_interval variables]
+ * ts_interval:
+ * This clocksource_interval is used in the fixed interval
+ * cycles to nanosecond calculation.
+ * INTERVAL_LEN:
+ * This constant is the requested fixed interval period
+ * in nanoseconds.
+ */
+struct clocksource_interval ts_interval;
+#define INTERVAL_LEN ((PERIODIC_INTERVAL_MS-1)*1000000)
+
+/* [clocksource data]
+ * clock:
+ * current clocksource pointer
+ */
+static struct clocksource *clock;
+
+/* [NTP adjustment]
+ * ntp_adj:
+ * value of the current ntp adjustment, stored in
+ * clocksource multiplier units.
+ */
+static int ntp_adj;
+
+/* [locks]
+ * system_time_lock:
+ * generic lock for all locally scoped time values
+ */
+static seqlock_t system_time_lock = SEQLOCK_UNLOCKED;
+
+
+/* [suspend/resume info]
+ * time_suspend_state:
+ * variable that keeps track of suspend state
+ * suspend_start:
+ * start of the suspend call
+ */
+static enum {
+ TIME_RUNNING,
+ TIME_SUSPENDED
+} time_suspend_state = TIME_RUNNING;
+
+static nsec_t suspend_start;
+
+/* [Soft-Timers]
+ * timeofday_timer:
+ * soft-timer used to call timeofday_periodic_hook()
+ */
+static struct ktimer timeofday_timer;
+
+/**
+ * update_legacy_time_values - sync legacy time values
+ *
+ * This function is necessary for a smooth transition to the
+ * new timekeeping code. When all the xtime/wall_to_monotonic
+ * users are converted this function can be removed.
+ *
+ * system_time_lock must be held by the caller
+ */
+static void update_legacy_time_values(void)
+{
+ unsigned long flags;
+
+ write_seqlock_irqsave(&xtime_lock, flags);
+
+ xtime = wall_time_ts;
+ set_normalized_timespec(&wall_to_monotonic,
+ -monotonic_time_offset_ts.tv_sec,
+ -monotonic_time_offset_ts.tv_nsec);
+
+ write_sequnlock_irqrestore(&xtime_lock, flags);
+
+ /* since time state has changed, notify vsyscall code */
+ arch_update_vsyscall_gtod(wall_time_ts, cycle_last, clock, ntp_adj);
+}
+
+/**
+ * __get_nsec_offset - Returns nanoseconds since last call to periodic_hook
+ *
+ * private function, must hold system_time_lock lock when being
+ * called. Returns the number of nanoseconds since the
+ * last call to timeofday_periodic_hook() (adjusted by NTP scaling)
+ */
+static inline nsec_t __get_nsec_offset(void)
+{
+ cycle_t cycle_now, cycle_delta;
+ nsec_t ns_offset;
+
+ /* read clocksource: */
+ cycle_now = read_clocksource(clock);
+
+ /* calculate the delta since the last timeofday_periodic_hook: */
+ cycle_delta = (cycle_now - cycle_last) & clock->mask;
+
+ /* convert to nanoseconds: */
+ ns_offset = cyc2ns(clock, ntp_adj, cycle_delta);
+
+ /*
+ * special case for jiffies tick/offset based systems,
+ * add arch-specific offset:
+ */
+ ns_offset += arch_getoffset();
+
+ return ns_offset;
+}
+
+/**
+ * __get_monotonic_clock - Returns monotonically increasing nanoseconds
+ *
+ * private function, must hold system_time_lock lock when being
+ * called. Returns the monotonically increasing number of
+ * nanoseconds since the system booted (adjusted by NTP scaling)
+ */
+static inline ktime_t __get_monotonic_clock(void)
+{
+ nsec_t offset = __get_nsec_offset();
+ return ktime_add_ns(system_time, offset);
+}
+
+/**
+ * get_monotonic_clock - Returns monotonic time in ktime_t format
+ *
+ * Returns the monotonically increasing number of nanoseconds
+ * since the system booted via __monotonic_clock()
+ */
+ktime_t get_monotonic_clock(void)
+{
+ unsigned long seq;
+ ktime_t ret;
+
+ /* atomically read __get_monotonic_clock() */
+ do {
+ seq = read_seqbegin(&system_time_lock);
+
+ ret = __get_monotonic_clock();
+
+ } while (read_seqretry(&system_time_lock, seq));
+
+ return ret;
+}
+
+EXPORT_SYMBOL_GPL(get_monotonic_clock);
+
+/**
+ * get_realtime_clock - Returns the timeofday in ktime_t format
+ *
+ * Returns the wall time in ktime_t format. The resolution
+ * is nanoseconds
+ */
+ktime_t get_realtime_clock(void)
+{
+ unsigned long seq;
+ ktime_t ret;
+
+ /* atomically read __get_monotonic_clock() */
+ do {
+ seq = read_seqbegin(&system_time_lock);
+
+ ret = __get_monotonic_clock();
+ ret = ktime_add(ret, wall_time_offset);
+
+ } while (read_seqretry(&system_time_lock, seq));
+
+ return ret;
+}
+
+/**
+ * get_realtime_offset - Returns the offset of realtime clock
+ *
+ * Returns the number of nanoseconds in ktime_t storage format which
+ * represents the offset of the realtime clock to the the monotonic clock
+ */
+ktime_t get_realtime_offset(void)
+{
+ unsigned long seq;
+ ktime_t ret;
+
+ /* atomically read wall_time_offset */
+ do {
+ seq = read_seqbegin(&system_time_lock);
+
+ ret = wall_time_offset;
+
+ } while (read_seqretry(&system_time_lock, seq));
+
+ return ret;
+}
+
+/**
+ * get_monotonic_clock_ts - Returns monotonic time in timespec format
+ * @ts: pointer to the timespec to be set
+ *
+ * Returns a timespec of nanoseconds since the system booted and
+ * store the result in the timespec variable pointed to by @ts
+ */
+void get_monotonic_clock_ts(struct timespec *ts)
+{
+ unsigned long seq;
+ nsec_t offset;
+
+ do {
+ seq = read_seqbegin(&system_time_lock);
+
+ *ts = mono_time_ts;
+ offset = __get_nsec_offset();
+ } while (read_seqretry(&system_time_lock, seq));
+
+ timespec_add_ns(ts, offset);
+}
+
+/**
+ * __get_realtime_clock_ts - Returns the time of day in a timespec
+ * @ts: pointer to the timespec to be set
+ *
+ * Returns the time of day in a timespec. Used by
+ * do_gettimeofday() and get_realtime_clock_ts().
+ */
+static inline void __get_realtime_clock_ts(struct timespec *ts)
+{
+ unsigned long seq;
+ nsec_t nsecs;
+
+ do {
+ seq = read_seqbegin(&system_time_lock);
+
+ *ts = wall_time_ts;
+ nsecs = __get_nsec_offset();
+
+ } while (read_seqretry(&system_time_lock, seq));
+
+ timespec_add_ns(ts, nsecs);
+}
+
+/**
+ * get_realtime_clock_ts - Returns the time of day in a timespec
+ * @ts: pointer to the timespec to be set
+ *
+ * Returns the time of day in a timespec.
+ */
+void get_realtime_clock_ts(struct timespec *ts)
+{
+ __get_realtime_clock_ts(ts);
+}
+
+EXPORT_SYMBOL(get_realtime_clock_ts);
+
+/**
+ * do_gettimeofday - Returns the time of day in a timeval
+ * @tv: pointer to the timeval to be set
+ *
+ * NOTE: Users should be converted to using get_realtime_clock_ts()
+ */
+void do_gettimeofday(struct timeval *tv)
+{
+ struct timespec now;
+
+ __get_realtime_clock_ts(&now);
+ tv->tv_sec = now.tv_sec;
+ tv->tv_usec = now.tv_nsec/1000;
+}
+
+EXPORT_SYMBOL(do_gettimeofday);
+
+/**
+ * do_settimeofday - Sets the time of day
+ * @tv: pointer to the timespec variable containing the new time
+ *
+ * Sets the time of day to the new time and update NTP and notify ktimers
+ */
+int do_settimeofday(struct timespec *tv)
+{
+ unsigned long flags;
+ ktime_t newtime;
+
+ newtime = timespec_to_ktime(*tv);
+
+ write_seqlock_irqsave(&system_time_lock, flags);
+
+ /* calculate the new offset from the monotonic clock */
+ wall_time_offset = ktime_sub(newtime, __get_monotonic_clock());
+
+ /* update the internal timespec variables */
+ ktime_to_timespec(&wall_time_ts,
+ ktime_add(system_time, wall_time_offset));
+ ktime_to_timespec(&monotonic_time_offset_ts, wall_time_offset);
+
+ ntp_clear();
+ update_legacy_time_values();
+
+ write_sequnlock_irqrestore(&system_time_lock, flags);
+
+ /* signal ktimers about time change */
+ clock_was_set();
+
+ return 0;
+}
+
+EXPORT_SYMBOL(do_settimeofday);
+
+/**
+ * __increment_system_time - Increments system time
+ * @delta: nanosecond delta to add to the time variables
+ *
+ * Private helper that increments system_time and related
+ * timekeeping variables.
+ */
+static inline void __increment_system_time(nsec_t delta)
+{
+ system_time = ktime_add_ns(system_time, delta);
+ timespec_add_ns(&wall_time_ts, delta);
+ timespec_add_ns(&mono_time_ts, delta);
+}
+
+/**
+ * timeofday_suspend_hook - allows the timeofday subsystem to be shutdown
+ * @dev: unused
+ * @state: unused
+ *
+ * This function allows the timeofday subsystem to be shutdown for a period
+ * of time. Called when going into suspend/hibernate mode.
+ */
+static int timeofday_suspend_hook(struct sys_device *dev, pm_message_t state)
+{
+ unsigned long flags;
+
+ write_seqlock_irqsave(&system_time_lock, flags);
+
+ BUG_ON(time_suspend_state != TIME_RUNNING);
+
+ /*
+ * First off, save suspend start time
+ * then quickly accumulate the current nsec offset.
+ * These two calls hopefully occur quickly
+ * because the difference between reads will
+ * accumulate as time drift on resume.
+ */
+ suspend_start = read_persistent_clock();
+ __increment_system_time(__get_nsec_offset());
+
+ time_suspend_state = TIME_SUSPENDED;
+
+ write_sequnlock_irqrestore(&system_time_lock, flags);
+
+ return 0;
+}
+
+/**
+ * timeofday_resume_hook - Resumes the timeofday subsystem.
+ * @dev: unused
+ *
+ * This function resumes the timeofday subsystem from a previous call
+ * to timeofday_suspend_hook.
+ */
+static int timeofday_resume_hook(struct sys_device *dev)
+{
+ nsec_t suspend_end, suspend_time;
+ unsigned long flags;
+
+ write_seqlock_irqsave(&system_time_lock, flags);
+
+ BUG_ON(time_suspend_state != TIME_SUSPENDED);
+
+ /*
+ * Read persistent clock to mark the end of
+ * the suspend interval then rebase the
+ * cycle_last to current clocksource value.
+ * Again, time between these two calls will
+ * not be accounted for and will show up as
+ * time drift.
+ */
+ suspend_end = read_persistent_clock();
+ cycle_last = read_clocksource(clock);
+
+ /* calculate suspend time and add it to system time: */
+ suspend_time = suspend_end - suspend_start;
+ __increment_system_time(suspend_time);
+
+ ntp_clear();
+
+ time_suspend_state = TIME_RUNNING;
+
+ update_legacy_time_values();
+
+ write_sequnlock_irqrestore(&system_time_lock, flags);
+
+ /* inform ktimers about time change: */
+ clock_was_set();
+
+ return 0;
+}
+
+/* sysfs resume/suspend bits */
+static struct sysdev_class timeofday_sysclass = {
+ .resume = timeofday_resume_hook,
+ .suspend = timeofday_suspend_hook,
+ set_kset_name("timeofday"),
+};
+
+static struct sys_device device_timer = {
+ .id = 0,
+ .cls = &timeofday_sysclass,
+};
+
+static int timeofday_init_device(void)
+{
+ int error = sysdev_class_register(&timeofday_sysclass);
+
+ if (!error)
+ error = sysdev_register(&device_timer);
+
+ return error;
+}
+
+device_initcall(timeofday_init_device);
+
+/**
+ * timeofday_periodic_hook - Does periodic update of timekeeping values.
+ * @unused: unused value
+ *
+ * Calculates the delta since the last call, updates system time and
+ * clears the offset.
+ *
+ * Called via timeofday_timer.
+ */
+static void timeofday_periodic_hook(void* unused)
+{
+ unsigned long flags;
+
+ cycle_t cycle_now, cycle_delta;
+ nsec_t delta_nsec;
+ static u64 remainder;
+
+ long leapsecond;
+ struct clocksource* next;
+
+ int ppm;
+ static int ppm_last;
+
+ int something_changed = 0;
+ struct clocksource old_clock;
+ static nsec_t second_check;
+ ktime_t expire_time;
+
+ write_seqlock_irqsave(&system_time_lock, flags);
+
+ /* read time source & calc time since last call: */
+ cycle_now = read_clocksource(clock);
+ cycle_delta = (cycle_now - cycle_last) & clock->mask;
+
+ delta_nsec = cyc2ns_fixed_rem(ts_interval, &cycle_delta, &remainder);
+ cycle_last = (cycle_now - cycle_delta)&clock->mask;
+
+ /* update system_time: */
+ __increment_system_time(delta_nsec);
+
+ /* advance the ntp state machine by ns interval: */
+ ntp_advance(delta_nsec);
+
+ /* only call ntp_leapsecond and ntp_sync once a sec: */
+ second_check += delta_nsec;
+ if (second_check >= NSEC_PER_SEC) {
+ /* do ntp leap second processing: */
+ leapsecond = ntp_leapsecond(wall_time_ts);
+ if (leapsecond) {
+ wall_time_offset = ktime_add_ns(wall_time_offset,
+ leapsecond * NSEC_PER_SEC);
+ wall_time_ts.tv_sec += leapsecond;
+ monotonic_time_offset_ts.tv_sec += leapsecond;
+ }
+ /* sync the persistent clock: */
+ if (ntp_synced())
+ sync_persistent_clock(wall_time_ts);
+ second_check -= NSEC_PER_SEC;
+ }
+
+ /* if necessary, switch clocksources: */
+ next = get_next_clocksource();
+ if (next != clock) {
+ /* immediately set new cycle_last: */
+ cycle_last = read_clocksource(next);
+ /* update cycle_now to avoid problems in accumulation later: */
+ cycle_now = cycle_last;
+ /* swap clocksources: */
+ old_clock = *clock;
+ clock = next;
+ printk(KERN_INFO "Time: %s clocksource has been installed.\n",
+ clock->name);
+ ntp_clear();
+ ntp_adj = 0;
+ remainder = 0;
+ something_changed = 1;
+ }
+
+ /*
+ * now is a safe time, so allow clocksource to adjust
+ * itself (for example: to make cpufreq changes):
+ */
+ if (clock->update_callback) {
+ /*
+ * since clocksource state might change,
+ * keep a copy, but only if we've not
+ * already changed timesources:
+ */
+ if (!something_changed)
+ old_clock = *clock;
+ if (clock->update_callback()) {
+ remainder = 0;
+ something_changed = 1;
+ }
+ }
+
+ /* check for new PPM adjustment: */
+ ppm = ntp_get_ppm_adjustment();
+ if (ppm_last != ppm) {
+ /* make sure old_clock is set: */
+ if (!something_changed)
+ old_clock = *clock;
+ something_changed = 1;
+ }
+
+ /* if something changed, recalculate the ntp adjustment value: */
+ if (something_changed) {
+ /* accumulate current leftover cycles using old_clock: */
+ if (cycle_delta) {
+ delta_nsec = cyc2ns_rem(&old_clock, ntp_adj,
+ cycle_delta, &remainder);
+ cycle_last = cycle_now;
+ __increment_system_time(delta_nsec);
+ ntp_advance(delta_nsec);
+ }
+
+ /* recalculate the ntp adjustment and fixed interval values: */
+ ppm_last = ppm;
+ ntp_adj = ppm_to_mult_adj(clock, ppm);
+ ts_interval = calculate_clocksource_interval(clock, ntp_adj,
+ INTERVAL_LEN);
+ }
+
+ update_legacy_time_values();
+
+ write_sequnlock_irqrestore(&system_time_lock, flags);
+
+ /* set us up to go off on the next interval: */
+ expire_time = ktime_set(0, PERIODIC_INTERVAL_MS*1000000);
+ ktimer_start(&timeofday_timer, &expire_time, KTIMER_REL);
+}
+
+/**
+ * timeofday_is_continuous - check to see if timekeeping is free running
+ */
+int timeofday_is_continuous(void)
+{
+ unsigned long seq;
+ int ret;
+
+ do {
+ seq = read_seqbegin(&system_time_lock);
+
+ ret = clock->is_continuous;
+
+ } while (read_seqretry(&system_time_lock, seq));
+
+ return ret;
+}
+
+/**
+ * timeofday_init - Initializes time variables
+ */
+void __init timeofday_init(void)
+{
+ unsigned long flags;
+ ktime_t expire_time;
+
+ write_seqlock_irqsave(&system_time_lock, flags);
+
+ /* initialize the clock variable: */
+ clock = get_next_clocksource();
+
+ /* initialize cycle_last offset base: */
+ cycle_last = read_clocksource(clock);
+
+ /* initialize wall_time_offset to now: */
+ /* XXX - this should be something like ns_to_ktime() */
+ wall_time_offset = ktime_add_ns(wall_time_offset,
+ read_persistent_clock());
+
+ /* initialize timespec values: */
+ ktime_to_timespec(&wall_time_ts,
+ ktime_add(system_time, wall_time_offset));
+ ktime_to_timespec(&monotonic_time_offset_ts, wall_time_offset);
+
+ /* clear NTP scaling factor & state machine: */
+ ntp_adj = 0;
+ ntp_clear();
+ ts_interval = calculate_clocksource_interval(clock, ntp_adj,
+ INTERVAL_LEN);
+
+ /* initialize legacy time values: */
+ update_legacy_time_values();
+
+ write_sequnlock_irqrestore(&system_time_lock, flags);
+
+ /* install timeofday_periodic_hook timer: */
+ ktimer_init(&timeofday_timer);
+ expire_time = ktime_set(0, PERIODIC_INTERVAL_MS*1000000);
+ timeofday_timer.function = timeofday_periodic_hook;
+ ktimer_start(&timeofday_timer, &expire_time, KTIMER_REL);
+}
diff -ruN tod-mm1/kernel/time.c tod-mm2/kernel/time.c
--- tod-mm1/kernel/time.c 2005-12-01 18:21:24.000000000 -0800
+++ tod-mm2/kernel/time.c 2005-12-01 18:22:50.000000000 -0800
@@ -38,6 +38,7 @@
#include <asm/uaccess.h>
#include <asm/unistd.h>
+#include <linux/timeofday.h>
/*
* The timezone where the local system is located. Used as a default by some
@@ -128,6 +129,7 @@
* as real UNIX machines always do it. This avoids all headaches about
* daylight saving times and warping kernel clocks.
*/
+#ifndef CONFIG_GENERIC_TIME
static inline void warp_clock(void)
{
write_seqlock_irq(&xtime_lock);
@@ -137,6 +139,18 @@
write_sequnlock_irq(&xtime_lock);
clock_was_set();
}
+#else /* !CONFIG_GENERIC_TIME */
+/* XXX - this is somewhat cracked out and should
+ be checked [email protected]
+*/
+static inline void warp_clock(void)
+{
+ struct timespec ts;
+ getnstimeofday(&ts);
+ ts.tv_sec += sys_tz.tz_minuteswest * 60;
+ do_settimeofday(&ts);
+}
+#endif /* !CONFIG_GENERIC_TIME */
/*
* In case for some reason the CMOS clock has not already been running
@@ -476,6 +490,7 @@
}
EXPORT_SYMBOL(timespec_trunc);
+#ifndef CONFIG_GENERIC_TIME
#ifdef CONFIG_TIME_INTERPOLATION
void getnstimeofday (struct timespec *tv)
{
@@ -561,6 +576,8 @@
EXPORT_SYMBOL_GPL(getnstimeofday);
#endif
+#endif /* !CONFIG_GENERIC_TIME */
+
/* Converts Gregorian date to seconds since 1970-01-01 00:00:00.
* Assumes input in normal date format, i.e. 1980-12-31 23:59:59
* => year=1980, mon=12, day=31, hour=23, min=59, sec=59.
diff -ruN tod-mm1/kernel/timer.c tod-mm2/kernel/timer.c
--- tod-mm1/kernel/timer.c 2005-12-01 18:21:24.000000000 -0800
+++ tod-mm2/kernel/timer.c 2005-12-01 18:22:50.000000000 -0800
@@ -28,7 +28,7 @@
#include <linux/swap.h>
#include <linux/notifier.h>
#include <linux/thread_info.h>
-#include <linux/time.h>
+#include <linux/timeofday.h>
#include <linux/jiffies.h>
#include <linux/posix-timers.h>
#include <linux/cpu.h>
@@ -888,6 +888,10 @@
write_sequnlock_irqrestore(&ntp_lock, flags);
}
+#ifdef CONFIG_GENERIC_TIME
+# define update_wall_time(x) do { } while (0)
+#else
+
/**
* phase_advance - advance the phase
*
@@ -963,6 +967,7 @@
} while (--ticks);
}
+#endif /* !CONFIG_GENERIC_TIME */
/*
* Called from the timer interrupt handler to charge one tick to the current
All,
The conversion of i386 to use the generic timeofday subsystem has been
split into 6 parts. This patch, the third of six, reworks some of the
code in the new tsc.c file, adding some new interfaces and hooks to use
these new interfaces appropriately.
It applies on top of my timeofday-arch-i386-part2 patch. This patch is
part the timeofday-arch-i386 patchset, so without the following parts it
is not expected to compile.
thanks
-john
Signed-off-by: John Stultz <[email protected]>
arch/i386/kernel/setup.c | 1
arch/i386/kernel/tsc.c | 213 +++++++++++++++-------------
drivers/acpi/processor_idle.c | 6
include/asm-i386/mach-default/mach_timer.h | 4
include/asm-i386/mach-summit/mach_mpparse.h | 3
include/asm-i386/tsc.h | 12 +
6 files changed, 138 insertions(+), 101 deletions(-)
linux-2.6.15-rc3-mm1_timeofday-arch-i386-part3_B12.patch
==========================
diff -ruN tod-mm1/arch/i386/kernel/setup.c tod-mm2/arch/i386/kernel/setup.c
--- tod-mm1/arch/i386/kernel/setup.c 2005-12-01 18:12:56.000000000 -0800
+++ tod-mm2/arch/i386/kernel/setup.c 2005-12-01 18:25:11.000000000 -0800
@@ -1628,6 +1628,7 @@
conswitchp = &dummy_con;
#endif
#endif
+ tsc_init();
}
#include "setup_arch_post.h"
diff -ruN tod-mm1/arch/i386/kernel/tsc.c tod-mm2/arch/i386/kernel/tsc.c
--- tod-mm1/arch/i386/kernel/tsc.c 2005-12-01 18:25:04.000000000 -0800
+++ tod-mm2/arch/i386/kernel/tsc.c 2005-12-01 18:25:11.000000000 -0800
@@ -6,13 +6,23 @@
#include <linux/workqueue.h>
#include <linux/cpufreq.h>
+#include <linux/jiffies.h>
#include <linux/init.h>
+#include <asm/tsc.h>
#include <asm/io.h>
#include "mach_timer.h"
+/*
+ * On some systems the TSC frequency does not
+ * change with the cpu frequency. So we need
+ * an extra value to store the TSC freq
+ */
+unsigned int tsc_khz;
+
int tsc_disable __initdata = 0;
+
#ifdef CONFIG_X86_TSC
static int __init tsc_setup(char *str)
{
@@ -38,15 +48,50 @@
int read_current_timer(unsigned long *timer_val)
{
- if (cur_timer->read_timer) {
- *timer_val = cur_timer->read_timer();
+ if (!tsc_disable && cpu_khz) {
+ rdtscl(*timer_val);
return 0;
}
return -1;
}
+/*
+ * code to mark and check if the TSC is unstable
+ * due to cpufreq or due to unsynced TSCs
+ */
+static int tsc_unstable;
+
+static inline int check_tsc_unstable(void)
+{
+ return tsc_unstable;
+}
+
+void mark_tsc_unstable(void)
+{
+ tsc_unstable = 1;
+}
+
+/* Code to compensate for C3 stalls */
+static u64 tsc_c3_offset;
+
+void tsc_c3_compensate(unsigned long nsecs)
+{
+ /* this could def be optimized */
+ u64 cycles = ((u64)nsecs * tsc_khz);
+
+ do_div(cycles, 1000000);
+ tsc_c3_offset += cycles;
+}
+
+EXPORT_SYMBOL_GPL(tsc_c3_compensate);
+
+static inline u64 tsc_read_c3_time(void)
+{
+ return tsc_c3_offset;
+}
-/* convert from cycles(64bits) => nanoseconds (64bits)
+/* Accellerators for sched_clock()
+ * convert from cycles(64bits) => nanoseconds (64bits)
* basic equation:
* ns = cycles / (freq / ns_per_sec)
* ns = cycles * (ns_per_sec / freq)
@@ -92,76 +137,56 @@
* synchronized across all CPUs.
*/
#ifndef CONFIG_NUMA
- if (!use_tsc)
+ if (!cpu_khz || check_tsc_unstable())
#endif
/* no locking but a rare wrong value is not a big deal */
- return jiffies_64 * (1000000000 / HZ);
+ return (jiffies_64 - INITIAL_JIFFIES) * (1000000000 / HZ);
/* read the Time Stamp Counter: */
rdtscll(this_offset);
+ this_offset += tsc_read_c3_time();
/* return the value in ns */
return cycles_2_ns(this_offset);
}
-/* ------ Calibrate the TSC -------
- * Return 2^32 * (1 / (TSC clocks per usec)) for do_fast_gettimeoffset().
- * Too much 64-bit arithmetic here to do this cleanly in C, and for
- * accuracy's sake we want to keep the overhead on the CTC speaker (channel 2)
- * output busy loop as low as possible. We avoid reading the CTC registers
- * directly because of the awkward 8-bit access mechanism of the 82C54
- * device.
- */
-
-#define CALIBRATE_TIME (5 * 1000020/HZ)
-
-unsigned long calibrate_tsc(void)
+static unsigned long calculate_cpu_khz(void)
{
- mach_prepare_counter();
-
- {
- unsigned long startlow, starthigh;
- unsigned long endlow, endhigh;
- unsigned long count;
-
- rdtsc(startlow,starthigh);
+ unsigned long long start, end;
+ unsigned long count;
+ u64 delta64;
+ int i;
+
+ /* run 3 times to ensure the cache is warm */
+ for (i = 0; i < 3; i++) {
+ mach_prepare_counter();
+ rdtscll(start);
mach_countup(&count);
- rdtsc(endlow,endhigh);
-
-
- /* Error: ECTCNEVERSET */
- if (count <= 1)
- goto bad_ctc;
-
- /* 64-bit subtract - gcc just messes up with long longs */
- __asm__("subl %2,%0\n\t"
- "sbbl %3,%1"
- :"=a" (endlow), "=d" (endhigh)
- :"g" (startlow), "g" (starthigh),
- "0" (endlow), "1" (endhigh));
-
- /* Error: ECPUTOOFAST */
- if (endhigh)
- goto bad_ctc;
-
- /* Error: ECPUTOOSLOW */
- if (endlow <= CALIBRATE_TIME)
- goto bad_ctc;
-
- __asm__("divl %2"
- :"=a" (endlow), "=d" (endhigh)
- :"r" (endlow), "0" (0), "1" (CALIBRATE_TIME));
-
- return endlow;
+ rdtscll(end);
}
-
/*
+ * Error: ECTCNEVERSET
* The CTC wasn't reliable: we got a hit on the very first read,
* or the CPU was so fast/slow that the quotient wouldn't fit in
* 32 bits..
*/
-bad_ctc:
- return 0;
+ if (count <= 1)
+ return 0;
+
+ delta64 = end - start;
+
+ /* cpu freq too fast: */
+ if (delta64 > (1ULL<<32))
+ return 0;
+
+ /* cpu freq too slow: */
+ if (delta64 <= CALIBRATE_TIME_MSEC)
+ return 0;
+
+ delta64 += CALIBRATE_TIME_MSEC/2; /* round for do_div */
+ do_div(delta64,CALIBRATE_TIME_MSEC);
+
+ return (unsigned long)delta64;
}
int recalibrate_cpu_khz(void)
@@ -170,11 +195,11 @@
unsigned long cpu_khz_old = cpu_khz;
if (cpu_has_tsc) {
- init_cpu_khz();
+ cpu_khz = calculate_cpu_khz();
+ tsc_khz = cpu_khz;
cpu_data[0].loops_per_jiffy =
- cpufreq_scale(cpu_data[0].loops_per_jiffy,
- cpu_khz_old,
- cpu_khz);
+ cpufreq_scale(cpu_data[0].loops_per_jiffy,
+ cpu_khz_old, cpu_khz);
return 0;
} else
return -ENODEV;
@@ -182,28 +207,25 @@
return -ENODEV;
#endif
}
-EXPORT_SYMBOL(recalibrate_cpu_khz);
+EXPORT_SYMBOL(recalibrate_cpu_khz);
-/* calculate cpu_khz */
-void init_cpu_khz(void)
+void tsc_init(void)
{
- if (cpu_has_tsc) {
- unsigned long tsc_quotient = calibrate_tsc();
- if (tsc_quotient) {
- /* report CPU clock rate in Hz.
- * The formula is (10^6 * 2^32) / (2^32 * 1 / (clocks/us)) =
- * clock/second. Our precision is about 100 ppm.
- */
- { unsigned long eax=0, edx=1000;
- __asm__("divl %2"
- :"=a" (cpu_khz), "=d" (edx)
- :"r" (tsc_quotient),
- "0" (eax), "1" (edx));
- printk("Detected %lu.%03lu MHz processor.\n", cpu_khz / 1000, cpu_khz % 1000);
- }
- }
- }
+ if (!cpu_has_tsc || tsc_disable)
+ return;
+
+ cpu_khz = calculate_cpu_khz();
+ tsc_khz = cpu_khz;
+
+ if (!cpu_khz)
+ return;
+
+ printk("Detected %lu.%03lu MHz processor.\n",
+ (unsigned long)cpu_khz / 1000,
+ (unsigned long)cpu_khz % 1000);
+
+ set_cyc2ns_scale(cpu_khz);
}
#ifdef CONFIG_CPU_FREQ
@@ -223,15 +245,15 @@
}
/*
- * if we notice lost ticks, schedule a call to cpufreq_get() as it tries
+ * if we notice cpufreq oddness, schedule a call to cpufreq_get() as it tries
* to verify the CPU frequency the timing core thinks the CPU is running
* at is still correct.
*/
-void cpufreq_delayed_get(void)
+static inline void cpufreq_delayed_get(void)
{
if (cpufreq_init && !cpufreq_delayed_issched) {
cpufreq_delayed_issched = 1;
- printk(KERN_DEBUG "Losing some ticks... checking if CPU frequency changed.\n");
+ printk(KERN_DEBUG "Checking if CPU frequency changed.\n");
schedule_work(&cpufreq_delayed_get_work);
}
}
@@ -240,14 +262,9 @@
* if the CPU frequency is scaled, TSC-based delays will need a different
* loops_per_jiffy value to function properly.
*/
-
static unsigned int ref_freq = 0;
static unsigned long loops_per_jiffy_ref = 0;
-
-#ifndef CONFIG_SMP
-static unsigned long fast_gettimeoffset_ref = 0;
static unsigned long cpu_khz_ref = 0;
-#endif
static int
time_cpufreq_notifier(struct notifier_block *nb, unsigned long val, void *data)
@@ -260,10 +277,7 @@
if (!ref_freq) {
ref_freq = freq->old;
loops_per_jiffy_ref = cpu_data[freq->cpu].loops_per_jiffy;
-#ifndef CONFIG_SMP
- fast_gettimeoffset_ref = fast_gettimeoffset_quotient;
cpu_khz_ref = cpu_khz;
-#endif
}
if ((val == CPUFREQ_PRECHANGE && freq->old < freq->new) ||
@@ -271,16 +285,22 @@
(val == CPUFREQ_RESUMECHANGE)) {
if (!(freq->flags & CPUFREQ_CONST_LOOPS))
cpu_data[freq->cpu].loops_per_jiffy = cpufreq_scale(loops_per_jiffy_ref, ref_freq, freq->new);
-#ifndef CONFIG_SMP
- if (cpu_khz)
- cpu_khz = cpufreq_scale(cpu_khz_ref, ref_freq, freq->new);
- if (use_tsc) {
+
+ if (cpu_khz) {
+
+ if (num_online_cpus() == 1)
+ cpu_khz = cpufreq_scale(cpu_khz_ref,
+ ref_freq, freq->new);
if (!(freq->flags & CPUFREQ_CONST_LOOPS)) {
- fast_gettimeoffset_quotient = cpufreq_scale(fast_gettimeoffset_ref, freq->new, ref_freq);
+ tsc_khz = cpu_khz;
set_cyc2ns_scale(cpu_khz);
+ /*
+ * TSC based sched_clock turns
+ * to junk w/ cpufreq
+ */
+ mark_tsc_unstable();
}
}
-#endif
}
if (val != CPUFREQ_RESUMECHANGE)
@@ -302,11 +322,10 @@
CPUFREQ_TRANSITION_NOTIFIER);
if (!ret)
cpufreq_init = 1;
+
return ret;
}
core_initcall(cpufreq_tsc);
-#else /* CONFIG_CPU_FREQ */
-void cpufreq_delayed_get(void) { return; }
#endif
diff -ruN tod-mm1/drivers/acpi/processor_idle.c tod-mm2/drivers/acpi/processor_idle.c
--- tod-mm1/drivers/acpi/processor_idle.c 2005-12-01 18:13:04.000000000 -0800
+++ tod-mm2/drivers/acpi/processor_idle.c 2005-12-01 18:25:11.000000000 -0800
@@ -268,6 +268,7 @@
}
static atomic_t c3_cpu_count;
+extern void tsc_c3_compensate(unsigned long nsecs);
static void acpi_processor_idle(void)
{
@@ -451,6 +452,11 @@
ACPI_MTX_DO_NOT_LOCK);
}
+#ifdef CONFIG_GENERIC_TIME
+ /* compensate for TSC pause */
+ tsc_c3_compensate((u32)(((u64)((t2-t1)&0xFFFFFF)*286070)>>10));
+#endif
+
/* Re-enable interrupts */
local_irq_enable();
/* Compute time (ticks) that we were actually asleep */
diff -ruN tod-mm1/include/asm-i386/mach-default/mach_timer.h tod-mm2/include/asm-i386/mach-default/mach_timer.h
--- tod-mm1/include/asm-i386/mach-default/mach_timer.h 2005-12-01 18:13:31.000000000 -0800
+++ tod-mm2/include/asm-i386/mach-default/mach_timer.h 2005-12-01 18:25:11.000000000 -0800
@@ -15,7 +15,9 @@
#ifndef _MACH_TIMER_H
#define _MACH_TIMER_H
-#define CALIBRATE_LATCH (5 * LATCH)
+#define CALIBRATE_TIME_MSEC 30 /* 30 msecs */
+#define CALIBRATE_LATCH \
+ ((CLOCK_TICK_RATE * CALIBRATE_TIME_MSEC + 1000/2)/1000)
static inline void mach_prepare_counter(void)
{
diff -ruN tod-mm1/include/asm-i386/mach-summit/mach_mpparse.h tod-mm2/include/asm-i386/mach-summit/mach_mpparse.h
--- tod-mm1/include/asm-i386/mach-summit/mach_mpparse.h 2005-12-01 18:13:31.000000000 -0800
+++ tod-mm2/include/asm-i386/mach-summit/mach_mpparse.h 2005-12-01 18:25:11.000000000 -0800
@@ -2,6 +2,7 @@
#define __ASM_MACH_MPPARSE_H
#include <mach_apic.h>
+#include <asm/tsc.h>
extern int use_cyclone;
@@ -29,6 +30,7 @@
(!strncmp(productid, "VIGIL SMP", 9)
|| !strncmp(productid, "EXA", 3)
|| !strncmp(productid, "RUTHLESS SMP", 12))){
+ mark_tsc_unstable();
use_cyclone = 1; /*enable cyclone-timer*/
setup_summit();
return 1;
@@ -42,6 +44,7 @@
if (!strncmp(oem_id, "IBM", 3) &&
(!strncmp(oem_table_id, "SERVIGIL", 8)
|| !strncmp(oem_table_id, "EXA", 3))){
+ mark_tsc_unstable();
use_cyclone = 1; /*enable cyclone-timer*/
setup_summit();
return 1;
diff -ruN tod-mm1/include/asm-i386/tsc.h tod-mm2/include/asm-i386/tsc.h
--- tod-mm1/include/asm-i386/tsc.h 2005-12-01 18:25:04.000000000 -0800
+++ tod-mm2/include/asm-i386/tsc.h 2005-12-01 18:25:11.000000000 -0800
@@ -25,9 +25,12 @@
*/
typedef unsigned long long cycles_t;
-static inline cycles_t get_cycles (void)
+extern unsigned int cpu_khz;
+extern unsigned int tsc_khz;
+
+static inline cycles_t get_cycles(void)
{
- unsigned long long ret=0;
+ unsigned long long ret = 0;
#ifndef CONFIG_X86_TSC
if (!cpu_has_tsc)
@@ -40,5 +43,8 @@
return ret;
}
-extern unsigned int cpu_khz;
+extern void tsc_init(void);
+extern void tsc_c3_compensate(unsigned long usecs);
+extern void mark_tsc_unstable(void);
+
#endif
All,
This patch converts the x86-64 arch to use the generic timeofday
infrastructure. It applies on top of my timeofday-core patch. This is a
full conversion, so most of this patch is subtractions removing the
existing arch specific time keeping code. This patch does not provide
any x86-64 clocksourcs, so using this patch alone on top of the
timeofday-core patch will only give you the jiffies clocksource. To get
full replacements for the code being removed here, the following
timeofday-clocks-i386 patch (x86-64 shares the same clocksources as
i386) will need to be applied.
Please note, this patch is not yet ready for submission, but is being
maintained as an example of how the generic framework supports multiple
arches with different features (such as vsyscall). That said, I'd still
appreciate any suggestions for improvements to this code.
thanks
-john
Signed-off-by: John Stultz <[email protected]>
arch/i386/kernel/acpi/boot.c | 9
arch/x86_64/Kconfig | 8
arch/x86_64/kernel/Makefile | 1
arch/x86_64/kernel/pmtimer.c | 101 -------
arch/x86_64/kernel/time.c | 540 +++++++++++++++------------------------
arch/x86_64/kernel/vmlinux.lds.S | 12
arch/x86_64/kernel/vsyscall.c | 137 +++++++--
include/asm-generic/div64.h | 9
include/asm-x86_64/hpet.h | 3
include/asm-x86_64/timeofday.h | 4
include/asm-x86_64/timex.h | 2
include/asm-x86_64/vsyscall.h | 8
12 files changed, 376 insertions(+), 458 deletions(-)
linux-2.6.15-rc3-mm1_timeofday-arch-x86-64_B12.patch
==========================
diff -ruN tod-mm1/arch/i386/kernel/acpi/boot.c tod-mm2/arch/i386/kernel/acpi/boot.c
--- tod-mm1/arch/i386/kernel/acpi/boot.c 2005-12-01 18:25:39.000000000 -0800
+++ tod-mm2/arch/i386/kernel/acpi/boot.c 2005-12-01 18:27:40.000000000 -0800
@@ -567,7 +567,7 @@
}
#ifdef CONFIG_HPET_TIMER
-
+#include <asm/hpet.h>
static int __init acpi_parse_hpet(unsigned long phys, unsigned long size)
{
struct acpi_table_hpet *hpet_tbl;
@@ -589,6 +589,7 @@
#ifdef CONFIG_X86_64
vxtime.hpet_address = hpet_tbl->addr.addrl |
((long)hpet_tbl->addr.addrh << 32);
+ hpet_address = vxtime.hpet_address;
printk(KERN_INFO PREFIX "HPET id: %#x base: %#lx\n",
hpet_tbl->id, vxtime.hpet_address);
@@ -597,10 +598,10 @@
extern unsigned long hpet_address;
hpet_address = hpet_tbl->addr.addrl;
- printk(KERN_INFO PREFIX "HPET id: %#x base: %#lx\n",
- hpet_tbl->id, hpet_address);
}
-#endif /* X86 */
+#endif /* X86 */
+ printk(KERN_INFO PREFIX "HPET id: %#x base: %#lx\n",
+ hpet_tbl->id, hpet_address);
return 0;
}
diff -ruN tod-mm1/arch/x86_64/Kconfig tod-mm2/arch/x86_64/Kconfig
--- tod-mm1/arch/x86_64/Kconfig 2005-12-01 18:13:01.000000000 -0800
+++ tod-mm2/arch/x86_64/Kconfig 2005-12-01 18:27:40.000000000 -0800
@@ -24,6 +24,14 @@
bool
default y
+config GENERIC_TIME
+ bool
+ default y
+
+config GENERIC_TIME_VSYSCALL
+ bool
+ default y
+
config SEMAPHORE_SLEEPERS
bool
default y
diff -ruN tod-mm1/arch/x86_64/kernel/Makefile tod-mm2/arch/x86_64/kernel/Makefile
--- tod-mm1/arch/x86_64/kernel/Makefile 2005-12-01 18:13:01.000000000 -0800
+++ tod-mm2/arch/x86_64/kernel/Makefile 2005-12-01 18:27:40.000000000 -0800
@@ -30,7 +30,6 @@
obj-$(CONFIG_GART_IOMMU) += pci-gart.o aperture.o
obj-$(CONFIG_DUMMY_IOMMU) += pci-nommu.o pci-dma.o
obj-$(CONFIG_KPROBES) += kprobes.o
-obj-$(CONFIG_X86_PM_TIMER) += pmtimer.o
obj-$(CONFIG_MODULES) += module.o
obj-$(CONFIG_KGDB) += kgdb_stub.o
diff -ruN tod-mm1/arch/x86_64/kernel/pmtimer.c tod-mm2/arch/x86_64/kernel/pmtimer.c
--- tod-mm1/arch/x86_64/kernel/pmtimer.c 2005-12-01 18:13:01.000000000 -0800
+++ tod-mm2/arch/x86_64/kernel/pmtimer.c 1969-12-31 16:00:00.000000000 -0800
@@ -1,101 +0,0 @@
-/* Ported over from i386 by AK, original copyright was:
- *
- * (C) Dominik Brodowski <[email protected]> 2003
- *
- * Driver to use the Power Management Timer (PMTMR) available in some
- * southbridges as primary timing source for the Linux kernel.
- *
- * Based on parts of linux/drivers/acpi/hardware/hwtimer.c, timer_pit.c,
- * timer_hpet.c, and on Arjan van de Ven's implementation for 2.4.
- *
- * This file is licensed under the GPL v2.
- *
- * Dropped all the hardware bug workarounds for now. Hopefully they
- * are not needed on 64bit chipsets.
- */
-
-#include <linux/jiffies.h>
-#include <linux/kernel.h>
-#include <linux/time.h>
-#include <linux/init.h>
-#include <linux/cpumask.h>
-#include <asm/io.h>
-#include <asm/proto.h>
-#include <asm/msr.h>
-#include <asm/vsyscall.h>
-
-/* The I/O port the PMTMR resides at.
- * The location is detected during setup_arch(),
- * in arch/i386/kernel/acpi/boot.c */
-u32 pmtmr_ioport;
-
-/* value of the Power timer at last timer interrupt */
-static u32 offset_delay;
-static u32 last_pmtmr_tick;
-
-#define ACPI_PM_MASK 0xFFFFFF /* limit it to 24 bits */
-
-static inline u32 cyc2us(u32 cycles)
-{
- /* The Power Management Timer ticks at 3.579545 ticks per microsecond.
- * 1 / PM_TIMER_FREQUENCY == 0.27936511 =~ 286/1024 [error: 0.024%]
- *
- * Even with HZ = 100, delta is at maximum 35796 ticks, so it can
- * easily be multiplied with 286 (=0x11E) without having to fear
- * u32 overflows.
- */
- cycles *= 286;
- return (cycles >> 10);
-}
-
-int pmtimer_mark_offset(void)
-{
- static int first_run = 1;
- unsigned long tsc;
- u32 lost;
-
- u32 tick = inl(pmtmr_ioport);
- u32 delta;
-
- delta = cyc2us((tick - last_pmtmr_tick) & ACPI_PM_MASK);
-
- last_pmtmr_tick = tick;
- monotonic_base += delta * NSEC_PER_USEC;
-
- delta += offset_delay;
-
- lost = delta / (USEC_PER_SEC / HZ);
- offset_delay = delta % (USEC_PER_SEC / HZ);
-
- rdtscll(tsc);
- vxtime.last_tsc = tsc - offset_delay * cpu_khz;
-
- /* don't calculate delay for first run,
- or if we've got less then a tick */
- if (first_run || (lost < 1)) {
- first_run = 0;
- offset_delay = 0;
- }
-
- return lost - 1;
-}
-
-unsigned int do_gettimeoffset_pm(void)
-{
- u32 now, offset, delta = 0;
-
- offset = last_pmtmr_tick;
- now = inl(pmtmr_ioport);
- delta = (now - offset) & ACPI_PM_MASK;
-
- return offset_delay + cyc2us(delta);
-}
-
-
-static int __init nopmtimer_setup(char *s)
-{
- pmtmr_ioport = 0;
- return 0;
-}
-
-__setup("nopmtimer", nopmtimer_setup);
diff -ruN tod-mm1/arch/x86_64/kernel/time.c tod-mm2/arch/x86_64/kernel/time.c
--- tod-mm1/arch/x86_64/kernel/time.c 2005-12-01 18:13:01.000000000 -0800
+++ tod-mm2/arch/x86_64/kernel/time.c 2005-12-01 18:27:40.000000000 -0800
@@ -26,6 +26,7 @@
#include <linux/bcd.h>
#include <linux/kallsyms.h>
#include <linux/acpi.h>
+
#ifdef CONFIG_ACPI
#include <acpi/achware.h> /* for PM timer frequency */
#endif
@@ -38,13 +39,12 @@
#include <asm/sections.h>
#include <linux/cpufreq.h>
#include <linux/hpet.h>
+#include <linux/clocksource.h>
+#include <linux/timeofday.h>
#ifdef CONFIG_X86_LOCAL_APIC
#include <asm/apic.h>
#endif
-#ifdef CONFIG_CPU_FREQ
-static void cpufreq_delayed_get(void);
-#endif
extern void i8254_timer_resume(void);
extern int using_apic_timer;
@@ -56,7 +56,9 @@
#undef HPET_HACK_ENABLE_DANGEROUS
-unsigned int cpu_khz; /* TSC clocks / usec, not used here */
+unsigned int cpu_khz; /* CPU clocks / usec, not used here */
+unsigned int tsc_khz; /* TSC clocks / usec, not used here */
+unsigned long hpet_address;
static unsigned long hpet_period; /* fsecs / HPET clock */
unsigned long hpet_tick; /* HPET clocks / interrupt */
static int hpet_use_timer;
@@ -79,112 +81,6 @@
rdtscll(*tsc);
}
-static inline unsigned long fixed_hpet_tick(void)
-{
- return hpet_tick ? hpet_tick : 1;
-}
-
-/*
- * do_gettimeoffset() returns microseconds since last timer interrupt was
- * triggered by hardware. A memory read of HPET is slower than a register read
- * of TSC, but much more reliable. It's also synchronized to the timer
- * interrupt. Note that do_gettimeoffset() may return more than hpet_tick, if a
- * timer interrupt has happened already, but vxtime.trigger wasn't updated yet.
- * This is not a problem, because jiffies hasn't updated either. They are bound
- * together by xtime_lock.
- */
-
-static inline unsigned int do_gettimeoffset_tsc(void)
-{
- unsigned long t;
- unsigned long x;
- rdtscll_sync(&t);
- if (t < vxtime.last_tsc) t = vxtime.last_tsc; /* hack */
- x = ((t - vxtime.last_tsc) * vxtime.tsc_quot) >> 32;
- return x;
-}
-
-static inline unsigned int do_gettimeoffset_hpet(void)
-{
- /* cap counter read to one tick to avoid inconsistencies */
- unsigned long counter = hpet_readl(HPET_COUNTER) - vxtime.last;
- return (min(counter,hpet_tick) * vxtime.quot) >> 32;
-}
-
-unsigned int (*do_gettimeoffset)(void) = do_gettimeoffset_tsc;
-
-/*
- * This version of gettimeofday() has microsecond resolution and better than
- * microsecond precision, as we're using at least a 10 MHz (usually 14.31818
- * MHz) HPET timer.
- */
-
-void do_gettimeofday(struct timeval *tv)
-{
- unsigned long seq, t;
- unsigned int sec, usec;
-
- do {
- seq = read_seqbegin(&xtime_lock);
-
- sec = xtime.tv_sec;
- usec = xtime.tv_nsec / 1000;
-
- /* i386 does some correction here to keep the clock
- monotonous even when ntpd is fixing drift.
- But they didn't work for me, there is a non monotonic
- clock anyways with ntp.
- I dropped all corrections now until a real solution can
- be found. Note when you fix it here you need to do the same
- in arch/x86_64/kernel/vsyscall.c and export all needed
- variables in vmlinux.lds. -AK */
-
- t = (jiffies - wall_jiffies) * (1000000L / HZ) +
- do_gettimeoffset();
- usec += t;
-
- } while (read_seqretry(&xtime_lock, seq));
-
- tv->tv_sec = sec + usec / 1000000;
- tv->tv_usec = usec % 1000000;
-}
-
-EXPORT_SYMBOL(do_gettimeofday);
-
-/*
- * settimeofday() first undoes the correction that gettimeofday would do
- * on the time, and then saves it. This is ugly, but has been like this for
- * ages already.
- */
-
-int do_settimeofday(struct timespec *tv)
-{
- time_t wtm_sec, sec = tv->tv_sec;
- long wtm_nsec, nsec = tv->tv_nsec;
-
- if ((unsigned long)tv->tv_nsec >= NSEC_PER_SEC)
- return -EINVAL;
-
- write_seqlock_irq(&xtime_lock);
-
- nsec -= do_gettimeoffset() * 1000 +
- (jiffies - wall_jiffies) * (NSEC_PER_SEC/HZ);
-
- wtm_sec = wall_to_monotonic.tv_sec + (xtime.tv_sec - sec);
- wtm_nsec = wall_to_monotonic.tv_nsec + (xtime.tv_nsec - nsec);
-
- set_normalized_timespec(&xtime, sec, nsec);
- set_normalized_timespec(&wall_to_monotonic, wtm_sec, wtm_nsec);
-
- ntp_clear();
-
- write_sequnlock_irq(&xtime_lock);
- clock_was_set();
- return 0;
-}
-
-EXPORT_SYMBOL(do_settimeofday);
-
unsigned long profile_pc(struct pt_regs *regs)
{
unsigned long pc = instruction_pointer(regs);
@@ -284,90 +180,8 @@
spin_unlock(&rtc_lock);
}
-
-/* monotonic_clock(): returns # of nanoseconds passed since time_init()
- * Note: This function is required to return accurate
- * time even in the absence of multiple timer ticks.
- */
-unsigned long long monotonic_clock(void)
-{
- unsigned long seq;
- u32 last_offset, this_offset, offset;
- unsigned long long base;
-
- if (vxtime.mode == VXTIME_HPET) {
- do {
- seq = read_seqbegin(&xtime_lock);
-
- last_offset = vxtime.last;
- base = monotonic_base;
- this_offset = hpet_readl(HPET_COUNTER);
-
- } while (read_seqretry(&xtime_lock, seq));
- offset = (this_offset - last_offset);
- offset *=(NSEC_PER_SEC/HZ)/fixed_hpet_tick();
- return base + offset;
- }else{
- do {
- seq = read_seqbegin(&xtime_lock);
-
- last_offset = vxtime.last_tsc;
- base = monotonic_base;
- } while (read_seqretry(&xtime_lock, seq));
- sync_core();
- rdtscll(this_offset);
- offset = (this_offset - last_offset)*1000/cpu_khz;
- return base + offset;
- }
-
-
-}
-EXPORT_SYMBOL(monotonic_clock);
-
-static noinline void handle_lost_ticks(int lost, struct pt_regs *regs)
-{
- static long lost_count;
- static int warned;
-
- if (report_lost_ticks) {
- printk(KERN_WARNING "time.c: Lost %d timer "
- "tick(s)! ", lost);
- print_symbol("rip %s)\n", regs->rip);
- }
-
- if (lost_count == 1000 && !warned) {
- printk(KERN_WARNING
- "warning: many lost ticks.\n"
- KERN_WARNING "Your time source seems to be instable or "
- "some driver is hogging interupts\n");
- print_symbol("rip %s\n", regs->rip);
- if (vxtime.mode == VXTIME_TSC && vxtime.hpet_address) {
- printk(KERN_WARNING "Falling back to HPET\n");
- vxtime.last = hpet_readl(HPET_T0_CMP) - hpet_tick;
- vxtime.mode = VXTIME_HPET;
- do_gettimeoffset = do_gettimeoffset_hpet;
- }
- /* else should fall back to PIT, but code missing. */
- warned = 1;
- } else
- lost_count++;
-
-#ifdef CONFIG_CPU_FREQ
- /* In some cases the CPU can change frequency without us noticing
- (like going into thermal throttle)
- Give cpufreq a change to catch up. */
- if ((lost_count+1) % 25 == 0) {
- cpufreq_delayed_get();
- }
-#endif
-}
-
static irqreturn_t timer_interrupt(int irq, void *dev_id, struct pt_regs *regs)
{
- static unsigned long rtc_update = 0;
- unsigned long tsc;
- int delay, offset = 0, lost = 0;
-
/*
* Here we are in the timer irq handler. We have irqs locally disabled (so we
* don't need spin_lock_irqsave()) but we don't know if the timer_bh is running
@@ -377,67 +191,6 @@
write_seqlock(&xtime_lock);
- if (vxtime.hpet_address)
- offset = hpet_readl(HPET_COUNTER);
-
- if (hpet_use_timer) {
- /* if we're using the hpet timer functionality,
- * we can more accurately know the counter value
- * when the timer interrupt occured.
- */
- offset = hpet_readl(HPET_T0_CMP) - hpet_tick;
- delay = hpet_readl(HPET_COUNTER) - offset;
- } else {
- spin_lock(&i8253_lock);
- outb_p(0x00, 0x43);
- delay = inb_p(0x40);
- delay |= inb(0x40) << 8;
- spin_unlock(&i8253_lock);
- delay = LATCH - 1 - delay;
- }
-
- rdtscll_sync(&tsc);
-
- if (vxtime.mode == VXTIME_HPET) {
- if (offset - vxtime.last > hpet_tick) {
- lost = (offset - vxtime.last) / fixed_hpet_tick() - 1;
- }
-
- monotonic_base += (offset - vxtime.last)*(NSEC_PER_SEC/HZ) /
- fixed_hpet_tick();
-
- vxtime.last = offset;
-#ifdef CONFIG_X86_PM_TIMER
- } else if (vxtime.mode == VXTIME_PMTMR) {
- lost = pmtimer_mark_offset();
-#endif
- } else {
- offset = (((tsc - vxtime.last_tsc) *
- vxtime.tsc_quot) >> 32) - (USEC_PER_SEC / HZ);
-
- if (offset < 0)
- offset = 0;
-
- if (offset > (USEC_PER_SEC / HZ)) {
- lost = offset / (USEC_PER_SEC / HZ);
- offset %= (USEC_PER_SEC / HZ);
- }
-
- monotonic_base += (tsc - vxtime.last_tsc)*1000000/cpu_khz ;
-
- vxtime.last_tsc = tsc - vxtime.quot * delay / vxtime.tsc_quot;
-
- if ((((tsc - vxtime.last_tsc) *
- vxtime.tsc_quot) >> 32) < offset)
- vxtime.last_tsc = tsc -
- (((long) offset << 32) / vxtime.tsc_quot) - 1;
- }
-
- if (lost > 0) {
- handle_lost_ticks(lost, regs);
- jiffies += lost;
- }
-
/*
* Do the timer stuff.
*/
@@ -460,20 +213,6 @@
smp_local_timer_interrupt(regs);
#endif
-/*
- * If we have an externally synchronized Linux clock, then update CMOS clock
- * accordingly every ~11 minutes. set_rtc_mmss() will be called in the jiffy
- * closest to exactly 500 ms before the next second. If the update fails, we
- * don't care, as it'll be updated on the next turn, and the problem (time way
- * off) isn't likely to go away much sooner anyway.
- */
-
- if (ntp_synced() && xtime.tv_sec > rtc_update &&
- abs(xtime.tv_nsec - 500000000) <= tick_nsec / 2) {
- set_rtc_mmss(xtime.tv_sec);
- rtc_update = xtime.tv_sec + 660;
- }
-
write_sequnlock(&xtime_lock);
return IRQ_HANDLED;
@@ -514,6 +253,33 @@
return cycles_2_ns(a);
}
+/* code to compensate for TSC C3 stalls: */
+static u64 tsc_c3_offset;
+static int tsc_unstable;
+
+static inline int check_tsc_unstable(void)
+{
+ return tsc_unstable;
+}
+static inline void mark_tsc_unstable(void)
+{
+ tsc_unstable = 1;;
+}
+
+void tsc_c3_compensate(unsigned long nsecs)
+{
+ u64 cycles = ((u64)nsecs * tsc_khz)/1000000;
+
+ tsc_c3_offset += cycles;
+}
+
+EXPORT_SYMBOL_GPL(tsc_c3_compensate);
+
+static inline u64 tsc_read_c3_time(void)
+{
+ return tsc_c3_offset;
+}
+
unsigned long get_cmos_time(void)
{
unsigned int timeout, year, mon, day, hour, min, sec;
@@ -574,6 +340,30 @@
return mktime(year, mon, day, hour, min, sec);
}
+/* arch specific timeofday hooks: */
+u64 read_persistent_clock(void)
+{
+ return (u64)get_cmos_time() * NSEC_PER_SEC;
+}
+
+void sync_persistent_clock(struct timespec ts)
+{
+ static unsigned long rtc_update = 0;
+ /*
+ * If we have an externally synchronized Linux clock, then update
+ * CMOS clock accordingly every ~11 minutes. set_rtc_mmss() will
+ * be called in the jiffy closest to exactly 500 ms before the
+ * next second. If the update fails, we don't care, as it'll be
+ * updated on the next turn, and the problem (time way off) isn't
+ * likely to go away much sooner anyway.
+ */
+ if (ts.tv_sec > rtc_update &&
+ abs(ts.tv_nsec - 500000000) <= tick_nsec / 2) {
+ set_rtc_mmss(xtime.tv_sec);
+ rtc_update = xtime.tv_sec + 660;
+ }
+}
+
#ifdef CONFIG_CPU_FREQ
/* Frequency scaling support. Adjust the TSC based timer when the cpu frequency
@@ -601,23 +391,6 @@
cpufreq_delayed_issched = 0;
}
-/* if we notice lost ticks, schedule a call to cpufreq_get() as it tries
- * to verify the CPU frequency the timing core thinks the CPU is running
- * at is still correct.
- */
-static void cpufreq_delayed_get(void)
-{
- static int warned;
- if (cpufreq_init && !cpufreq_delayed_issched) {
- cpufreq_delayed_issched = 1;
- if (!warned) {
- warned = 1;
- printk(KERN_DEBUG "Losing some ticks... checking if CPU frequency changed.\n");
- }
- schedule_work(&cpufreq_delayed_get_work);
- }
-}
-
static unsigned int ref_freq = 0;
static unsigned long loops_per_jiffy_ref = 0;
@@ -652,8 +425,11 @@
cpufreq_scale(loops_per_jiffy_ref, ref_freq, freq->new);
cpu_khz = cpufreq_scale(cpu_khz_ref, ref_freq, freq->new);
- if (!(freq->flags & CPUFREQ_CONST_LOOPS))
+ if (!(freq->flags & CPUFREQ_CONST_LOOPS)) {
vxtime.tsc_quot = (1000L << 32) / cpu_khz;
+ tsc_khz = cpu_khz;
+ }
+
}
set_cyc2ns_scale(cpu_khz_ref);
@@ -917,18 +693,12 @@
if (hpet_use_timer) {
cpu_khz = hpet_calibrate_tsc();
timename = "HPET";
-#ifdef CONFIG_X86_PM_TIMER
- } else if (pmtmr_ioport) {
- vxtime_hz = PM_TIMER_FREQUENCY;
- timename = "PM";
- pit_init();
- cpu_khz = pit_calibrate_tsc();
-#endif
} else {
pit_init();
cpu_khz = pit_calibrate_tsc();
timename = "PIT";
}
+ tsc_khz = cpu_khz;
printk(KERN_INFO "time.c: Using %ld.%06ld MHz %s timer.\n",
vxtime_hz / 1000000, vxtime_hz % 1000000, timename);
@@ -970,31 +740,8 @@
*/
void __init time_init_gtod(void)
{
- char *timetype;
-
if (unsynchronized_tsc())
- notsc = 1;
- if (vxtime.hpet_address && notsc) {
- timetype = hpet_use_timer ? "HPET" : "PIT/HPET";
- vxtime.last = hpet_readl(HPET_T0_CMP) - hpet_tick;
- vxtime.mode = VXTIME_HPET;
- do_gettimeoffset = do_gettimeoffset_hpet;
-#ifdef CONFIG_X86_PM_TIMER
- /* Using PM for gettimeofday is quite slow, but we have no other
- choice because the TSC is too unreliable on some systems. */
- } else if (pmtmr_ioport && !vxtime.hpet_address && notsc) {
- timetype = "PM";
- do_gettimeoffset = do_gettimeoffset_pm;
- vxtime.mode = VXTIME_PMTMR;
- sysctl_vsyscall = 0;
- printk(KERN_INFO "Disabling vsyscall due to use of PM timer\n");
-#endif
- } else {
- timetype = hpet_use_timer ? "HPET/TSC" : "PIT/TSC";
- vxtime.mode = VXTIME_TSC;
- }
-
- printk(KERN_INFO "time.c: Using %s based timekeeping.\n", timetype);
+ mark_tsc_unstable();
}
__setup("report_lost_ticks", time_setup);
@@ -1017,7 +764,6 @@
static int timer_resume(struct sys_device *dev)
{
- unsigned long flags;
unsigned long sec;
unsigned long ctime = get_cmos_time();
unsigned long sleep_length = (ctime - sleep_start) * HZ;
@@ -1028,10 +774,6 @@
i8254_timer_resume();
sec = ctime + clock_cmos_diff;
- write_seqlock_irqsave(&xtime_lock,flags);
- xtime.tv_sec = sec;
- xtime.tv_nsec = 0;
- write_sequnlock_irqrestore(&xtime_lock,flags);
jiffies += sleep_length;
wall_jiffies += sleep_length;
touch_softlockup_watchdog();
@@ -1306,3 +1048,159 @@
__setup("notsc", notsc_setup);
+/* clock source code: */
+
+static unsigned long current_tsc_khz = 0;
+
+static int tsc_update_callback(void);
+
+static cycle_t read_tsc(void)
+{
+ cycle_t ret;
+
+ rdtscll(ret);
+
+ return ret;
+}
+
+static cycle_t __vsyscall_fn vread_tsc(void* unused)
+{
+ cycle_t ret;
+
+ rdtscll(ret);
+
+ return ret;
+}
+
+static cycle_t read_tsc_c3(void)
+{
+ cycle_t ret;
+
+ rdtscll(ret);
+
+ return ret + tsc_read_c3_time();
+}
+
+static struct clocksource clocksource_tsc = {
+ .name = "tsc",
+ .rating = 300,
+ .read = read_tsc,
+ .vread = vread_tsc,
+ .mask = (cycle_t)-1,
+ .mult = 0, /* to be set */
+ .shift = 22,
+ .update_callback = tsc_update_callback,
+ .is_continuous = 1,
+};
+
+static int tsc_update_callback(void)
+{
+ int change = 0;
+
+ /* check to see if we should switch to the safe clocksource: */
+ if (tsc_read_c3_time() &&
+ strncmp(clocksource_tsc.name, "c3tsc", 5)) {
+ printk("Falling back to C3 safe TSC\n");
+ clocksource_tsc.read = read_tsc_c3;
+ clocksource_tsc.vread = 0;
+ clocksource_tsc.name = "c3tsc";
+ change = 1;
+ }
+
+ if (clocksource_tsc.rating != 50 && check_tsc_unstable()) {
+ clocksource_tsc.rating = 50;
+ reselect_clocksource();
+ change = 1;
+ }
+
+ /* only update if tsc_khz has changed: */
+ if (current_tsc_khz != tsc_khz){
+ current_tsc_khz = tsc_khz;
+ clocksource_tsc.mult = clocksource_khz2mult(current_tsc_khz,
+ clocksource_tsc.shift);
+ change = 1;
+ }
+ return change;
+}
+
+static int __init init_tsc_clocksource(void)
+{
+ if (!notsc && tsc_khz) {
+ current_tsc_khz = tsc_khz;
+ clocksource_tsc.mult = clocksource_khz2mult(current_tsc_khz,
+ clocksource_tsc.shift);
+ register_clocksource(&clocksource_tsc);
+ }
+ return 0;
+}
+
+module_init(init_tsc_clocksource);
+
+
+#define HPET_MASK 0xFFFFFFFF
+#define HPET_SHIFT 22
+
+/* FSEC = 10^-15 NSEC = 10^-9 */
+#define FSEC_PER_NSEC 1000000
+
+static void *hpet_ptr;
+
+static cycle_t read_hpet(void)
+{
+ return (cycle_t)readl(hpet_ptr);
+}
+
+static cycle_t __vsyscall_fn vread_hpet(void* ptr)
+{
+ return (cycle_t)readl((void *)fix_to_virt(VSYSCALL_HPET) + 0xf0);
+}
+
+struct clocksource clocksource_hpet = {
+ .name = "hpet",
+ .rating = 250,
+ .read = read_hpet,
+ .vread = vread_hpet,
+ .mask = (cycle_t)HPET_MASK,
+ .mult = 0, /* set below */
+ .shift = HPET_SHIFT,
+ .is_continuous = 1,
+};
+
+static int __init init_hpet_clocksource(void)
+{
+ unsigned long hpet_period;
+ void __iomem *hpet_base;
+ u64 tmp;
+
+ if (!hpet_address)
+ return -ENODEV;
+
+ /* calculate the hpet address: */
+ hpet_base =
+ (void __iomem*)ioremap_nocache(hpet_address, HPET_MMAP_SIZE);
+ hpet_ptr = hpet_base + HPET_COUNTER;
+
+ /* calculate the frequency: */
+ hpet_period = readl(hpet_base + HPET_PERIOD);
+
+ /*
+ * hpet period is in femto seconds per cycle
+ * so we need to convert this to ns/cyc units
+ * aproximated by mult/2^shift
+ *
+ * fsec/cyc * 1nsec/1000000fsec = nsec/cyc = mult/2^shift
+ * fsec/cyc * 1ns/1000000fsec * 2^shift = mult
+ * fsec/cyc * 2^shift * 1nsec/1000000fsec = mult
+ * (fsec/cyc << shift)/1000000 = mult
+ * (hpet_period << shift)/FSEC_PER_NSEC = mult
+ */
+ tmp = (u64)hpet_period << HPET_SHIFT;
+ do_div(tmp, FSEC_PER_NSEC);
+ clocksource_hpet.mult = (u32)tmp;
+
+ register_clocksource(&clocksource_hpet);
+
+ return 0;
+}
+
+module_init(init_hpet_clocksource);
diff -ruN tod-mm1/arch/x86_64/kernel/vmlinux.lds.S tod-mm2/arch/x86_64/kernel/vmlinux.lds.S
--- tod-mm1/arch/x86_64/kernel/vmlinux.lds.S 2005-12-01 18:13:01.000000000 -0800
+++ tod-mm2/arch/x86_64/kernel/vmlinux.lds.S 2005-12-01 18:27:40.000000000 -0800
@@ -99,6 +99,18 @@
.jiffies : AT(VLOAD(.jiffies)) { *(.jiffies) }
jiffies = VVIRT(.jiffies);
+ .vsyscall_fn : AT(VLOAD(.vsyscall_fn)) { *(.vsyscall_fn) }
+ .vsyscall_data : AT(VLOAD(.vsyscall_data)) { *(.vsyscall_data) }
+
+ .vsyscall_gtod_data : AT(VLOAD(.vsyscall_gtod_data)) { *(.vsyscall_gtod_data) }
+ vsyscall_gtod_data = VVIRT(.vsyscall_gtod_data);
+
+ .vsyscall_gtod_lock : AT(VLOAD(.vsyscall_gtod_lock)) { *(.vsyscall_gtod_lock) }
+ vsyscall_gtod_lock = VVIRT(.vsyscall_gtod_lock);
+
+ .vsyscall_fn : AT(VLOAD(.vsyscall_fn)) { *(.vsyscall_fn) }
+ .vsyscall_data : AT(VLOAD(.vsyscall_data)) { *(.vsyscall_data) }
+
.vsyscall_1 ADDR(.vsyscall_0) + 1024: AT(VLOAD(.vsyscall_1)) { *(.vsyscall_1) }
.vsyscall_2 ADDR(.vsyscall_0) + 2048: AT(VLOAD(.vsyscall_2)) { *(.vsyscall_2) }
.vsyscall_3 ADDR(.vsyscall_0) + 3072: AT(VLOAD(.vsyscall_3)) { *(.vsyscall_3) }
diff -ruN tod-mm1/arch/x86_64/kernel/vsyscall.c tod-mm2/arch/x86_64/kernel/vsyscall.c
--- tod-mm1/arch/x86_64/kernel/vsyscall.c 2005-12-01 18:13:01.000000000 -0800
+++ tod-mm2/arch/x86_64/kernel/vsyscall.c 2005-12-01 18:27:40.000000000 -0800
@@ -19,6 +19,8 @@
* want per guest time just set the kernel.vsyscall64 sysctl to 0.
*/
+#include <linux/timeofday.h>
+#include <linux/clocksource.h>
#include <linux/time.h>
#include <linux/init.h>
#include <linux/kernel.h>
@@ -27,8 +29,10 @@
#include <linux/jiffies.h>
#include <linux/sysctl.h>
+
#include <asm/vsyscall.h>
#include <asm/pgtable.h>
+#include <asm/unistd.h>
#include <asm/page.h>
#include <asm/fixmap.h>
#include <asm/errno.h>
@@ -40,7 +44,18 @@
int __sysctl_vsyscall __section_sysctl_vsyscall = 1;
seqlock_t __xtime_lock __section_xtime_lock = SEQLOCK_UNLOCKED;
-#include <asm/unistd.h>
+struct vsyscall_gtod_data_t {
+ struct timeval wall_time_tv;
+ struct timezone sys_tz;
+ cycle_t offset_base;
+ struct clocksource clock;
+};
+
+extern struct vsyscall_gtod_data_t vsyscall_gtod_data;
+struct vsyscall_gtod_data_t __vsyscall_gtod_data __section_vsyscall_gtod_data;
+
+extern seqlock_t vsyscall_gtod_lock;
+seqlock_t __vsyscall_gtod_lock __section_vsyscall_gtod_lock = SEQLOCK_UNLOCKED;
static force_inline void timeval_normalize(struct timeval * tv)
{
@@ -53,40 +68,66 @@
}
}
-static force_inline void do_vgettimeofday(struct timeval * tv)
+/*
+ * XXX - this is ugly. gettimeofday() has a label in it so we can't
+ * call it twice.
+ */
+static force_inline int syscall_gtod(struct timeval *tv, struct timezone *tz)
+{
+ int ret;
+
+ asm volatile("syscall"
+ : "=a" (ret)
+ : "0" (__NR_gettimeofday),"D" (tv),"S" (tz)
+ : __syscall_clobber);
+
+ return ret;
+}
+
+static force_inline void do_vgettimeofday(struct timeval *tv)
{
- long sequence, t;
- unsigned long sec, usec;
+ cycle_t now, cycle_delta;
+ nsec_t nsec_delta;
+ unsigned long seq;
do {
- sequence = read_seqbegin(&__xtime_lock);
-
- sec = __xtime.tv_sec;
- usec = (__xtime.tv_nsec / 1000) +
- (__jiffies - __wall_jiffies) * (1000000 / HZ);
-
- if (__vxtime.mode != VXTIME_HPET) {
- sync_core();
- rdtscll(t);
- if (t < __vxtime.last_tsc)
- t = __vxtime.last_tsc;
- usec += ((t - __vxtime.last_tsc) *
- __vxtime.tsc_quot) >> 32;
- /* See comment in x86_64 do_gettimeofday. */
- } else {
- usec += ((readl((void *)fix_to_virt(VSYSCALL_HPET) + 0xf0) -
- __vxtime.last) * __vxtime.quot) >> 32;
+ seq = read_seqbegin(&__vsyscall_gtod_lock);
+
+ if (!__vsyscall_gtod_data.clock.vread) {
+ syscall_gtod(tv, NULL);
+ return;
}
- } while (read_seqretry(&__xtime_lock, sequence));
- tv->tv_sec = sec + usec / 1000000;
- tv->tv_usec = usec % 1000000;
+ /* read the timeosurce and calc cycle_delta */
+ now = __vsyscall_gtod_data.clock.vread(
+ __vsyscall_gtod_data.clock.vdata);
+
+ cycle_delta = (now - __vsyscall_gtod_data.offset_base)
+ & __vsyscall_gtod_data.clock.mask;
+
+ /* convert cycles to nsecs */
+ nsec_delta = cycle_delta * __vsyscall_gtod_data.clock.mult;
+ nsec_delta = nsec_delta >> __vsyscall_gtod_data.clock.shift;
+
+ /* add nsec offset to wall_time_tv */
+ *tv = __vsyscall_gtod_data.wall_time_tv;
+ do_div(nsec_delta, NSEC_PER_USEC);
+ tv->tv_usec += (unsigned long) nsec_delta;
+
+ while (tv->tv_usec > USEC_PER_SEC) {
+ tv->tv_sec += 1;
+ tv->tv_usec -= USEC_PER_SEC;
+ }
+ } while (read_seqretry(&__vsyscall_gtod_lock, seq));
}
-/* RED-PEN may want to readd seq locking, but then the variable should be write-once. */
+/*
+ * RED-PEN may want to readd seq locking, but then the variable should be
+ * write-once.
+ */
static force_inline void do_get_tz(struct timezone * tz)
{
- *tz = __sys_tz;
+ *tz = __vsyscall_gtod_data.sys_tz;
}
static force_inline int gettimeofday(struct timeval *tv, struct timezone *tz)
@@ -122,11 +163,16 @@
* unlikely */
time_t __vsyscall(1) vtime(time_t *t)
{
+ struct timeval tv;
+
if (unlikely(!__sysctl_vsyscall))
return time_syscall(t);
- else if (t)
- *t = __xtime.tv_sec;
- return __xtime.tv_sec;
+
+ vgettimeofday(&tv, 0);
+ if (t)
+ *t = tv.tv_sec;
+
+ return tv.tv_sec;
}
long __vsyscall(2) venosys_0(void)
@@ -139,6 +185,38 @@
return -ENOSYS;
}
+struct clocksource *curr_clock;
+
+void arch_update_vsyscall_gtod(struct timespec wall_time, cycle_t offset_base,
+ struct clocksource *clock, int ntp_adj)
+{
+ unsigned long flags;
+
+ write_seqlock_irqsave(&vsyscall_gtod_lock, flags);
+
+ /* XXX - hackitty hack hack. this is terrible! */
+ if (curr_clock != clock)
+ curr_clock = clock;
+
+ /* save off wall time as timeval: */
+ vsyscall_gtod_data.wall_time_tv.tv_sec = wall_time.tv_sec;
+ vsyscall_gtod_data.wall_time_tv.tv_usec = wall_time.tv_nsec/1000;
+
+ /* save offset_base: */
+ vsyscall_gtod_data.offset_base = offset_base;
+
+ /* copy current clocksource: */
+ vsyscall_gtod_data.clock = *clock;
+
+ /* apply ntp adjustment to clocksource mult: */
+ vsyscall_gtod_data.clock.mult += ntp_adj;
+
+ /* save off current timezone: */
+ vsyscall_gtod_data.sys_tz = sys_tz;
+
+ write_sequnlock_irqrestore(&vsyscall_gtod_lock, flags);
+}
+
#ifdef CONFIG_SYSCTL
#define SYSCALL 0x050f
@@ -217,6 +295,7 @@
BUG_ON((unsigned long) &vtime != VSYSCALL_ADDR(__NR_vtime));
BUG_ON((VSYSCALL_ADDR(0) != __fix_to_virt(VSYSCALL_FIRST_PAGE)));
map_vsyscall();
+ sysctl_vsyscall = 1;
#ifdef CONFIG_SYSCTL
register_sysctl_table(kernel_root_table2, 0);
#endif
diff -ruN tod-mm1/include/asm-generic/div64.h tod-mm2/include/asm-generic/div64.h
--- tod-mm1/include/asm-generic/div64.h 2005-12-01 18:13:31.000000000 -0800
+++ tod-mm2/include/asm-generic/div64.h 2005-12-01 18:27:40.000000000 -0800
@@ -55,4 +55,13 @@
#endif /* BITS_PER_LONG */
+#ifndef div_long_long_rem
+#define div_long_long_rem(dividend,divisor,remainder) \
+({ \
+ u64 result = dividend; \
+ *remainder = do_div(result,divisor); \
+ result; \
+})
+#endif
+
#endif /* _ASM_GENERIC_DIV64_H */
diff -ruN tod-mm1/include/asm-x86_64/hpet.h tod-mm2/include/asm-x86_64/hpet.h
--- tod-mm1/include/asm-x86_64/hpet.h 2005-12-01 18:13:35.000000000 -0800
+++ tod-mm2/include/asm-x86_64/hpet.h 2005-12-01 18:27:40.000000000 -0800
@@ -1,6 +1,6 @@
#ifndef _ASM_X8664_HPET_H
#define _ASM_X8664_HPET_H 1
-
+#include <asm/fixmap.h>
/*
* Documentation on HPET can be found at:
* http://www.intel.com/ial/home/sp/pcmmspec.htm
@@ -51,6 +51,7 @@
#define HPET_TN_ROUTE_SHIFT 9
+extern unsigned long hpet_address; /* hpet memory map physical address */
extern int is_hpet_enabled(void);
extern int hpet_rtc_timer_init(void);
extern int oem_force_hpet_timer(void);
diff -ruN tod-mm1/include/asm-x86_64/timeofday.h tod-mm2/include/asm-x86_64/timeofday.h
--- tod-mm1/include/asm-x86_64/timeofday.h 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/include/asm-x86_64/timeofday.h 2005-12-01 18:27:40.000000000 -0800
@@ -0,0 +1,4 @@
+#ifndef _ASM_X86_64_TIMEOFDAY_H
+#define _ASM_X86_64_TIMEOFDAY_H
+#include <asm-generic/timeofday.h>
+#endif
diff -ruN tod-mm1/include/asm-x86_64/timex.h tod-mm2/include/asm-x86_64/timex.h
--- tod-mm1/include/asm-x86_64/timex.h 2005-12-01 18:13:35.000000000 -0800
+++ tod-mm2/include/asm-x86_64/timex.h 2005-12-01 18:27:40.000000000 -0800
@@ -24,6 +24,8 @@
}
extern unsigned int cpu_khz;
+extern unsigned int tsc_khz;
+extern void tsc_c3_compensate(unsigned long usecs);
extern int read_current_timer(unsigned long *timer_value);
#define ARCH_HAS_READ_CURRENT_TIMER 1
diff -ruN tod-mm1/include/asm-x86_64/vsyscall.h tod-mm2/include/asm-x86_64/vsyscall.h
--- tod-mm1/include/asm-x86_64/vsyscall.h 2005-12-01 18:13:35.000000000 -0800
+++ tod-mm2/include/asm-x86_64/vsyscall.h 2005-12-01 18:27:40.000000000 -0800
@@ -14,7 +14,7 @@
#define VSYSCALL_ADDR(vsyscall_nr) (VSYSCALL_START+VSYSCALL_SIZE*(vsyscall_nr))
#ifdef __KERNEL__
-
+/* XXX - All of these are unused w/ CONFIG_GENERIC_TIME and should be removed */
#define __section_vxtime __attribute__ ((unused, __section__ (".vxtime"), aligned(16)))
#define __section_wall_jiffies __attribute__ ((unused, __section__ (".wall_jiffies"), aligned(16)))
#define __section_jiffies __attribute__ ((unused, __section__ (".jiffies"), aligned(16)))
@@ -23,6 +23,12 @@
#define __section_xtime __attribute__ ((unused, __section__ (".xtime"), aligned(16)))
#define __section_xtime_lock __attribute__ ((unused, __section__ (".xtime_lock"), aligned(16)))
+/* Definitions for CONFIG_GENERIC_TIME definitions */
+#define __section_vsyscall_gtod_data __attribute__ ((unused, __section__ (".vsyscall_gtod_data"),aligned(16)))
+#define __section_vsyscall_gtod_lock __attribute__ ((unused, __section__ (".vsyscall_gtod_lock"),aligned(16)))
+#define __vsyscall_fn __attribute__ ((unused,__section__(".vsyscall_fn")))
+#define __vsyscall_data __attribute__ ((unused,__section__(".vsyscall_data")))
+
#define VXTIME_TSC 1
#define VXTIME_HPET 2
#define VXTIME_PMTMR 3
All,
The conversion of i386 to use the generic timeofday subsystem has been
split into 6 parts. This patch, the second of six, is a cleanup patch
for the i386 arch in preparation of moving the the generic timeofday
infrastructure. It moves some code from timer_tsc.c to a new tsc.c
file.
It applies on top of my timeofday-arch-i386-part1 patch. This patch is
part the timeofday-arch-i386 patchset, so without the following parts it
is not expected to compile.
thanks
-john
Signed-off-by: John Stultz <[email protected]>
arch/i386/kernel/Makefile | 2
arch/i386/kernel/timers/common.c | 84 ---------
arch/i386/kernel/timers/timer_tsc.c | 212 ------------------------
arch/i386/kernel/tsc.c | 312 ++++++++++++++++++++++++++++++++++++
include/asm-i386/timex.h | 34 ---
include/asm-i386/tsc.h | 44 +++++
6 files changed, 358 insertions(+), 330 deletions(-)
linux-2.6.15-rc3-mm1_timeofday-arch-i386-part2_B12.patch
==========================
diff -ruN tod-mm1/arch/i386/kernel/Makefile tod-mm2/arch/i386/kernel/Makefile
--- tod-mm1/arch/i386/kernel/Makefile 2005-12-01 18:24:31.000000000 -0800
+++ tod-mm2/arch/i386/kernel/Makefile 2005-12-01 18:24:47.000000000 -0800
@@ -7,7 +7,7 @@
obj-y := process.o semaphore.o signal.o entry.o traps.o irq.o vm86.o \
ptrace.o time.o ioport.o ldt.o setup.o i8259.o sys_i386.o \
pci-dma.o i386_ksyms.o i387.o dmi_scan.o bootflag.o \
- quirks.o i8237.o i8253.o
+ quirks.o i8237.o i8253.o tsc.o
obj-y += cpu/
obj-y += timers/
diff -ruN tod-mm1/arch/i386/kernel/timers/common.c tod-mm2/arch/i386/kernel/timers/common.c
--- tod-mm1/arch/i386/kernel/timers/common.c 2005-12-01 18:12:56.000000000 -0800
+++ tod-mm2/arch/i386/kernel/timers/common.c 2005-12-01 18:24:34.000000000 -0800
@@ -14,66 +14,6 @@
#include "mach_timer.h"
-/* ------ Calibrate the TSC -------
- * Return 2^32 * (1 / (TSC clocks per usec)) for do_fast_gettimeoffset().
- * Too much 64-bit arithmetic here to do this cleanly in C, and for
- * accuracy's sake we want to keep the overhead on the CTC speaker (channel 2)
- * output busy loop as low as possible. We avoid reading the CTC registers
- * directly because of the awkward 8-bit access mechanism of the 82C54
- * device.
- */
-
-#define CALIBRATE_TIME (5 * 1000020/HZ)
-
-unsigned long calibrate_tsc(void)
-{
- mach_prepare_counter();
-
- {
- unsigned long startlow, starthigh;
- unsigned long endlow, endhigh;
- unsigned long count;
-
- rdtsc(startlow,starthigh);
- mach_countup(&count);
- rdtsc(endlow,endhigh);
-
-
- /* Error: ECTCNEVERSET */
- if (count <= 1)
- goto bad_ctc;
-
- /* 64-bit subtract - gcc just messes up with long longs */
- __asm__("subl %2,%0\n\t"
- "sbbl %3,%1"
- :"=a" (endlow), "=d" (endhigh)
- :"g" (startlow), "g" (starthigh),
- "0" (endlow), "1" (endhigh));
-
- /* Error: ECPUTOOFAST */
- if (endhigh)
- goto bad_ctc;
-
- /* Error: ECPUTOOSLOW */
- if (endlow <= CALIBRATE_TIME)
- goto bad_ctc;
-
- __asm__("divl %2"
- :"=a" (endlow), "=d" (endhigh)
- :"r" (endlow), "0" (0), "1" (CALIBRATE_TIME));
-
- return endlow;
- }
-
- /*
- * The CTC wasn't reliable: we got a hit on the very first read,
- * or the CPU was so fast/slow that the quotient wouldn't fit in
- * 32 bits..
- */
-bad_ctc:
- return 0;
-}
-
#ifdef CONFIG_HPET_TIMER
/* ------ Calibrate the TSC using HPET -------
* Return 2^32 * (1 / (TSC clocks per usec)) for getting the CPU freq.
@@ -146,27 +86,3 @@
rdtscl(retval);
return retval;
}
-
-
-/* calculate cpu_khz */
-void init_cpu_khz(void)
-{
- if (cpu_has_tsc) {
- unsigned long tsc_quotient = calibrate_tsc();
- if (tsc_quotient) {
- /* report CPU clock rate in Hz.
- * The formula is (10^6 * 2^32) / (2^32 * 1 / (clocks/us)) =
- * clock/second. Our precision is about 100 ppm.
- */
- { unsigned long eax=0, edx=1000;
- __asm__("divl %2"
- :"=a" (cpu_khz), "=d" (edx)
- :"r" (tsc_quotient),
- "0" (eax), "1" (edx));
- printk("Detected %u.%03u MHz processor.\n",
- cpu_khz / 1000, cpu_khz % 1000);
- }
- }
- }
-}
-
diff -ruN tod-mm1/arch/i386/kernel/timers/timer_tsc.c tod-mm2/arch/i386/kernel/timers/timer_tsc.c
--- tod-mm1/arch/i386/kernel/timers/timer_tsc.c 2005-12-01 18:12:56.000000000 -0800
+++ tod-mm2/arch/i386/kernel/timers/timer_tsc.c 2005-12-01 18:24:34.000000000 -0800
@@ -32,10 +32,6 @@
static struct timer_opts timer_tsc;
#endif
-static inline void cpufreq_delayed_get(void);
-
-int tsc_disable __devinitdata = 0;
-
static int use_tsc;
/* Number of usecs that the last interrupt was delayed */
static int delay_at_last_interrupt;
@@ -45,39 +41,6 @@
static unsigned long long monotonic_base;
static seqlock_t monotonic_lock = SEQLOCK_UNLOCKED;
-/* convert from cycles(64bits) => nanoseconds (64bits)
- * basic equation:
- * ns = cycles / (freq / ns_per_sec)
- * ns = cycles * (ns_per_sec / freq)
- * ns = cycles * (10^9 / (cpu_khz * 10^3))
- * ns = cycles * (10^6 / cpu_khz)
- *
- * Then we use scaling math (suggested by [email protected]) to get:
- * ns = cycles * (10^6 * SC / cpu_khz) / SC
- * ns = cycles * cyc2ns_scale / SC
- *
- * And since SC is a constant power of two, we can convert the div
- * into a shift.
- *
- * We can use khz divisor instead of mhz to keep a better percision, since
- * cyc2ns_scale is limited to 10^6 * 2^10, which fits in 32 bits.
- * ([email protected])
- *
- * [email protected] "math is hard, lets go shopping!"
- */
-static unsigned long cyc2ns_scale;
-#define CYC2NS_SCALE_FACTOR 10 /* 2^10, carefully chosen */
-
-static inline void set_cyc2ns_scale(unsigned long cpu_khz)
-{
- cyc2ns_scale = (1000000 << CYC2NS_SCALE_FACTOR)/cpu_khz;
-}
-
-static inline unsigned long long cycles_2_ns(unsigned long long cyc)
-{
- return (cyc * cyc2ns_scale) >> CYC2NS_SCALE_FACTOR;
-}
-
static int count2; /* counter for mark_offset_tsc() */
/* Cached *multiplier* to convert TSC counts to microseconds.
@@ -135,29 +98,6 @@
return base + cycles_2_ns(this_offset - last_offset);
}
-/*
- * Scheduler clock - returns current time in nanosec units.
- */
-unsigned long long sched_clock(void)
-{
- unsigned long long this_offset;
-
- /*
- * In the NUMA case we dont use the TSC as they are not
- * synchronized across all CPUs.
- */
-#ifndef CONFIG_NUMA
- if (!use_tsc)
-#endif
- /* no locking but a rare wrong value is not a big deal */
- return jiffies_64 * (1000000000 / HZ);
-
- /* Read the Time Stamp Counter */
- rdtscll(this_offset);
-
- /* return the value in ns */
- return cycles_2_ns(this_offset);
-}
static void delay_tsc(unsigned long loops)
{
@@ -222,127 +162,6 @@
#endif
-#ifdef CONFIG_CPU_FREQ
-#include <linux/workqueue.h>
-
-static unsigned int cpufreq_delayed_issched = 0;
-static unsigned int cpufreq_init = 0;
-static struct work_struct cpufreq_delayed_get_work;
-
-static void handle_cpufreq_delayed_get(void *v)
-{
- unsigned int cpu;
- for_each_online_cpu(cpu) {
- cpufreq_get(cpu);
- }
- cpufreq_delayed_issched = 0;
-}
-
-/* if we notice lost ticks, schedule a call to cpufreq_get() as it tries
- * to verify the CPU frequency the timing core thinks the CPU is running
- * at is still correct.
- */
-static inline void cpufreq_delayed_get(void)
-{
- if (cpufreq_init && !cpufreq_delayed_issched) {
- cpufreq_delayed_issched = 1;
- printk(KERN_DEBUG "Losing some ticks... checking if CPU frequency changed.\n");
- schedule_work(&cpufreq_delayed_get_work);
- }
-}
-
-/* If the CPU frequency is scaled, TSC-based delays will need a different
- * loops_per_jiffy value to function properly.
- */
-
-static unsigned int ref_freq = 0;
-static unsigned long loops_per_jiffy_ref = 0;
-
-#ifndef CONFIG_SMP
-static unsigned long fast_gettimeoffset_ref = 0;
-static unsigned int cpu_khz_ref = 0;
-#endif
-
-static int
-time_cpufreq_notifier(struct notifier_block *nb, unsigned long val,
- void *data)
-{
- struct cpufreq_freqs *freq = data;
-
- if (val != CPUFREQ_RESUMECHANGE)
- write_seqlock_irq(&xtime_lock);
- if (!ref_freq) {
- ref_freq = freq->old;
- loops_per_jiffy_ref = cpu_data[freq->cpu].loops_per_jiffy;
-#ifndef CONFIG_SMP
- fast_gettimeoffset_ref = fast_gettimeoffset_quotient;
- cpu_khz_ref = cpu_khz;
-#endif
- }
-
- if ((val == CPUFREQ_PRECHANGE && freq->old < freq->new) ||
- (val == CPUFREQ_POSTCHANGE && freq->old > freq->new) ||
- (val == CPUFREQ_RESUMECHANGE)) {
- if (!(freq->flags & CPUFREQ_CONST_LOOPS))
- cpu_data[freq->cpu].loops_per_jiffy = cpufreq_scale(loops_per_jiffy_ref, ref_freq, freq->new);
-#ifndef CONFIG_SMP
- if (cpu_khz)
- cpu_khz = cpufreq_scale(cpu_khz_ref, ref_freq, freq->new);
- if (use_tsc) {
- if (!(freq->flags & CPUFREQ_CONST_LOOPS)) {
- fast_gettimeoffset_quotient = cpufreq_scale(fast_gettimeoffset_ref, freq->new, ref_freq);
- set_cyc2ns_scale(cpu_khz);
- }
- }
-#endif
- }
-
- if (val != CPUFREQ_RESUMECHANGE)
- write_sequnlock_irq(&xtime_lock);
-
- return 0;
-}
-
-static struct notifier_block time_cpufreq_notifier_block = {
- .notifier_call = time_cpufreq_notifier
-};
-
-
-static int __init cpufreq_tsc(void)
-{
- int ret;
- INIT_WORK(&cpufreq_delayed_get_work, handle_cpufreq_delayed_get, NULL);
- ret = cpufreq_register_notifier(&time_cpufreq_notifier_block,
- CPUFREQ_TRANSITION_NOTIFIER);
- if (!ret)
- cpufreq_init = 1;
- return ret;
-}
-core_initcall(cpufreq_tsc);
-
-#else /* CONFIG_CPU_FREQ */
-static inline void cpufreq_delayed_get(void) { return; }
-#endif
-
-int recalibrate_cpu_khz(void)
-{
-#ifndef CONFIG_SMP
- unsigned int cpu_khz_old = cpu_khz;
-
- if (cpu_has_tsc) {
- init_cpu_khz();
- cpu_data[0].loops_per_jiffy =
- cpufreq_scale(cpu_data[0].loops_per_jiffy,
- cpu_khz_old,
- cpu_khz);
- return 0;
- } else
- return -ENODEV;
-#else
- return -ENODEV;
-#endif
-}
-EXPORT_SYMBOL(recalibrate_cpu_khz);
static void mark_offset_tsc(void)
{
@@ -548,37 +367,6 @@
return -ENODEV;
}
-static int tsc_resume(void)
-{
- write_seqlock(&monotonic_lock);
- /* Assume this is the last mark offset time */
- rdtsc(last_tsc_low, last_tsc_high);
-#ifdef CONFIG_HPET_TIMER
- if (is_hpet_enabled() && hpet_use_timer)
- hpet_last = hpet_readl(HPET_COUNTER);
-#endif
- write_sequnlock(&monotonic_lock);
- return 0;
-}
-
-#ifndef CONFIG_X86_TSC
-/* disable flag for tsc. Takes effect by clearing the TSC cpu flag
- * in cpu/common.c */
-static int __init tsc_setup(char *str)
-{
- tsc_disable = 1;
- return 1;
-}
-#else
-static int __init tsc_setup(char *str)
-{
- printk(KERN_WARNING "notsc: Kernel compiled with CONFIG_X86_TSC, "
- "cannot disable TSC.\n");
- return 1;
-}
-#endif
-__setup("notsc", tsc_setup);
-
/************************************************************/
diff -ruN tod-mm1/arch/i386/kernel/tsc.c tod-mm2/arch/i386/kernel/tsc.c
--- tod-mm1/arch/i386/kernel/tsc.c 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/arch/i386/kernel/tsc.c 2005-12-01 18:24:34.000000000 -0800
@@ -0,0 +1,312 @@
+/*
+ * This code largely moved from arch/i386/kernel/timer/timer_tsc.c
+ * which was originally moved from arch/i386/kernel/time.c.
+ * See comments there for proper credits.
+ */
+
+#include <linux/workqueue.h>
+#include <linux/cpufreq.h>
+#include <linux/init.h>
+
+#include <asm/io.h>
+
+#include "mach_timer.h"
+
+int tsc_disable __initdata = 0;
+#ifdef CONFIG_X86_TSC
+static int __init tsc_setup(char *str)
+{
+ printk(KERN_WARNING "notsc: Kernel compiled with CONFIG_X86_TSC, "
+ "cannot disable TSC.\n");
+ return 1;
+}
+#else
+/*
+ * disable flag for tsc. Takes effect by clearing the TSC cpu flag
+ * in cpu/common.c
+ */
+static int __init tsc_setup(char *str)
+{
+ tsc_disable = 1;
+
+ return 1;
+}
+#endif
+
+__setup("notsc", tsc_setup);
+
+
+int read_current_timer(unsigned long *timer_val)
+{
+ if (cur_timer->read_timer) {
+ *timer_val = cur_timer->read_timer();
+ return 0;
+ }
+ return -1;
+}
+
+
+/* convert from cycles(64bits) => nanoseconds (64bits)
+ * basic equation:
+ * ns = cycles / (freq / ns_per_sec)
+ * ns = cycles * (ns_per_sec / freq)
+ * ns = cycles * (10^9 / (cpu_khz * 10^3))
+ * ns = cycles * (10^6 / cpu_khz)
+ *
+ * Then we use scaling math (suggested by [email protected]) to get:
+ * ns = cycles * (10^6 * SC / cpu_khz) / SC
+ * ns = cycles * cyc2ns_scale / SC
+ *
+ * And since SC is a constant power of two, we can convert the div
+ * into a shift.
+ *
+ * We can use khz divisor instead of mhz to keep a better percision, since
+ * cyc2ns_scale is limited to 10^6 * 2^10, which fits in 32 bits.
+ * ([email protected])
+ *
+ * [email protected] "math is hard, lets go shopping!"
+ */
+static unsigned long cyc2ns_scale;
+
+#define CYC2NS_SCALE_FACTOR 10 /* 2^10, carefully chosen */
+
+static inline void set_cyc2ns_scale(unsigned long cpu_khz)
+{
+ cyc2ns_scale = (1000000 << CYC2NS_SCALE_FACTOR)/cpu_khz;
+}
+
+static inline unsigned long long cycles_2_ns(unsigned long long cyc)
+{
+ return (cyc * cyc2ns_scale) >> CYC2NS_SCALE_FACTOR;
+}
+
+/*
+ * Scheduler clock - returns current time in nanosec units.
+ */
+unsigned long long sched_clock(void)
+{
+ unsigned long long this_offset;
+
+ /*
+ * in the NUMA case we dont use the TSC as they are not
+ * synchronized across all CPUs.
+ */
+#ifndef CONFIG_NUMA
+ if (!use_tsc)
+#endif
+ /* no locking but a rare wrong value is not a big deal */
+ return jiffies_64 * (1000000000 / HZ);
+
+ /* read the Time Stamp Counter: */
+ rdtscll(this_offset);
+
+ /* return the value in ns */
+ return cycles_2_ns(this_offset);
+}
+
+/* ------ Calibrate the TSC -------
+ * Return 2^32 * (1 / (TSC clocks per usec)) for do_fast_gettimeoffset().
+ * Too much 64-bit arithmetic here to do this cleanly in C, and for
+ * accuracy's sake we want to keep the overhead on the CTC speaker (channel 2)
+ * output busy loop as low as possible. We avoid reading the CTC registers
+ * directly because of the awkward 8-bit access mechanism of the 82C54
+ * device.
+ */
+
+#define CALIBRATE_TIME (5 * 1000020/HZ)
+
+unsigned long calibrate_tsc(void)
+{
+ mach_prepare_counter();
+
+ {
+ unsigned long startlow, starthigh;
+ unsigned long endlow, endhigh;
+ unsigned long count;
+
+ rdtsc(startlow,starthigh);
+ mach_countup(&count);
+ rdtsc(endlow,endhigh);
+
+
+ /* Error: ECTCNEVERSET */
+ if (count <= 1)
+ goto bad_ctc;
+
+ /* 64-bit subtract - gcc just messes up with long longs */
+ __asm__("subl %2,%0\n\t"
+ "sbbl %3,%1"
+ :"=a" (endlow), "=d" (endhigh)
+ :"g" (startlow), "g" (starthigh),
+ "0" (endlow), "1" (endhigh));
+
+ /* Error: ECPUTOOFAST */
+ if (endhigh)
+ goto bad_ctc;
+
+ /* Error: ECPUTOOSLOW */
+ if (endlow <= CALIBRATE_TIME)
+ goto bad_ctc;
+
+ __asm__("divl %2"
+ :"=a" (endlow), "=d" (endhigh)
+ :"r" (endlow), "0" (0), "1" (CALIBRATE_TIME));
+
+ return endlow;
+ }
+
+ /*
+ * The CTC wasn't reliable: we got a hit on the very first read,
+ * or the CPU was so fast/slow that the quotient wouldn't fit in
+ * 32 bits..
+ */
+bad_ctc:
+ return 0;
+}
+
+int recalibrate_cpu_khz(void)
+{
+#ifndef CONFIG_SMP
+ unsigned long cpu_khz_old = cpu_khz;
+
+ if (cpu_has_tsc) {
+ init_cpu_khz();
+ cpu_data[0].loops_per_jiffy =
+ cpufreq_scale(cpu_data[0].loops_per_jiffy,
+ cpu_khz_old,
+ cpu_khz);
+ return 0;
+ } else
+ return -ENODEV;
+#else
+ return -ENODEV;
+#endif
+}
+EXPORT_SYMBOL(recalibrate_cpu_khz);
+
+
+/* calculate cpu_khz */
+void init_cpu_khz(void)
+{
+ if (cpu_has_tsc) {
+ unsigned long tsc_quotient = calibrate_tsc();
+ if (tsc_quotient) {
+ /* report CPU clock rate in Hz.
+ * The formula is (10^6 * 2^32) / (2^32 * 1 / (clocks/us)) =
+ * clock/second. Our precision is about 100 ppm.
+ */
+ { unsigned long eax=0, edx=1000;
+ __asm__("divl %2"
+ :"=a" (cpu_khz), "=d" (edx)
+ :"r" (tsc_quotient),
+ "0" (eax), "1" (edx));
+ printk("Detected %lu.%03lu MHz processor.\n", cpu_khz / 1000, cpu_khz % 1000);
+ }
+ }
+ }
+}
+
+#ifdef CONFIG_CPU_FREQ
+
+static unsigned int cpufreq_delayed_issched = 0;
+static unsigned int cpufreq_init = 0;
+static struct work_struct cpufreq_delayed_get_work;
+
+static void handle_cpufreq_delayed_get(void *v)
+{
+ unsigned int cpu;
+
+ for_each_online_cpu(cpu)
+ cpufreq_get(cpu);
+
+ cpufreq_delayed_issched = 0;
+}
+
+/*
+ * if we notice lost ticks, schedule a call to cpufreq_get() as it tries
+ * to verify the CPU frequency the timing core thinks the CPU is running
+ * at is still correct.
+ */
+void cpufreq_delayed_get(void)
+{
+ if (cpufreq_init && !cpufreq_delayed_issched) {
+ cpufreq_delayed_issched = 1;
+ printk(KERN_DEBUG "Losing some ticks... checking if CPU frequency changed.\n");
+ schedule_work(&cpufreq_delayed_get_work);
+ }
+}
+
+/*
+ * if the CPU frequency is scaled, TSC-based delays will need a different
+ * loops_per_jiffy value to function properly.
+ */
+
+static unsigned int ref_freq = 0;
+static unsigned long loops_per_jiffy_ref = 0;
+
+#ifndef CONFIG_SMP
+static unsigned long fast_gettimeoffset_ref = 0;
+static unsigned long cpu_khz_ref = 0;
+#endif
+
+static int
+time_cpufreq_notifier(struct notifier_block *nb, unsigned long val, void *data)
+{
+ struct cpufreq_freqs *freq = data;
+
+ if (val != CPUFREQ_RESUMECHANGE)
+ write_seqlock_irq(&xtime_lock);
+
+ if (!ref_freq) {
+ ref_freq = freq->old;
+ loops_per_jiffy_ref = cpu_data[freq->cpu].loops_per_jiffy;
+#ifndef CONFIG_SMP
+ fast_gettimeoffset_ref = fast_gettimeoffset_quotient;
+ cpu_khz_ref = cpu_khz;
+#endif
+ }
+
+ if ((val == CPUFREQ_PRECHANGE && freq->old < freq->new) ||
+ (val == CPUFREQ_POSTCHANGE && freq->old > freq->new) ||
+ (val == CPUFREQ_RESUMECHANGE)) {
+ if (!(freq->flags & CPUFREQ_CONST_LOOPS))
+ cpu_data[freq->cpu].loops_per_jiffy = cpufreq_scale(loops_per_jiffy_ref, ref_freq, freq->new);
+#ifndef CONFIG_SMP
+ if (cpu_khz)
+ cpu_khz = cpufreq_scale(cpu_khz_ref, ref_freq, freq->new);
+ if (use_tsc) {
+ if (!(freq->flags & CPUFREQ_CONST_LOOPS)) {
+ fast_gettimeoffset_quotient = cpufreq_scale(fast_gettimeoffset_ref, freq->new, ref_freq);
+ set_cyc2ns_scale(cpu_khz);
+ }
+ }
+#endif
+ }
+
+ if (val != CPUFREQ_RESUMECHANGE)
+ write_sequnlock_irq(&xtime_lock);
+
+ return 0;
+}
+
+static struct notifier_block time_cpufreq_notifier_block = {
+ .notifier_call = time_cpufreq_notifier
+};
+
+static int __init cpufreq_tsc(void)
+{
+ int ret;
+
+ INIT_WORK(&cpufreq_delayed_get_work, handle_cpufreq_delayed_get, NULL);
+ ret = cpufreq_register_notifier(&time_cpufreq_notifier_block,
+ CPUFREQ_TRANSITION_NOTIFIER);
+ if (!ret)
+ cpufreq_init = 1;
+ return ret;
+}
+
+core_initcall(cpufreq_tsc);
+
+#else /* CONFIG_CPU_FREQ */
+void cpufreq_delayed_get(void) { return; }
+#endif
diff -ruN tod-mm1/include/asm-i386/timex.h tod-mm2/include/asm-i386/timex.h
--- tod-mm1/include/asm-i386/timex.h 2005-12-01 18:13:31.000000000 -0800
+++ tod-mm2/include/asm-i386/timex.h 2005-12-01 18:24:34.000000000 -0800
@@ -8,6 +8,7 @@
#include <linux/config.h>
#include <asm/processor.h>
+#include <asm/tsc.h>
#ifdef CONFIG_X86_ELAN
# define CLOCK_TICK_RATE 1189200 /* AMD Elan has different frequency! */
@@ -16,39 +17,6 @@
#endif
-/*
- * Standard way to access the cycle counter on i586+ CPUs.
- * Currently only used on SMP.
- *
- * If you really have a SMP machine with i486 chips or older,
- * compile for that, and this will just always return zero.
- * That's ok, it just means that the nicer scheduling heuristics
- * won't work for you.
- *
- * We only use the low 32 bits, and we'd simply better make sure
- * that we reschedule before that wraps. Scheduling at least every
- * four billion cycles just basically sounds like a good idea,
- * regardless of how fast the machine is.
- */
-typedef unsigned long long cycles_t;
-
-static inline cycles_t get_cycles (void)
-{
- unsigned long long ret=0;
-
-#ifndef CONFIG_X86_TSC
- if (!cpu_has_tsc)
- return 0;
-#endif
-
-#if defined(CONFIG_X86_GENERIC) || defined(CONFIG_X86_TSC)
- rdtscll(ret);
-#endif
- return ret;
-}
-
-extern unsigned int cpu_khz;
-
extern int read_current_timer(unsigned long *timer_value);
#define ARCH_HAS_READ_CURRENT_TIMER 1
diff -ruN tod-mm1/include/asm-i386/tsc.h tod-mm2/include/asm-i386/tsc.h
--- tod-mm1/include/asm-i386/tsc.h 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/include/asm-i386/tsc.h 2005-12-01 18:24:34.000000000 -0800
@@ -0,0 +1,44 @@
+/*
+ * linux/include/asm-i386/tsc.h
+ *
+ * i386 TSC related functions
+ */
+#ifndef _ASM_i386_TSC_H
+#define _ASM_i386_TSC_H
+
+#include <linux/config.h>
+#include <asm/processor.h>
+
+/*
+ * Standard way to access the cycle counter on i586+ CPUs.
+ * Currently only used on SMP.
+ *
+ * If you really have a SMP machine with i486 chips or older,
+ * compile for that, and this will just always return zero.
+ * That's ok, it just means that the nicer scheduling heuristics
+ * won't work for you.
+ *
+ * We only use the low 32 bits, and we'd simply better make sure
+ * that we reschedule before that wraps. Scheduling at least every
+ * four billion cycles just basically sounds like a good idea,
+ * regardless of how fast the machine is.
+ */
+typedef unsigned long long cycles_t;
+
+static inline cycles_t get_cycles (void)
+{
+ unsigned long long ret=0;
+
+#ifndef CONFIG_X86_TSC
+ if (!cpu_has_tsc)
+ return 0;
+#endif
+
+#if defined(CONFIG_X86_GENERIC) || defined(CONFIG_X86_TSC)
+ rdtscll(ret);
+#endif
+ return ret;
+}
+
+extern unsigned int cpu_khz;
+#endif
All,
The conversion of i386 to use the generic timeofday subsystem has been
split into 6 parts. This patch, the fourth of six, renames some ACPI PM
variables.
It applies on top of my timeofday-arch-i386-part3 patch. This patch is
part the timeofday-arch-i386 patchset, so without the following parts it
is not expected to compile.
thanks
-john
Signed-off-by: John Stultz <[email protected]>
boot.c | 18 ++++++++++--------
1 files changed, 10 insertions(+), 8 deletions(-)
linux-2.6.15-rc3-mm1_timeofday-arch-i386-part4_B12.patch
==========================
diff -ruN tod-mm1/arch/i386/kernel/acpi/boot.c tod-mm2/arch/i386/kernel/acpi/boot.c
--- tod-mm1/arch/i386/kernel/acpi/boot.c 2005-12-01 18:12:56.000000000 -0800
+++ tod-mm2/arch/i386/kernel/acpi/boot.c 2005-12-01 18:25:30.000000000 -0800
@@ -609,7 +609,8 @@
#endif
#ifdef CONFIG_X86_PM_TIMER
-extern u32 pmtmr_ioport;
+u32 acpi_pmtmr_ioport;
+int acpi_pmtmr_buggy;
#endif
static int __init acpi_parse_fadt(unsigned long phys, unsigned long size)
@@ -637,21 +638,22 @@
ACPI_ADR_SPACE_SYSTEM_IO)
return 0;
- pmtmr_ioport = fadt->xpm_tmr_blk.address;
+ acpi_pmtmr_ioport = fadt->xpm_tmr_blk.address;
/*
* "X" fields are optional extensions to the original V1.0
* fields, so we must selectively expand V1.0 fields if the
* corresponding X field is zero.
*/
- if (!pmtmr_ioport)
- pmtmr_ioport = fadt->V1_pm_tmr_blk;
+ if (!acpi_pmtmr_ioport)
+ acpi_pmtmr_ioport = fadt->V1_pm_tmr_blk;
} else {
/* FADT rev. 1 */
- pmtmr_ioport = fadt->V1_pm_tmr_blk;
+ acpi_pmtmr_ioport = fadt->V1_pm_tmr_blk;
}
- if (pmtmr_ioport)
- printk(KERN_INFO PREFIX "PM-Timer IO Port: %#x\n",
- pmtmr_ioport);
+
+ if (acpi_pmtmr_ioport)
+ printk(KERN_INFO PREFIX "PM-Timer IO Port: %#x\n", acpi_pmtmr_ioport);
+
#endif
return 0;
}
All,
This patch provides paranoid checking of the timekeeping code. It is
not intended to be submitted to mainline, but to allow developers and
testers using the timeofday patches to better find or rule-out potential
timekeeping problems.
thanks
-john
Signed-off-by: John Stultz <[email protected]>
arch/i386/kernel/tsc.c | 62 +++++++++++++++++++++++
arch/x86_64/kernel/time.c | 47 +++++++++++++++++
include/asm-generic/bug.h | 10 +++
kernel/time/timeofday.c | 124 +++++++++++++++++++++++++++++++++++++++++++++-
lib/Kconfig.debug | 5 +
5 files changed, 245 insertions(+), 3 deletions(-)
linux-2.6.15-rc3-mm1_timeofday-paranoid-debug_B12.patch
==========================
diff -ruN tod-mm1/arch/i386/kernel/tsc.c tod-mm2/arch/i386/kernel/tsc.c
--- tod-mm1/arch/i386/kernel/tsc.c 2005-12-01 18:27:29.000000000 -0800
+++ tod-mm2/arch/i386/kernel/tsc.c 2005-12-01 18:28:12.000000000 -0800
@@ -328,6 +328,67 @@
static unsigned long current_tsc_khz = 0;
static int tsc_update_callback(void);
+#ifdef CONFIG_PARANOID_GENERIC_TIME
+/* This will hurt performance! */
+static DEFINE_SPINLOCK(checktsc_lock);
+static cycle_t last_tsc;
+
+static cycle_t read_tsc(void)
+{
+ static int once = 1;
+
+ unsigned long flags;
+ cycle_t ret;
+
+ spin_lock_irqsave(&checktsc_lock, flags);
+
+ rdtscll(ret);
+
+ if (once && ret < last_tsc) {
+ once = 0;
+ spin_unlock_irqrestore(&checktsc_lock, flags);
+ printk("BUG in read_tsc(): TSC went backward!\n");
+ if (num_online_cpus() > 1)
+ printk("... Unsynced TSCs?\n");
+ printk("... [ from %016Lx to %016Lx ]\n", last_tsc, ret);
+
+ } else {
+ last_tsc = ret;
+ spin_unlock_irqrestore(&checktsc_lock, flags);
+ }
+
+ return ret;
+}
+
+static cycle_t read_tsc_c3(void)
+{
+ static int once = 1;
+
+ unsigned long flags;
+ cycle_t ret;
+
+ spin_lock_irqsave(&checktsc_lock, flags);
+
+ rdtscll(ret);
+ ret += tsc_read_c3_time();
+
+ if (once && ret < last_tsc) {
+ once = 0;
+ spin_unlock_irqrestore(&checktsc_lock, flags);
+ printk("BUG in read_tsc_c3(): TSC went backward!\n");
+ if (num_online_cpus() > 1)
+ printk("... Unsynced TSCs?\n");
+ printk("... [ from %016Lx to %016Lx ]\n", last_tsc, ret);
+ } else {
+ last_tsc = ret;
+ spin_unlock_irqrestore(&checktsc_lock, flags);
+ }
+
+ return ret;
+}
+
+#else /* CONFIG_PARANOID_GENERIC_TIME */
+
static cycle_t read_tsc(void)
{
cycle_t ret;
@@ -346,6 +407,7 @@
return ret + tsc_read_c3_time();
}
+#endif /* CONFIG_PARANOID_GENERIC_TIME */
static struct clocksource clocksource_tsc = {
.name = "tsc",
diff -ruN tod-mm1/arch/x86_64/kernel/time.c tod-mm2/arch/x86_64/kernel/time.c
--- tod-mm1/arch/x86_64/kernel/time.c 2005-12-01 18:28:04.000000000 -0800
+++ tod-mm2/arch/x86_64/kernel/time.c 2005-12-01 18:28:12.000000000 -0800
@@ -1054,16 +1054,50 @@
static int tsc_update_callback(void);
+#ifdef CONFIG_PARANOID_GENERIC_TIME
+/* This will hurt performance! */
+static DEFINE_SPINLOCK(checktsc_lock);
+static cycle_t last_tsc;
+
static cycle_t read_tsc(void)
{
+ unsigned long flags;
cycle_t ret;
+ spin_lock_irqsave(&checktsc_lock, flags);
+
rdtscll(ret);
+ if (ret < last_tsc)
+ printk("read_tsc: ACK! TSC went backward! Unsynced TSCs?\n");
+ last_tsc = ret;
+
+ spin_unlock_irqrestore(&checktsc_lock, flags);
return ret;
}
-static cycle_t __vsyscall_fn vread_tsc(void* unused)
+static cycle_t read_tsc_c3(void)
+{
+ unsigned long flags;
+ cycle_t ret;
+
+ spin_lock_irqsave(&checktsc_lock, flags);
+
+ rdtscll(ret);
+ ret += tsc_read_c3_time();
+
+ if (ret < last_tsc)
+ printk("read_tsc_c3: ACK! TSC went backward! Unsynced TSCs?\n");
+ last_tsc = ret;
+
+ spin_unlock_irqrestore(&checktsc_lock, flags);
+
+ return ret;
+}
+
+#else /* CONFIG_PARANOID_GENERIC_TIME */
+
+static cycle_t read_tsc(void)
{
cycle_t ret;
@@ -1081,6 +1115,17 @@
return ret + tsc_read_c3_time();
}
+#endif /* CONFIG_PARANOID_GENERIC_TIME */
+
+static cycle_t __vsyscall_fn vread_tsc(void* unused)
+{
+ cycle_t ret;
+
+ rdtscll(ret);
+
+ return ret;
+}
+
static struct clocksource clocksource_tsc = {
.name = "tsc",
.rating = 300,
diff -ruN tod-mm1/include/asm-generic/bug.h tod-mm2/include/asm-generic/bug.h
--- tod-mm1/include/asm-generic/bug.h 2005-12-01 18:13:31.000000000 -0800
+++ tod-mm2/include/asm-generic/bug.h 2005-12-01 18:28:12.000000000 -0800
@@ -39,4 +39,14 @@
#endif
#endif
+#define WARN_ON_ONCE(condition) \
+do { \
+ static int warn_once = 1; \
+ \
+ if (condition) { \
+ warn_once = 0; \
+ WARN_ON(1); \
+ } \
+} while (0);
+
#endif
diff -ruN tod-mm1/kernel/time/timeofday.c tod-mm2/kernel/time/timeofday.c
--- tod-mm1/kernel/time/timeofday.c 2005-12-01 18:23:10.000000000 -0800
+++ tod-mm2/kernel/time/timeofday.c 2005-12-01 18:28:12.000000000 -0800
@@ -121,6 +121,109 @@
*/
static struct ktimer timeofday_timer;
+
+#ifdef CONFIG_PARANOID_GENERIC_TIME
+/* This will hurt performance! */
+static DEFINE_SPINLOCK(check_monotonic_lock);
+static ktime_t last_monotonic_ktime;
+
+static ktime_t get_check_value(void)
+{
+ unsigned long flags;
+ ktime_t ret;
+
+ spin_lock_irqsave(&check_monotonic_lock, flags);
+ ret = last_monotonic_ktime;
+ spin_unlock_irqrestore(&check_monotonic_lock, flags);
+
+ return ret;
+}
+
+static void check_monotonic_clock(ktime_t prev, ktime_t now)
+{
+ unsigned long flags;
+
+ /* check for monotonic inconsistencies */
+ if (ktime_cmp(now, <, prev)) {
+ static int warn = 1;
+
+ if (warn) {
+ warn = 0;
+
+ printk("check_monotonic_clock: monotonic inconsistency"
+ " detected!\n");
+ printk(" from %16Lx (%llu) to %16Lx (%llu).\n",
+ ktime_to_ns(prev),
+ ktime_to_ns(prev),
+ ktime_to_ns(now),
+ ktime_to_ns(now));
+ WARN_ON(1);
+ }
+ }
+ spin_lock_irqsave(&check_monotonic_lock, flags);
+ last_monotonic_ktime = now;
+ spin_unlock_irqrestore(&check_monotonic_lock, flags);
+}
+
+/* timespec version */
+#define check_monotonic_clock_ts(prev, now) \
+ check_monotonic_clock(prev, timespec_to_ktime(now))
+
+/* Call holding atleast a readlock on system_time_lock */
+void verify_timekeeping_state(void)
+{
+ /* ensure all the timespec and ktime values are consistent: */
+ WARN_ON_ONCE(ktime_cmp(system_time, !=,
+ timespec_to_ktime(mono_time_ts)));
+ WARN_ON_ONCE(ktime_cmp(ktime_add(system_time, wall_time_offset), !=,
+ timespec_to_ktime(wall_time_ts)));
+ WARN_ON_ONCE(ktime_cmp(wall_time_offset, !=,
+ timespec_to_ktime(monotonic_time_offset_ts)));
+}
+
+static void check_periodic_interval(cycle_t now)
+{
+ static cycle_t last;
+
+ cycle_t delta;
+ nsec_t ns_offset;
+
+ if (last != 0 && now != 0) {
+ delta = (now - last)& clock->mask;
+
+ ns_offset = cyc2ns(clock, ntp_adj, delta);
+
+ if (ns_offset > (nsec_t)2*PERIODIC_INTERVAL_MS *1000000) {
+ static int warn_count = 10;
+ if (warn_count > 0) {
+ warn_count--;
+ printk("check_periodic_interval: Long interval! %llu.\n",
+ ns_offset);
+ printk(" Something may be blocking interrupts.\n");
+ }
+ }
+ if (ns_offset < (nsec_t)PERIODIC_INTERVAL_MS *1000000) {
+ static int warn_count = 10;
+ if (warn_count > 0) {
+ warn_count--;
+ printk("check_periodic_interval: short interval! %llu.\n",
+ ns_offset);
+ printk(" bad calibration or ktimers may be broken.\n");
+ }
+ }
+ }
+ last = now;
+}
+
+#else /* CONFIG_PARANOID_GENERIC_TIME */
+ /* XXX can we optimize this out? */
+# define get_check_value(void) ktime_set(0,0)
+# define check_monotonic_clock(x,y) do { } while (0)
+# define check_monotonic_clock_ts(x,ts) do { } while (0)
+# define verify_timekeeping_state() do { } while (0)
+# define check_periodic_interval(x) do { } while (0)
+#endif /* CONFIG_PARANOID_GENERIC_TIME */
+
/**
* update_legacy_time_values - sync legacy time values
*
@@ -187,7 +290,14 @@
static inline ktime_t __get_monotonic_clock(void)
{
nsec_t offset = __get_nsec_offset();
- return ktime_add_ns(system_time, offset);
+#ifdef CONFIG_PARANOID_GENERIC_TIME
+ ktime_t check = get_check_value();
+#endif
+ ktime_t ret;
+
+ ret = ktime_add_ns(system_time, offset);
+ check_monotonic_clock(check,ret);
+ return ret;
}
/**
@@ -268,6 +378,9 @@
*/
void get_monotonic_clock_ts(struct timespec *ts)
{
+#ifdef CONFIG_PARANOID_GENERIC_TIME
+ ktime_t check = get_check_value();
+#endif
unsigned long seq;
nsec_t offset;
@@ -279,6 +392,7 @@
} while (read_seqretry(&system_time_lock, seq));
timespec_add_ns(ts, offset);
+ check_monotonic_clock_ts(check, *ts);
}
/**
@@ -361,7 +475,9 @@
update_legacy_time_values();
write_sequnlock_irqrestore(&system_time_lock, flags);
-
+#ifdef CONFIG_PARANOID_GENERIC_TIME
+ printk("do_settimeofday() was called!\n");
+#endif
/* signal ktimers about time change */
clock_was_set();
@@ -518,6 +634,7 @@
/* read time source & calc time since last call: */
cycle_now = read_clocksource(clock);
+ check_periodic_interval(cycle_now);
cycle_delta = (cycle_now - cycle_last) & clock->mask;
delta_nsec = cyc2ns_fixed_rem(ts_interval, &cycle_delta, &remainder);
@@ -562,6 +679,7 @@
ntp_adj = 0;
remainder = 0;
something_changed = 1;
+ check_periodic_interval(0);
}
/*
@@ -611,6 +729,8 @@
update_legacy_time_values();
+ verify_timekeeping_state();
+
write_sequnlock_irqrestore(&system_time_lock, flags);
/* set us up to go off on the next interval: */
diff -ruN tod-mm1/lib/Kconfig.debug tod-mm2/lib/Kconfig.debug
--- tod-mm1/lib/Kconfig.debug 2005-12-01 18:13:40.000000000 -0800
+++ tod-mm2/lib/Kconfig.debug 2005-12-01 18:28:12.000000000 -0800
@@ -46,6 +46,11 @@
13 => 8 KB
12 => 4 KB
+config PARANOID_GENERIC_TIME
+ default y
+ depends on GENERIC_TIME
+ bool "Paraniod Timekeeping Checks"
+
config DETECT_SOFTLOCKUP
bool "Detect Soft Lockups"
depends on DEBUG_KERNEL
All,
This patch implements the time sources shared between i386 and x86-64
(acpi_pm, cyclone, hpet, pit, tsc and tsc-interp). The patch should
apply on top of the timeofday-arch-i386-part4 patch as well as the
arch-x86-64 patch
The patch should be fairly straight forward, only adding the new
clocksources.
thanks
-john
Signed-off-by: John Stultz <[email protected]>
arch/i386/kernel/Makefile | 1
arch/i386/kernel/hpet.c | 69 ++++++++++++++++++
arch/i386/kernel/i8253.c | 60 ++++++++++++++++
arch/i386/kernel/tsc.c | 102 +++++++++++++++++++++++++++
drivers/Makefile | 1
drivers/clocksource/Makefile | 2
drivers/clocksource/acpi_pm.c | 154 ++++++++++++++++++++++++++++++++++++++++++
drivers/clocksource/cyclone.c | 121 +++++++++++++++++++++++++++++++++
8 files changed, 509 insertions(+), 1 deletion(-)
linux-2.6.15-rc3-mm1_timeofday-clocks-i386_B12.patch
==========================
diff -ruN tod-mm1/arch/i386/kernel/hpet.c tod-mm2/arch/i386/kernel/hpet.c
--- tod-mm1/arch/i386/kernel/hpet.c 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/arch/i386/kernel/hpet.c 2005-12-01 18:26:50.000000000 -0800
@@ -0,0 +1,69 @@
+#include <linux/clocksource.h>
+#include <linux/errno.h>
+#include <linux/hpet.h>
+#include <linux/init.h>
+
+#include <asm/hpet.h>
+#include <asm/io.h>
+
+#define HPET_MASK 0xFFFFFFFF
+#define HPET_SHIFT 22
+
+/* FSEC = 10^-15 NSEC = 10^-9 */
+#define FSEC_PER_NSEC 1000000
+
+static void *hpet_ptr;
+
+static cycle_t read_hpet(void)
+{
+ return (cycle_t)readl(hpet_ptr);
+}
+
+struct clocksource clocksource_hpet = {
+ .name = "hpet",
+ .rating = 250,
+ .read = read_hpet,
+ .mask = (cycle_t)HPET_MASK,
+ .mult = 0, /* set below */
+ .shift = HPET_SHIFT,
+ .is_continuous = 1,
+};
+
+static int __init init_hpet_clocksource(void)
+{
+ unsigned long hpet_period;
+ void __iomem* hpet_base;
+ u64 tmp;
+
+ if (!hpet_address)
+ return -ENODEV;
+
+ /* calculate the hpet address: */
+ hpet_base =
+ (void __iomem*)ioremap_nocache(hpet_address, HPET_MMAP_SIZE);
+ hpet_ptr = hpet_base + HPET_COUNTER;
+
+ /* calculate the frequency: */
+ hpet_period = readl(hpet_base + HPET_PERIOD);
+
+ /*
+ * hpet period is in femto seconds per cycle
+ * so we need to convert this to ns/cyc units
+ * aproximated by mult/2^shift
+ *
+ * fsec/cyc * 1nsec/1000000fsec = nsec/cyc = mult/2^shift
+ * fsec/cyc * 1ns/1000000fsec * 2^shift = mult
+ * fsec/cyc * 2^shift * 1nsec/1000000fsec = mult
+ * (fsec/cyc << shift)/1000000 = mult
+ * (hpet_period << shift)/FSEC_PER_NSEC = mult
+ */
+ tmp = (u64)hpet_period << HPET_SHIFT;
+ do_div(tmp, FSEC_PER_NSEC);
+ clocksource_hpet.mult = (u32)tmp;
+
+ register_clocksource(&clocksource_hpet);
+
+ return 0;
+}
+
+module_init(init_hpet_clocksource);
diff -ruN tod-mm1/arch/i386/kernel/i8253.c tod-mm2/arch/i386/kernel/i8253.c
--- tod-mm1/arch/i386/kernel/i8253.c 2005-12-01 18:24:31.000000000 -0800
+++ tod-mm2/arch/i386/kernel/i8253.c 2005-12-01 18:26:50.000000000 -0800
@@ -2,6 +2,7 @@
* i8253.c 8253/PIT functions
*
*/
+#include <linux/clocksource.h>
#include <linux/spinlock.h>
#include <linux/jiffies.h>
#include <linux/sysdev.h>
@@ -57,3 +58,62 @@
}
device_initcall(init_timer_sysfs);
+
+
+/*
+ * Since the PIT overflows every tick, its not very useful
+ * to just read by itself. So use jiffies to emulate a free
+ * running counter:
+ */
+static cycle_t pit_read(void)
+{
+ unsigned long flags, seq;
+ int count;
+ u64 jifs;
+
+ do {
+ seq = read_seqbegin(&xtime_lock);
+
+ spin_lock_irqsave(&i8253_lock, flags);
+ outb_p(0x00, PIT_MODE); /* latch the count ASAP */
+ count = inb_p(PIT_CH0); /* read the latched count */
+ count |= inb_p(PIT_CH0) << 8;
+
+ /* VIA686a test code... reset the latch if count > max + 1 */
+ if (count > LATCH) {
+ outb_p(0x34, PIT_MODE);
+ outb_p(LATCH & 0xff, PIT_CH0);
+ outb(LATCH >> 8, PIT_CH0);
+ count = LATCH - 1;
+ }
+ spin_unlock_irqrestore(&i8253_lock, flags);
+
+ jifs = jiffies_64;
+ } while (read_seqretry(&xtime_lock, seq));
+
+ jifs -= INITIAL_JIFFIES;
+ count = (LATCH-1) - count;
+
+ return (cycle_t)(jifs * LATCH) + count;
+}
+
+static struct clocksource clocksource_pit = {
+ .name = "pit",
+ .rating = 110,
+ .read = pit_read,
+ .mask = (cycle_t)-1,
+ .mult = 0,
+ .shift = 20,
+};
+
+static int __init init_pit_clocksource(void)
+{
+ if (num_possible_cpus() > 4) /* PIT does not scale! */
+ return 0;
+
+ clocksource_pit.mult = clocksource_hz2mult(CLOCK_TICK_RATE, 20);
+ register_clocksource(&clocksource_pit);
+
+ return 0;
+}
+module_init(init_pit_clocksource);
diff -ruN tod-mm1/arch/i386/kernel/Makefile tod-mm2/arch/i386/kernel/Makefile
--- tod-mm1/arch/i386/kernel/Makefile 2005-12-01 18:26:05.000000000 -0800
+++ tod-mm2/arch/i386/kernel/Makefile 2005-12-01 18:26:50.000000000 -0800
@@ -36,6 +36,7 @@
obj-$(CONFIG_EFI) += efi.o efi_stub.o
obj-$(CONFIG_DOUBLEFAULT) += doublefault.o
obj-$(CONFIG_EARLY_PRINTK) += early_printk.o
+obj-$(CONFIG_HPET_TIMER) += hpet.o
EXTRA_AFLAGS := -traditional
diff -ruN tod-mm1/arch/i386/kernel/tsc.c tod-mm2/arch/i386/kernel/tsc.c
--- tod-mm1/arch/i386/kernel/tsc.c 2005-12-01 18:26:05.000000000 -0800
+++ tod-mm2/arch/i386/kernel/tsc.c 2005-12-01 18:26:50.000000000 -0800
@@ -4,13 +4,14 @@
* See comments there for proper credits.
*/
+#include <linux/clocksource.h>
#include <linux/workqueue.h>
#include <linux/cpufreq.h>
#include <linux/jiffies.h>
#include <linux/init.h>
-#include <asm/tsc.h>
#include <asm/delay.h>
+#include <asm/tsc.h>
#include <asm/io.h>
#include "mach_timer.h"
@@ -321,3 +322,102 @@
core_initcall(cpufreq_tsc);
#endif
+
+/* clock source code */
+
+static unsigned long current_tsc_khz = 0;
+static int tsc_update_callback(void);
+
+static cycle_t read_tsc(void)
+{
+ cycle_t ret;
+
+ rdtscll(ret);
+
+ return ret;
+}
+
+static cycle_t read_tsc_c3(void)
+{
+ cycle_t ret;
+
+ rdtscll(ret);
+
+ return ret + tsc_read_c3_time();
+}
+
+
+static struct clocksource clocksource_tsc = {
+ .name = "tsc",
+ .rating = 300,
+ .read = read_tsc,
+ .mask = (cycle_t)-1,
+ .mult = 0, /* to be set */
+ .shift = 22,
+ .update_callback = tsc_update_callback,
+ .is_continuous = 1,
+};
+
+static int tsc_update_callback(void)
+{
+ int change = 0;
+
+ /* check to see if we should switch to the safe clocksource: */
+ if (tsc_read_c3_time() && strncmp(clocksource_tsc.name, "c3tsc", 5)) {
+ printk("Falling back to C3 safe TSC\n");
+ clocksource_tsc.read = read_tsc_c3;
+ clocksource_tsc.name = "c3tsc";
+ change = 1;
+ }
+
+ if (clocksource_tsc.rating != 50 && check_tsc_unstable()) {
+ clocksource_tsc.rating = 50;
+ reselect_clocksource();
+ change = 1;
+ }
+
+ /* only update if tsc_khz has changed: */
+ if (current_tsc_khz != tsc_khz) {
+ current_tsc_khz = tsc_khz;
+ clocksource_tsc.mult = clocksource_khz2mult(current_tsc_khz,
+ clocksource_tsc.shift);
+ change = 1;
+ }
+
+ return change;
+}
+
+/*
+ * Make an educated guess if the TSC is trustworthy and synchronized
+ * over all CPUs.
+ */
+static __init int unsynchronized_tsc(void)
+{
+ /*
+ * Intel systems are normally all synchronized.
+ * Exceptions must mark TSC as unstable:
+ */
+ if (boot_cpu_data.x86_vendor == X86_VENDOR_INTEL)
+ return 0;
+
+ /* assume multi socket systems are not synchronized: */
+ return num_possible_cpus() > 1;
+}
+
+/* NUMAQ can't use TSC: */
+static int __init init_tsc_clocksource(void)
+{
+ /* TSC initialization is done in arch/i386/kernel/tsc.c */
+ if (cpu_has_tsc && tsc_khz && !tsc_disable) {
+ if (unsynchronized_tsc()) /* lower rating if unsynced */
+ clocksource_tsc.rating = 150;
+ current_tsc_khz = tsc_khz;
+ clocksource_tsc.mult = clocksource_khz2mult(current_tsc_khz,
+ clocksource_tsc.shift);
+ register_clocksource(&clocksource_tsc);
+ }
+
+ return 0;
+}
+
+module_init(init_tsc_clocksource);
diff -ruN tod-mm1/drivers/clocksource/acpi_pm.c tod-mm2/drivers/clocksource/acpi_pm.c
--- tod-mm1/drivers/clocksource/acpi_pm.c 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/drivers/clocksource/acpi_pm.c 2005-12-01 18:26:50.000000000 -0800
@@ -0,0 +1,154 @@
+/*
+ * linux/drivers/clocksource/acpi_pm.c
+ *
+ * This file contains the ACPI PM based clocksource.
+ *
+ * This code was largely moved from the i386 timer_pm.c file
+ * which was (C) Dominik Brodowski <[email protected]> 2003
+ * and contained the following comments:
+ *
+ * Driver to use the Power Management Timer (PMTMR) available in some
+ * southbridges as primary timing source for the Linux kernel.
+ *
+ * Based on parts of linux/drivers/acpi/hardware/hwtimer.c, timer_pit.c,
+ * timer_hpet.c, and on Arjan van de Ven's implementation for 2.4.
+ *
+ * This file is licensed under the GPL v2.
+ */
+
+#include <linux/clocksource.h>
+#include <linux/errno.h>
+#include <linux/init.h>
+#include <asm/io.h>
+
+/* Number of PMTMR ticks expected during calibration run */
+#define PMTMR_TICKS_PER_SEC 3579545
+
+#if (defined(CONFIG_X86) && (!defined(CONFIG_X86_64)))
+# include "mach_timer.h"
+# define PMTMR_EXPECTED_RATE ((PMTMR_TICKS_PER_SEC*CALIBRATE_TIME_MSEC)/1000)
+#endif
+
+/*
+ * The I/O port the PMTMR resides at.
+ * The location is detected during setup_arch(),
+ * in arch/i386/acpi/boot.c
+ */
+extern u32 acpi_pmtmr_ioport;
+extern int acpi_pmtmr_buggy;
+
+#define ACPI_PM_MASK 0xFFFFFF /* limit it to 24 bits */
+
+static inline u32 read_pmtmr(void)
+{
+ /* mask the output to 24 bits */
+ return inl(acpi_pmtmr_ioport) & ACPI_PM_MASK;
+}
+
+static cycle_t acpi_pm_read_verified(void)
+{
+ u32 v1 = 0, v2 = 0, v3 = 0;
+
+ /*
+ * It has been reported that because of various broken
+ * chipsets (ICH4, PIIX4 and PIIX4E) where the ACPI PM clock
+ * source is not latched, so you must read it multiple
+ * times to ensure a safe value is read:
+ */
+ do {
+ v1 = read_pmtmr();
+ v2 = read_pmtmr();
+ v3 = read_pmtmr();
+ } while ((v1 > v2 && v1 < v3) || (v2 > v3 && v2 < v1)
+ || (v3 > v1 && v3 < v2));
+
+ return (cycle_t)v2;
+}
+
+static cycle_t acpi_pm_read(void)
+{
+ return (cycle_t)read_pmtmr();
+}
+
+struct clocksource clocksource_acpi_pm = {
+ .name = "acpi_pm",
+ .rating = 200,
+ .read = acpi_pm_read,
+ .mask = (cycle_t)ACPI_PM_MASK,
+ .mult = 0, /*to be caluclated*/
+ .shift = 22,
+ .is_continuous = 1,
+};
+
+#if defined(CONFIG_X86) && !defined(CONFIG_X86_64)
+/*
+ * Some boards have the PMTMR running way too fast. We check
+ * the PMTMR rate against PIT channel 2 to catch these cases.
+ */
+static int __init verify_pmtmr_rate(void)
+{
+ unsigned long count, delta;
+ u32 value1, value2;
+
+ mach_prepare_counter();
+ value1 = read_pmtmr();
+ mach_countup(&count);
+ value2 = read_pmtmr();
+ delta = (value2 - value1) & ACPI_PM_MASK;
+
+ /* check that the PMTMR delta is within 5% of what we expect: */
+ if (delta < (PMTMR_EXPECTED_RATE * 19) / 20 ||
+ delta > (PMTMR_EXPECTED_RATE * 21) / 20) {
+ printk(KERN_INFO "PM-Timer running at invalid rate: %lu%% of normal - aborting.\n", 100UL * delta / PMTMR_EXPECTED_RATE);
+ return -1;
+ }
+
+ return 0;
+}
+#else
+# define verify_pmtmr_rate() (0)
+#endif
+
+static int __init init_acpi_pm_clocksource(void)
+{
+ u32 value1, value2;
+ unsigned int i;
+
+ if (!acpi_pmtmr_ioport)
+ return -ENODEV;
+
+ clocksource_acpi_pm.mult = clocksource_hz2mult(PMTMR_TICKS_PER_SEC,
+ clocksource_acpi_pm.shift);
+
+ /* "verify" this timing source: */
+ value1 = read_pmtmr();
+ for (i = 0; i < 10000; i++) {
+ value2 = read_pmtmr();
+ if (value2 == value1)
+ continue;
+ if (value2 > value1)
+ goto pm_good;
+ if ((value2 < value1) && ((value2) < 0xFFF))
+ goto pm_good;
+ printk(KERN_INFO "PM-Timer had inconsistent results: 0x%#x, 0x%#x - aborting.\n", value1, value2);
+ return -EINVAL;
+ }
+ printk(KERN_INFO "PM-Timer had no reasonable result: 0x%#x - aborting.\n", value1);
+ return -ENODEV;
+
+pm_good:
+ if (verify_pmtmr_rate() != 0)
+ return -ENODEV;
+
+ /* check to see if pmtmr is known buggy: */
+ if (acpi_pmtmr_buggy) {
+ clocksource_acpi_pm.read = acpi_pm_read_verified;
+ clocksource_acpi_pm.rating = 110;
+ }
+
+ register_clocksource(&clocksource_acpi_pm);
+
+ return 0;
+}
+
+module_init(init_acpi_pm_clocksource);
diff -ruN tod-mm1/drivers/clocksource/cyclone.c tod-mm2/drivers/clocksource/cyclone.c
--- tod-mm1/drivers/clocksource/cyclone.c 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/drivers/clocksource/cyclone.c 2005-12-01 18:26:50.000000000 -0800
@@ -0,0 +1,121 @@
+#include <linux/clocksource.h>
+#include <linux/string.h>
+#include <linux/errno.h>
+#include <linux/timex.h>
+#include <linux/init.h>
+
+#include <asm/pgtable.h>
+#include <asm/io.h>
+
+#include "mach_timer.h"
+
+#define CYCLONE_CBAR_ADDR 0xFEB00CD0 /* base address ptr */
+#define CYCLONE_PMCC_OFFSET 0x51A0 /* offset to control register */
+#define CYCLONE_MPCS_OFFSET 0x51A8 /* offset to select register */
+#define CYCLONE_MPMC_OFFSET 0x51D0 /* offset to count register */
+#define CYCLONE_TIMER_FREQ 99780000 /* 100Mhz, but not really */
+#define CYCLONE_TIMER_MASK 0xFFFFFFFF /* 32 bit mask */
+
+int use_cyclone = 0;
+static void __iomem *cyclone_ptr;
+
+static cycle_t read_cyclone(void)
+{
+ return (cycle_t)readl(cyclone_ptr);
+}
+
+struct clocksource clocksource_cyclone = {
+ .name = "cyclone",
+ .rating = 250,
+ .read = read_cyclone,
+ .mask = (cycle_t)CYCLONE_TIMER_MASK,
+ .mult = 10,
+ .shift = 0,
+ .is_continuous = 1,
+};
+
+static int __init init_cyclone_clocksource(void)
+{
+ unsigned long base; /* saved value from CBAR */
+ unsigned long offset;
+ u32 __iomem* volatile cyclone_timer; /* Cyclone MPMC0 register */
+ u32 __iomem* reg;
+ int i;
+
+ /* make sure we're on a summit box: */
+ if (!use_cyclone)
+ return -ENODEV;
+
+ printk(KERN_INFO "Summit chipset: Starting Cyclone Counter.\n");
+
+ /* find base address: */
+ offset = CYCLONE_CBAR_ADDR;
+ reg = ioremap_nocache(offset, sizeof(reg));
+ if (!reg) {
+ printk(KERN_ERR "Summit chipset: Could not find valid CBAR register.\n");
+ return -ENODEV;
+ }
+ /* even on 64bit systems, this is only 32bits: */
+ base = readl(reg);
+ if (!base) {
+ printk(KERN_ERR "Summit chipset: Could not find valid CBAR value.\n");
+ return -ENODEV;
+ }
+ iounmap(reg);
+
+ /* setup PMCC: */
+ offset = base + CYCLONE_PMCC_OFFSET;
+ reg = ioremap_nocache(offset, sizeof(reg));
+ if (!reg) {
+ printk(KERN_ERR "Summit chipset: Could not find valid PMCC register.\n");
+ return -ENODEV;
+ }
+ writel(0x00000001,reg);
+ iounmap(reg);
+
+ /* setup MPCS: */
+ offset = base + CYCLONE_MPCS_OFFSET;
+ reg = ioremap_nocache(offset, sizeof(reg));
+ if (!reg) {
+ printk(KERN_ERR "Summit chipset: Could not find valid MPCS register.\n");
+ return -ENODEV;
+ }
+ writel(0x00000001,reg);
+ iounmap(reg);
+
+ /* map in cyclone_timer: */
+ offset = base + CYCLONE_MPMC_OFFSET;
+ cyclone_timer = ioremap_nocache(offset, sizeof(u64));
+ if (!cyclone_timer) {
+ printk(KERN_ERR "Summit chipset: Could not find valid MPMC register.\n");
+ return -ENODEV;
+ }
+
+ /* quick test to make sure its ticking: */
+ for (i = 0; i < 3; i++){
+ u32 old = readl(cyclone_timer);
+ int stall = 100;
+
+ while (stall--)
+ barrier();
+
+ if (readl(cyclone_timer) == old) {
+ printk(KERN_ERR "Summit chipset: Counter not counting! DISABLED\n");
+ iounmap(cyclone_timer);
+ cyclone_timer = NULL;
+ return -ENODEV;
+ }
+ }
+ cyclone_ptr = cyclone_timer;
+
+ /* sort out mult/shift values: */
+ clocksource_cyclone.shift = 22;
+ clocksource_cyclone.mult = clocksource_hz2mult(CYCLONE_TIMER_FREQ,
+ clocksource_cyclone.shift);
+
+ register_clocksource(&clocksource_cyclone);
+
+ return 0;
+}
+
+module_init(init_cyclone_clocksource);
diff -ruN tod-mm1/drivers/clocksource/Makefile tod-mm2/drivers/clocksource/Makefile
--- tod-mm1/drivers/clocksource/Makefile 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/drivers/clocksource/Makefile 2005-12-01 18:26:50.000000000 -0800
@@ -0,0 +1,2 @@
+obj-$(CONFIG_X86_CYCLONE_TIMER) += cyclone.o
+obj-$(CONFIG_X86_PM_TIMER) += acpi_pm.o
diff -ruN tod-mm1/drivers/Makefile tod-mm2/drivers/Makefile
--- tod-mm1/drivers/Makefile 2005-12-01 18:13:03.000000000 -0800
+++ tod-mm2/drivers/Makefile 2005-12-01 18:26:50.000000000 -0800
@@ -72,3 +72,4 @@
obj-y += firmware/
obj-$(CONFIG_CRYPTO) += crypto/
obj-$(CONFIG_SUPERH) += sh/
+obj-$(CONFIG_GENERIC_TIME) += clocksource/
All,
The conversion of i386 to use the generic timeofday subsystem has been
split into 6 parts. This patch, the fifth of six, converts the i386 arch
to use the generic timeofday subsystem.
It applies on top of my timeofday-arch-i386-part4 patch. This patch is
the last in the timeofday-arch-i386 patchset, so you should be able to
build and boot a kernel after it has been applied.
Note that this patch does not provide any i386 clocksource, so you will
only have the jiffies clocksource. To get full replacements for the code
being removed here, the following timeofday-clocks-i386 patch will need
to be applied.
thanks
-john
Signed-off-by: John Stultz <[email protected]>
arch/i386/Kconfig | 4
arch/i386/kernel/Makefile | 1
arch/i386/kernel/time.c | 219 ++++++-------------------------------------
arch/i386/kernel/tsc.c | 12 --
arch/i386/lib/delay.c | 66 +++++++++++-
include/asm-i386/delay.h | 2
include/asm-i386/timeofday.h | 4
include/asm-i386/timer.h | 57 -----------
8 files changed, 104 insertions(+), 261 deletions(-)
linux-2.6.15-rc3-mm1_timeofday-arch-i386-part5_B12.patch
==========================
diff -ruN tod-mm1/arch/i386/Kconfig tod-mm2/arch/i386/Kconfig
--- tod-mm1/arch/i386/Kconfig 2005-12-01 18:12:56.000000000 -0800
+++ tod-mm2/arch/i386/Kconfig 2005-12-01 18:25:45.000000000 -0800
@@ -14,6 +14,10 @@
486, 586, Pentiums, and various instruction-set-compatible chips by
AMD, Cyrix, and others.
+config GENERIC_TIME
+ bool
+ default y
+
config SEMAPHORE_SLEEPERS
bool
default y
diff -ruN tod-mm1/arch/i386/kernel/Makefile tod-mm2/arch/i386/kernel/Makefile
--- tod-mm1/arch/i386/kernel/Makefile 2005-12-01 18:25:04.000000000 -0800
+++ tod-mm2/arch/i386/kernel/Makefile 2005-12-01 18:25:45.000000000 -0800
@@ -10,7 +10,6 @@
quirks.o i8237.o i8253.o tsc.o
obj-y += cpu/
-obj-y += timers/
obj-$(CONFIG_ACPI) += acpi/
obj-$(CONFIG_X86_BIOS_REBOOT) += reboot.o
obj-$(CONFIG_MCA) += mca.o
diff -ruN tod-mm1/arch/i386/kernel/time.c tod-mm2/arch/i386/kernel/time.c
--- tod-mm1/arch/i386/kernel/time.c 2005-12-01 18:24:31.000000000 -0800
+++ tod-mm2/arch/i386/kernel/time.c 2005-12-01 18:25:45.000000000 -0800
@@ -56,6 +56,7 @@
#include <asm/uaccess.h>
#include <asm/processor.h>
#include <asm/timer.h>
+#include <asm/timeofday.h>
#include "mach_time.h"
@@ -82,8 +83,6 @@
DEFINE_SPINLOCK(rtc_lock);
EXPORT_SYMBOL(rtc_lock);
-struct timer_opts *cur_timer __read_mostly = &timer_none;
-
/*
* This is a special lock that is owned by the CPU and holds the index
* register we are working with. It is required for NMI access to the
@@ -113,99 +112,19 @@
}
EXPORT_SYMBOL(rtc_cmos_write);
-/*
- * This version of gettimeofday has microsecond resolution
- * and better than microsecond precision on fast x86 machines with TSC.
- */
-void do_gettimeofday(struct timeval *tv)
-{
- unsigned long seq;
- unsigned long usec, sec;
- unsigned long max_ntp_tick;
-
- do {
- unsigned long lost;
-
- seq = read_seqbegin(&xtime_lock);
-
- usec = cur_timer->get_offset();
- lost = jiffies - wall_jiffies;
-
- /*
- * If time_adjust is negative then NTP is slowing the clock
- * so make sure not to go into next possible interval.
- * Better to lose some accuracy than have time go backwards..
- */
- if (unlikely(time_adjust < 0)) {
- max_ntp_tick = (USEC_PER_SEC / HZ) - tickadj;
- usec = min(usec, max_ntp_tick);
-
- if (lost)
- usec += lost * max_ntp_tick;
- }
- else if (unlikely(lost))
- usec += lost * (USEC_PER_SEC / HZ);
-
- sec = xtime.tv_sec;
- usec += (xtime.tv_nsec / 1000);
- } while (read_seqretry(&xtime_lock, seq));
-
- while (usec >= 1000000) {
- usec -= 1000000;
- sec++;
- }
-
- tv->tv_sec = sec;
- tv->tv_usec = usec;
-}
-
-EXPORT_SYMBOL(do_gettimeofday);
-
-int do_settimeofday(struct timespec *tv)
-{
- time_t wtm_sec, sec = tv->tv_sec;
- long wtm_nsec, nsec = tv->tv_nsec;
-
- if ((unsigned long)tv->tv_nsec >= NSEC_PER_SEC)
- return -EINVAL;
-
- write_seqlock_irq(&xtime_lock);
- /*
- * This is revolting. We need to set "xtime" correctly. However, the
- * value in this location is the value at the most recent update of
- * wall time. Discover what correction gettimeofday() would have
- * made, and then undo it!
- */
- nsec -= cur_timer->get_offset() * NSEC_PER_USEC;
- nsec -= (jiffies - wall_jiffies) * TICK_NSEC;
-
- wtm_sec = wall_to_monotonic.tv_sec + (xtime.tv_sec - sec);
- wtm_nsec = wall_to_monotonic.tv_nsec + (xtime.tv_nsec - nsec);
-
- set_normalized_timespec(&xtime, sec, nsec);
- set_normalized_timespec(&wall_to_monotonic, wtm_sec, wtm_nsec);
-
- ntp_clear();
- write_sequnlock_irq(&xtime_lock);
- clock_was_set();
- return 0;
-}
-
-EXPORT_SYMBOL(do_settimeofday);
-
static int set_rtc_mmss(unsigned long nowtime)
{
int retval;
-
- WARN_ON(irqs_disabled());
+ unsigned long flags;
/* gets recalled with irq locally disabled */
- spin_lock_irq(&rtc_lock);
+ /* XXX - does irqsave resolve this? -johnstul */
+ spin_lock_irqsave(&rtc_lock, flags);
if (efi_enabled)
retval = efi_set_rtc_mmss(nowtime);
else
retval = mach_set_rtc_mmss(nowtime);
- spin_unlock_irq(&rtc_lock);
+ spin_unlock_irqrestore(&rtc_lock, flags);
return retval;
}
@@ -213,16 +132,6 @@
int timer_ack;
-/* monotonic_clock(): returns # of nanoseconds passed since time_init()
- * Note: This function is required to return accurate
- * time even in the absence of multiple timer ticks.
- */
-unsigned long long monotonic_clock(void)
-{
- return cur_timer->monotonic_clock();
-}
-EXPORT_SYMBOL(monotonic_clock);
-
#if defined(CONFIG_SMP) && defined(CONFIG_FRAME_POINTER)
unsigned long profile_pc(struct pt_regs *regs)
{
@@ -237,11 +146,21 @@
#endif
/*
- * timer_interrupt() needs to keep up the real-time clock,
- * as well as call the "do_timer()" routine every clocktick
+ * This is the same as the above, except we _also_ save the current
+ * Time Stamp Counter value at the time of the timer interrupt, so that
+ * we later on can estimate the time of day more exactly.
*/
-static inline void do_timer_interrupt(int irq, struct pt_regs *regs)
+irqreturn_t timer_interrupt(int irq, void *dev_id, struct pt_regs *regs)
{
+ /*
+ * Here we are in the timer irq handler. We just have irqs locally
+ * disabled but we don't know if the timer_bh is running on the other
+ * CPU. We need to avoid to SMP race with it. NOTE: we don' t need
+ * the irq version of write_lock because as just said we have irq
+ * locally disabled. -arca
+ */
+ write_seqlock(&xtime_lock);
+
#ifdef CONFIG_X86_IO_APIC
if (timer_ack) {
/*
@@ -274,27 +193,6 @@
irq = inb_p( 0x61 ); /* read the current state */
outb_p( irq|0x80, 0x61 ); /* reset the IRQ */
}
-}
-
-/*
- * This is the same as the above, except we _also_ save the current
- * Time Stamp Counter value at the time of the timer interrupt, so that
- * we later on can estimate the time of day more exactly.
- */
-irqreturn_t timer_interrupt(int irq, void *dev_id, struct pt_regs *regs)
-{
- /*
- * Here we are in the timer irq handler. We just have irqs locally
- * disabled but we don't know if the timer_bh is running on the other
- * CPU. We need to avoid to SMP race with it. NOTE: we don' t need
- * the irq version of write_lock because as just said we have irq
- * locally disabled. -arca
- */
- write_seqlock(&xtime_lock);
-
- cur_timer->mark_offset();
-
- do_timer_interrupt(irq, regs);
write_sequnlock(&xtime_lock);
return IRQ_HANDLED;
@@ -318,58 +216,37 @@
}
EXPORT_SYMBOL(get_cmos_time);
-static void sync_cmos_clock(unsigned long dummy);
-
-static DEFINE_TIMER(sync_cmos_timer, sync_cmos_clock, 0, 0);
-
-static void sync_cmos_clock(unsigned long dummy)
+/* arch specific timeofday hooks */
+nsec_t read_persistent_clock(void)
{
- struct timeval now, next;
- int fail = 1;
+ return (nsec_t)get_cmos_time() * NSEC_PER_SEC;
+}
+void sync_persistent_clock(struct timespec ts)
+{
+ static unsigned long last_rtc_update;
/*
* If we have an externally synchronized Linux clock, then update
* CMOS clock accordingly every ~11 minutes. Set_rtc_mmss() has to be
* called as close as possible to 500 ms before the new second starts.
- * This code is run on a timer. If the clock is set, that timer
- * may not expire at the correct time. Thus, we adjust...
*/
- if (!ntp_synced())
- /*
- * Not synced, exit, do not restart a timer (if one is
- * running, let it run out).
- */
+ if (ts.tv_sec <= last_rtc_update + 660)
return;
- do_gettimeofday(&now);
- if (now.tv_usec >= USEC_AFTER - ((unsigned) TICK_SIZE) / 2 &&
- now.tv_usec <= USEC_BEFORE + ((unsigned) TICK_SIZE) / 2)
- fail = set_rtc_mmss(now.tv_sec);
-
- next.tv_usec = USEC_AFTER - now.tv_usec;
- if (next.tv_usec <= 0)
- next.tv_usec += USEC_PER_SEC;
-
- if (!fail)
- next.tv_sec = 659;
- else
- next.tv_sec = 0;
-
- if (next.tv_usec >= USEC_PER_SEC) {
- next.tv_sec++;
- next.tv_usec -= USEC_PER_SEC;
+ if((ts.tv_nsec / 1000) >= USEC_AFTER - ((unsigned) TICK_SIZE) / 2 &&
+ (ts.tv_nsec / 1000) <= USEC_BEFORE + ((unsigned) TICK_SIZE) / 2) {
+ /* horrible...FIXME */
+ if (set_rtc_mmss(ts.tv_sec) == 0)
+ last_rtc_update = ts.tv_sec;
+ else
+ last_rtc_update = ts.tv_sec - 600; /* do it again in 60 s */
}
- mod_timer(&sync_cmos_timer, jiffies + timeval_to_jiffies(&next));
}
-void notify_arch_cmos_timer(void)
-{
- mod_timer(&sync_cmos_timer, jiffies + 1);
-}
+
static long clock_cmos_diff, sleep_start;
-static struct timer_opts *last_timer;
static int timer_suspend(struct sys_device *dev, pm_message_t state)
{
/*
@@ -378,16 +255,11 @@
clock_cmos_diff = -get_cmos_time();
clock_cmos_diff += get_seconds();
sleep_start = get_cmos_time();
- last_timer = cur_timer;
- cur_timer = &timer_none;
- if (last_timer->suspend)
- last_timer->suspend(state);
return 0;
}
static int timer_resume(struct sys_device *dev)
{
- unsigned long flags;
unsigned long sec;
unsigned long sleep_length;
@@ -397,16 +269,8 @@
#endif
sec = get_cmos_time() + clock_cmos_diff;
sleep_length = (get_cmos_time() - sleep_start) * HZ;
- write_seqlock_irqsave(&xtime_lock, flags);
- xtime.tv_sec = sec;
- xtime.tv_nsec = 0;
- write_sequnlock_irqrestore(&xtime_lock, flags);
jiffies += sleep_length;
wall_jiffies += sleep_length;
- if (last_timer->resume)
- last_timer->resume();
- cur_timer = last_timer;
- last_timer = NULL;
touch_softlockup_watchdog();
return 0;
}
@@ -439,17 +303,10 @@
/* Duplicate of time_init() below, with hpet_enable part added */
static void __init hpet_time_init(void)
{
- xtime.tv_sec = get_cmos_time();
- xtime.tv_nsec = (INITIAL_JIFFIES % HZ) * (NSEC_PER_SEC / HZ);
- set_normalized_timespec(&wall_to_monotonic,
- -xtime.tv_sec, -xtime.tv_nsec);
-
if ((hpet_enable() >= 0) && hpet_use_timer) {
printk("Using HPET for base-timer\n");
}
- cur_timer = select_timer();
- printk(KERN_INFO "Using %s for high-res timesource\n",cur_timer->name);
time_init_hook();
}
@@ -467,13 +324,5 @@
return;
}
#endif
- xtime.tv_sec = get_cmos_time();
- xtime.tv_nsec = (INITIAL_JIFFIES % HZ) * (NSEC_PER_SEC / HZ);
- set_normalized_timespec(&wall_to_monotonic,
- -xtime.tv_sec, -xtime.tv_nsec);
-
- cur_timer = select_timer();
- printk(KERN_INFO "Using %s for high-res timesource\n",cur_timer->name);
-
time_init_hook();
}
diff -ruN tod-mm1/arch/i386/kernel/tsc.c tod-mm2/arch/i386/kernel/tsc.c
--- tod-mm1/arch/i386/kernel/tsc.c 2005-12-01 18:25:25.000000000 -0800
+++ tod-mm2/arch/i386/kernel/tsc.c 2005-12-01 18:25:45.000000000 -0800
@@ -10,6 +10,7 @@
#include <linux/init.h>
#include <asm/tsc.h>
+#include <asm/delay.h>
#include <asm/io.h>
#include "mach_timer.h"
@@ -45,16 +46,6 @@
__setup("notsc", tsc_setup);
-
-int read_current_timer(unsigned long *timer_val)
-{
- if (!tsc_disable && cpu_khz) {
- rdtscl(*timer_val);
- return 0;
- }
- return -1;
-}
-
/*
* code to mark and check if the TSC is unstable
* due to cpufreq or due to unsynced TSCs
@@ -226,6 +217,7 @@
(unsigned long)cpu_khz % 1000);
set_cyc2ns_scale(cpu_khz);
+ use_tsc_delay();
}
#ifdef CONFIG_CPU_FREQ
diff -ruN tod-mm1/arch/i386/lib/delay.c tod-mm2/arch/i386/lib/delay.c
--- tod-mm1/arch/i386/lib/delay.c 2005-12-01 18:12:56.000000000 -0800
+++ tod-mm2/arch/i386/lib/delay.c 2005-12-01 18:25:45.000000000 -0800
@@ -10,43 +10,93 @@
* we have to worry about.
*/
+#include <linux/timeofday.h>
+#include <linux/module.h>
#include <linux/config.h>
#include <linux/sched.h>
#include <linux/delay.h>
-#include <linux/module.h>
+
#include <asm/processor.h>
#include <asm/delay.h>
#include <asm/timer.h>
#ifdef CONFIG_SMP
-#include <asm/smp.h>
+# include <asm/smp.h>
#endif
-extern struct timer_opts* timer;
+/* simple loop based delay: */
+static void delay_loop(unsigned long loops)
+{
+ int d0;
+
+ __asm__ __volatile__(
+ "\tjmp 1f\n"
+ ".align 16\n"
+ "1:\tjmp 2f\n"
+ ".align 16\n"
+ "2:\tdecl %0\n\tjns 2b"
+ :"=&a" (d0)
+ :"0" (loops));
+}
+
+/* TSC based delay: */
+static void delay_tsc(unsigned long loops)
+{
+ unsigned long bclock, now;
+
+ rdtscl(bclock);
+ do {
+ rep_nop();
+ rdtscl(now);
+ } while ((now-bclock) < loops);
+}
+
+/*
+ * Since we calibrate only once at boot, this
+ * function should be set once at boot and not changed
+ */
+static void (*delay_fn)(unsigned long) = delay_loop;
+
+void use_tsc_delay(void)
+{
+ delay_fn = delay_tsc;
+}
+
+int read_current_timer(unsigned long *timer_val)
+{
+ if (delay_fn == delay_tsc) {
+ rdtscl(*timer_val);
+ return 0;
+ }
+ return -1;
+}
void __delay(unsigned long loops)
{
- cur_timer->delay(loops);
+ delay_fn(loops);
}
inline void __const_udelay(unsigned long xloops)
{
int d0;
+
xloops *= 4;
__asm__("mull %0"
:"=d" (xloops), "=&a" (d0)
- :"1" (xloops),"0" (cpu_data[raw_smp_processor_id()].loops_per_jiffy * (HZ/4)));
- __delay(++xloops);
+ :"1" (xloops), "0"
+ (cpu_data[raw_smp_processor_id()].loops_per_jiffy * (HZ/4)));
+
+ __delay(++xloops);
}
void __udelay(unsigned long usecs)
{
- __const_udelay(usecs * 0x000010c7); /* 2**32 / 1000000 (rounded up) */
+ __const_udelay(usecs * 0x000010c7); /* 2**32 / 1000000 (rounded up) */
}
void __ndelay(unsigned long nsecs)
{
- __const_udelay(nsecs * 0x00005); /* 2**32 / 1000000000 (rounded up) */
+ __const_udelay(nsecs * 0x00005); /* 2**32 / 1000000000 (rounded up) */
}
EXPORT_SYMBOL(__delay);
diff -ruN tod-mm1/include/asm-i386/delay.h tod-mm2/include/asm-i386/delay.h
--- tod-mm1/include/asm-i386/delay.h 2005-12-01 18:13:31.000000000 -0800
+++ tod-mm2/include/asm-i386/delay.h 2005-12-01 18:25:45.000000000 -0800
@@ -23,4 +23,6 @@
((n) > 20000 ? __bad_ndelay() : __const_udelay((n) * 5ul)) : \
__ndelay(n))
+void use_tsc_delay(void);
+
#endif /* defined(_I386_DELAY_H) */
diff -ruN tod-mm1/include/asm-i386/timeofday.h tod-mm2/include/asm-i386/timeofday.h
--- tod-mm1/include/asm-i386/timeofday.h 1969-12-31 16:00:00.000000000 -0800
+++ tod-mm2/include/asm-i386/timeofday.h 2005-12-01 18:25:45.000000000 -0800
@@ -0,0 +1,4 @@
+#ifndef _ASM_I386_TIMEOFDAY_H
+#define _ASM_I386_TIMEOFDAY_H
+#include <asm-generic/timeofday.h>
+#endif
diff -ruN tod-mm1/include/asm-i386/timer.h tod-mm2/include/asm-i386/timer.h
--- tod-mm1/include/asm-i386/timer.h 2005-12-01 18:13:31.000000000 -0800
+++ tod-mm2/include/asm-i386/timer.h 2005-12-01 18:25:45.000000000 -0800
@@ -3,68 +3,11 @@
#include <linux/init.h>
#include <linux/pm.h>
-/**
- * struct timer_ops - used to define a timer source
- *
- * @name: name of the timer.
- * @init: Probes and initializes the timer. Takes clock= override
- * string as an argument. Returns 0 on success, anything else
- * on failure.
- * @mark_offset: called by the timer interrupt.
- * @get_offset: called by gettimeofday(). Returns the number of microseconds
- * since the last timer interupt.
- * @monotonic_clock: returns the number of nanoseconds since the init of the
- * timer.
- * @delay: delays this many clock cycles.
- */
-struct timer_opts {
- char* name;
- void (*mark_offset)(void);
- unsigned long (*get_offset)(void);
- unsigned long long (*monotonic_clock)(void);
- void (*delay)(unsigned long);
- unsigned long (*read_timer)(void);
- int (*suspend)(pm_message_t state);
- int (*resume)(void);
-};
-
-struct init_timer_opts {
- int (*init)(char *override);
- struct timer_opts *opts;
-};
-
#define TICK_SIZE (tick_nsec / 1000)
-
-extern struct timer_opts* __init select_timer(void);
-extern void clock_fallback(void);
void setup_pit_timer(void);
-
/* Modifiers for buggy PIT handling */
-
extern int pit_latch_buggy;
-
-extern struct timer_opts *cur_timer;
extern int timer_ack;
-
-/* list of externed timers */
-extern struct timer_opts timer_none;
-extern struct timer_opts timer_pit;
-extern struct init_timer_opts timer_pit_init;
-extern struct init_timer_opts timer_tsc_init;
-#ifdef CONFIG_X86_CYCLONE_TIMER
-extern struct init_timer_opts timer_cyclone_init;
-#endif
-
-extern unsigned long calibrate_tsc(void);
-extern unsigned long read_timer_tsc(void);
-extern void init_cpu_khz(void);
extern int recalibrate_cpu_khz(void);
-#ifdef CONFIG_HPET_TIMER
-extern struct init_timer_opts timer_hpet_init;
-extern unsigned long calibrate_tsc_hpet(unsigned long *tsc_hpet_quotient_ptr);
-#endif
-#ifdef CONFIG_X86_PM_TIMER
-extern struct init_timer_opts timer_pmtmr_init;
-#endif
#endif
All,
The conversion of i386 to use the generic timeofday subsystem has been
split into 6 parts. This patch, the final of four, removes the old
timers/timer_opts infrastructure.
It applies on top of my timeofday-arch-i386-part5 patch. This patch is
the last in the timeofday-arch-i386 patchset, so you should be able to
build and boot a kernel after it has been applied.
Note that this patch does not provide any i386 clocksources, so you will
only have the jiffies clocksource. To get full replacements for the code
being removed here, the following timeofday-clocks-i386 patch will need
to be applied.
thanks
-john
Signed-off-by: John Stultz <[email protected]>
Makefile | 9 -
common.c | 88 ------------
timer.c | 75 ----------
timer_cyclone.c | 259 -------------------------------------
timer_hpet.c | 217 -------------------------------
timer_none.c | 39 -----
timer_pit.c | 164 -----------------------
timer_pm.c | 268 --------------------------------------
timer_tsc.c | 388 --------------------------------------------------------
9 files changed, 1507 deletions(-)
linux-2.6.15-rc3-mm1_timeofday-arch-i386-part6_B12.patch
==========================
diff -ruN tod-mm1/arch/i386/kernel/timers/common.c tod-mm2/arch/i386/kernel/timers/common.c
--- tod-mm1/arch/i386/kernel/timers/common.c 2005-12-01 18:25:04.000000000 -0800
+++ tod-mm2/arch/i386/kernel/timers/common.c 1969-12-31 16:00:00.000000000 -0800
@@ -1,88 +0,0 @@
-/*
- * Common functions used across the timers go here
- */
-
-#include <linux/init.h>
-#include <linux/timex.h>
-#include <linux/errno.h>
-#include <linux/jiffies.h>
-#include <linux/module.h>
-
-#include <asm/io.h>
-#include <asm/timer.h>
-#include <asm/hpet.h>
-
-#include "mach_timer.h"
-
-#ifdef CONFIG_HPET_TIMER
-/* ------ Calibrate the TSC using HPET -------
- * Return 2^32 * (1 / (TSC clocks per usec)) for getting the CPU freq.
- * Second output is parameter 1 (when non NULL)
- * Set 2^32 * (1 / (tsc per HPET clk)) for delay_hpet().
- * calibrate_tsc() calibrates the processor TSC by comparing
- * it to the HPET timer of known frequency.
- * Too much 64-bit arithmetic here to do this cleanly in C
- */
-#define CALIBRATE_CNT_HPET (5 * hpet_tick)
-#define CALIBRATE_TIME_HPET (5 * KERNEL_TICK_USEC)
-
-unsigned long __devinit calibrate_tsc_hpet(unsigned long *tsc_hpet_quotient_ptr)
-{
- unsigned long tsc_startlow, tsc_starthigh;
- unsigned long tsc_endlow, tsc_endhigh;
- unsigned long hpet_start, hpet_end;
- unsigned long result, remain;
-
- hpet_start = hpet_readl(HPET_COUNTER);
- rdtsc(tsc_startlow, tsc_starthigh);
- do {
- hpet_end = hpet_readl(HPET_COUNTER);
- } while ((hpet_end - hpet_start) < CALIBRATE_CNT_HPET);
- rdtsc(tsc_endlow, tsc_endhigh);
-
- /* 64-bit subtract - gcc just messes up with long longs */
- __asm__("subl %2,%0\n\t"
- "sbbl %3,%1"
- :"=a" (tsc_endlow), "=d" (tsc_endhigh)
- :"g" (tsc_startlow), "g" (tsc_starthigh),
- "0" (tsc_endlow), "1" (tsc_endhigh));
-
- /* Error: ECPUTOOFAST */
- if (tsc_endhigh)
- goto bad_calibration;
-
- /* Error: ECPUTOOSLOW */
- if (tsc_endlow <= CALIBRATE_TIME_HPET)
- goto bad_calibration;
-
- ASM_DIV64_REG(result, remain, tsc_endlow, 0, CALIBRATE_TIME_HPET);
- if (remain > (tsc_endlow >> 1))
- result++; /* rounding the result */
-
- if (tsc_hpet_quotient_ptr) {
- unsigned long tsc_hpet_quotient;
-
- ASM_DIV64_REG(tsc_hpet_quotient, remain, tsc_endlow, 0,
- CALIBRATE_CNT_HPET);
- if (remain > (tsc_endlow >> 1))
- tsc_hpet_quotient++; /* rounding the result */
- *tsc_hpet_quotient_ptr = tsc_hpet_quotient;
- }
-
- return result;
-bad_calibration:
- /*
- * the CPU was so fast/slow that the quotient wouldn't fit in
- * 32 bits..
- */
- return 0;
-}
-#endif
-
-
-unsigned long read_timer_tsc(void)
-{
- unsigned long retval;
- rdtscl(retval);
- return retval;
-}
diff -ruN tod-mm1/arch/i386/kernel/timers/Makefile tod-mm2/arch/i386/kernel/timers/Makefile
--- tod-mm1/arch/i386/kernel/timers/Makefile 2005-12-01 18:12:56.000000000 -0800
+++ tod-mm2/arch/i386/kernel/timers/Makefile 1969-12-31 16:00:00.000000000 -0800
@@ -1,9 +0,0 @@
-#
-# Makefile for x86 timers
-#
-
-obj-y := timer.o timer_none.o timer_tsc.o timer_pit.o common.o
-
-obj-$(CONFIG_X86_CYCLONE_TIMER) += timer_cyclone.o
-obj-$(CONFIG_HPET_TIMER) += timer_hpet.o
-obj-$(CONFIG_X86_PM_TIMER) += timer_pm.o
diff -ruN tod-mm1/arch/i386/kernel/timers/timer.c tod-mm2/arch/i386/kernel/timers/timer.c
--- tod-mm1/arch/i386/kernel/timers/timer.c 2005-12-01 18:12:56.000000000 -0800
+++ tod-mm2/arch/i386/kernel/timers/timer.c 1969-12-31 16:00:00.000000000 -0800
@@ -1,75 +0,0 @@
-#include <linux/init.h>
-#include <linux/kernel.h>
-#include <linux/string.h>
-#include <asm/timer.h>
-
-#ifdef CONFIG_HPET_TIMER
-/*
- * HPET memory read is slower than tsc reads, but is more dependable as it
- * always runs at constant frequency and reduces complexity due to
- * cpufreq. So, we prefer HPET timer to tsc based one. Also, we cannot use
- * timer_pit when HPET is active. So, we default to timer_tsc.
- */
-#endif
-/* list of timers, ordered by preference, NULL terminated */
-static struct init_timer_opts* __initdata timers[] = {
-#ifdef CONFIG_X86_CYCLONE_TIMER
- &timer_cyclone_init,
-#endif
-#ifdef CONFIG_HPET_TIMER
- &timer_hpet_init,
-#endif
-#ifdef CONFIG_X86_PM_TIMER
- &timer_pmtmr_init,
-#endif
- &timer_tsc_init,
- &timer_pit_init,
- NULL,
-};
-
-static char clock_override[10] __initdata;
-
-static int __init clock_setup(char* str)
-{
- if (str)
- strlcpy(clock_override, str, sizeof(clock_override));
- return 1;
-}
-__setup("clock=", clock_setup);
-
-
-/* The chosen timesource has been found to be bad.
- * Fall back to a known good timesource (the PIT)
- */
-void clock_fallback(void)
-{
- cur_timer = &timer_pit;
-}
-
-/* iterates through the list of timers, returning the first
- * one that initializes successfully.
- */
-struct timer_opts* __init select_timer(void)
-{
- int i = 0;
-
- /* find most preferred working timer */
- while (timers[i]) {
- if (timers[i]->init)
- if (timers[i]->init(clock_override) == 0)
- return timers[i]->opts;
- ++i;
- }
-
- panic("select_timer: Cannot find a suitable timer\n");
- return NULL;
-}
-
-int read_current_timer(unsigned long *timer_val)
-{
- if (cur_timer->read_timer) {
- *timer_val = cur_timer->read_timer();
- return 0;
- }
- return -1;
-}
diff -ruN tod-mm1/arch/i386/kernel/timers/timer_cyclone.c tod-mm2/arch/i386/kernel/timers/timer_cyclone.c
--- tod-mm1/arch/i386/kernel/timers/timer_cyclone.c 2005-12-01 18:12:56.000000000 -0800
+++ tod-mm2/arch/i386/kernel/timers/timer_cyclone.c 1969-12-31 16:00:00.000000000 -0800
@@ -1,259 +0,0 @@
-/* Cyclone-timer:
- * This code implements timer_ops for the cyclone counter found
- * on IBM x440, x360, and other Summit based systems.
- *
- * Copyright (C) 2002 IBM, John Stultz ([email protected])
- */
-
-
-#include <linux/spinlock.h>
-#include <linux/init.h>
-#include <linux/timex.h>
-#include <linux/errno.h>
-#include <linux/string.h>
-#include <linux/jiffies.h>
-
-#include <asm/timer.h>
-#include <asm/io.h>
-#include <asm/pgtable.h>
-#include <asm/fixmap.h>
-#include <asm/i8253.h>
-
-#include "io_ports.h"
-
-/* Number of usecs that the last interrupt was delayed */
-static int delay_at_last_interrupt;
-
-#define CYCLONE_CBAR_ADDR 0xFEB00CD0
-#define CYCLONE_PMCC_OFFSET 0x51A0
-#define CYCLONE_MPMC_OFFSET 0x51D0
-#define CYCLONE_MPCS_OFFSET 0x51A8
-#define CYCLONE_TIMER_FREQ 100000000
-#define CYCLONE_TIMER_MASK (((u64)1<<40)-1) /* 40 bit mask */
-int use_cyclone = 0;
-
-static u32* volatile cyclone_timer; /* Cyclone MPMC0 register */
-static u32 last_cyclone_low;
-static u32 last_cyclone_high;
-static unsigned long long monotonic_base;
-static seqlock_t monotonic_lock = SEQLOCK_UNLOCKED;
-
-/* helper macro to atomically read both cyclone counter registers */
-#define read_cyclone_counter(low,high) \
- do{ \
- high = cyclone_timer[1]; low = cyclone_timer[0]; \
- } while (high != cyclone_timer[1]);
-
-
-static void mark_offset_cyclone(void)
-{
- unsigned long lost, delay;
- unsigned long delta = last_cyclone_low;
- int count;
- unsigned long long this_offset, last_offset;
-
- write_seqlock(&monotonic_lock);
- last_offset = ((unsigned long long)last_cyclone_high<<32)|last_cyclone_low;
-
- spin_lock(&i8253_lock);
- read_cyclone_counter(last_cyclone_low,last_cyclone_high);
-
- /* read values for delay_at_last_interrupt */
- outb_p(0x00, 0x43); /* latch the count ASAP */
-
- count = inb_p(0x40); /* read the latched count */
- count |= inb(0x40) << 8;
-
- /*
- * VIA686a test code... reset the latch if count > max + 1
- * from timer_pit.c - cjb
- */
- if (count > LATCH) {
- outb_p(0x34, PIT_MODE);
- outb_p(LATCH & 0xff, PIT_CH0);
- outb(LATCH >> 8, PIT_CH0);
- count = LATCH - 1;
- }
- spin_unlock(&i8253_lock);
-
- /* lost tick compensation */
- delta = last_cyclone_low - delta;
- delta /= (CYCLONE_TIMER_FREQ/1000000);
- delta += delay_at_last_interrupt;
- lost = delta/(1000000/HZ);
- delay = delta%(1000000/HZ);
- if (lost >= 2)
- jiffies_64 += lost-1;
-
- /* update the monotonic base value */
- this_offset = ((unsigned long long)last_cyclone_high<<32)|last_cyclone_low;
- monotonic_base += (this_offset - last_offset) & CYCLONE_TIMER_MASK;
- write_sequnlock(&monotonic_lock);
-
- /* calculate delay_at_last_interrupt */
- count = ((LATCH-1) - count) * TICK_SIZE;
- delay_at_last_interrupt = (count + LATCH/2) / LATCH;
-
-
- /* catch corner case where tick rollover occured
- * between cyclone and pit reads (as noted when
- * usec delta is > 90% # of usecs/tick)
- */
- if (lost && abs(delay - delay_at_last_interrupt) > (900000/HZ))
- jiffies_64++;
-}
-
-static unsigned long get_offset_cyclone(void)
-{
- u32 offset;
-
- if(!cyclone_timer)
- return delay_at_last_interrupt;
-
- /* Read the cyclone timer */
- offset = cyclone_timer[0];
-
- /* .. relative to previous jiffy */
- offset = offset - last_cyclone_low;
-
- /* convert cyclone ticks to microseconds */
- /* XXX slow, can we speed this up? */
- offset = offset/(CYCLONE_TIMER_FREQ/1000000);
-
- /* our adjusted time offset in microseconds */
- return delay_at_last_interrupt + offset;
-}
-
-static unsigned long long monotonic_clock_cyclone(void)
-{
- u32 now_low, now_high;
- unsigned long long last_offset, this_offset, base;
- unsigned long long ret;
- unsigned seq;
-
- /* atomically read monotonic base & last_offset */
- do {
- seq = read_seqbegin(&monotonic_lock);
- last_offset = ((unsigned long long)last_cyclone_high<<32)|last_cyclone_low;
- base = monotonic_base;
- } while (read_seqretry(&monotonic_lock, seq));
-
-
- /* Read the cyclone counter */
- read_cyclone_counter(now_low,now_high);
- this_offset = ((unsigned long long)now_high<<32)|now_low;
-
- /* convert to nanoseconds */
- ret = base + ((this_offset - last_offset)&CYCLONE_TIMER_MASK);
- return ret * (1000000000 / CYCLONE_TIMER_FREQ);
-}
-
-static int __init init_cyclone(char* override)
-{
- u32* reg;
- u32 base; /* saved cyclone base address */
- u32 pageaddr; /* page that contains cyclone_timer register */
- u32 offset; /* offset from pageaddr to cyclone_timer register */
- int i;
-
- /* check clock override */
- if (override[0] && strncmp(override,"cyclone",7))
- return -ENODEV;
-
- /*make sure we're on a summit box*/
- if(!use_cyclone) return -ENODEV;
-
- printk(KERN_INFO "Summit chipset: Starting Cyclone Counter.\n");
-
- /* find base address */
- pageaddr = (CYCLONE_CBAR_ADDR)&PAGE_MASK;
- offset = (CYCLONE_CBAR_ADDR)&(~PAGE_MASK);
- set_fixmap_nocache(FIX_CYCLONE_TIMER, pageaddr);
- reg = (u32*)(fix_to_virt(FIX_CYCLONE_TIMER) + offset);
- if(!reg){
- printk(KERN_ERR "Summit chipset: Could not find valid CBAR register.\n");
- return -ENODEV;
- }
- base = *reg;
- if(!base){
- printk(KERN_ERR "Summit chipset: Could not find valid CBAR value.\n");
- return -ENODEV;
- }
-
- /* setup PMCC */
- pageaddr = (base + CYCLONE_PMCC_OFFSET)&PAGE_MASK;
- offset = (base + CYCLONE_PMCC_OFFSET)&(~PAGE_MASK);
- set_fixmap_nocache(FIX_CYCLONE_TIMER, pageaddr);
- reg = (u32*)(fix_to_virt(FIX_CYCLONE_TIMER) + offset);
- if(!reg){
- printk(KERN_ERR "Summit chipset: Could not find valid PMCC register.\n");
- return -ENODEV;
- }
- reg[0] = 0x00000001;
-
- /* setup MPCS */
- pageaddr = (base + CYCLONE_MPCS_OFFSET)&PAGE_MASK;
- offset = (base + CYCLONE_MPCS_OFFSET)&(~PAGE_MASK);
- set_fixmap_nocache(FIX_CYCLONE_TIMER, pageaddr);
- reg = (u32*)(fix_to_virt(FIX_CYCLONE_TIMER) + offset);
- if(!reg){
- printk(KERN_ERR "Summit chipset: Could not find valid MPCS register.\n");
- return -ENODEV;
- }
- reg[0] = 0x00000001;
-
- /* map in cyclone_timer */
- pageaddr = (base + CYCLONE_MPMC_OFFSET)&PAGE_MASK;
- offset = (base + CYCLONE_MPMC_OFFSET)&(~PAGE_MASK);
- set_fixmap_nocache(FIX_CYCLONE_TIMER, pageaddr);
- cyclone_timer = (u32*)(fix_to_virt(FIX_CYCLONE_TIMER) + offset);
- if(!cyclone_timer){
- printk(KERN_ERR "Summit chipset: Could not find valid MPMC register.\n");
- return -ENODEV;
- }
-
- /*quick test to make sure its ticking*/
- for(i=0; i<3; i++){
- u32 old = cyclone_timer[0];
- int stall = 100;
- while(stall--) barrier();
- if(cyclone_timer[0] == old){
- printk(KERN_ERR "Summit chipset: Counter not counting! DISABLED\n");
- cyclone_timer = 0;
- return -ENODEV;
- }
- }
-
- init_cpu_khz();
-
- /* Everything looks good! */
- return 0;
-}
-
-
-static void delay_cyclone(unsigned long loops)
-{
- unsigned long bclock, now;
- if(!cyclone_timer)
- return;
- bclock = cyclone_timer[0];
- do {
- rep_nop();
- now = cyclone_timer[0];
- } while ((now-bclock) < loops);
-}
-/************************************************************/
-
-/* cyclone timer_opts struct */
-static struct timer_opts timer_cyclone = {
- .name = "cyclone",
- .mark_offset = mark_offset_cyclone,
- .get_offset = get_offset_cyclone,
- .monotonic_clock = monotonic_clock_cyclone,
- .delay = delay_cyclone,
-};
-
-struct init_timer_opts __initdata timer_cyclone_init = {
- .init = init_cyclone,
- .opts = &timer_cyclone,
-};
diff -ruN tod-mm1/arch/i386/kernel/timers/timer_hpet.c tod-mm2/arch/i386/kernel/timers/timer_hpet.c
--- tod-mm1/arch/i386/kernel/timers/timer_hpet.c 2005-12-01 18:12:56.000000000 -0800
+++ tod-mm2/arch/i386/kernel/timers/timer_hpet.c 1969-12-31 16:00:00.000000000 -0800
@@ -1,217 +0,0 @@
-/*
- * This code largely moved from arch/i386/kernel/time.c.
- * See comments there for proper credits.
- */
-
-#include <linux/spinlock.h>
-#include <linux/init.h>
-#include <linux/timex.h>
-#include <linux/errno.h>
-#include <linux/string.h>
-#include <linux/jiffies.h>
-
-#include <asm/timer.h>
-#include <asm/io.h>
-#include <asm/processor.h>
-
-#include "io_ports.h"
-#include "mach_timer.h"
-#include <asm/hpet.h>
-
-static unsigned long hpet_usec_quotient __read_mostly; /* convert hpet clks to usec */
-static unsigned long tsc_hpet_quotient __read_mostly; /* convert tsc to hpet clks */
-static unsigned long hpet_last; /* hpet counter value at last tick*/
-static unsigned long last_tsc_low; /* lsb 32 bits of Time Stamp Counter */
-static unsigned long last_tsc_high; /* msb 32 bits of Time Stamp Counter */
-static unsigned long long monotonic_base;
-static seqlock_t monotonic_lock = SEQLOCK_UNLOCKED;
-
-/* convert from cycles(64bits) => nanoseconds (64bits)
- * basic equation:
- * ns = cycles / (freq / ns_per_sec)
- * ns = cycles * (ns_per_sec / freq)
- * ns = cycles * (10^9 / (cpu_khz * 10^3))
- * ns = cycles * (10^6 / cpu_khz)
- *
- * Then we use scaling math (suggested by [email protected]) to get:
- * ns = cycles * (10^6 * SC / cpu_khz) / SC
- * ns = cycles * cyc2ns_scale / SC
- *
- * And since SC is a constant power of two, we can convert the div
- * into a shift.
- *
- * We can use khz divisor instead of mhz to keep a better percision, since
- * cyc2ns_scale is limited to 10^6 * 2^10, which fits in 32 bits.
- * ([email protected])
- *
- * [email protected] "math is hard, lets go shopping!"
- */
-static unsigned long cyc2ns_scale;
-#define CYC2NS_SCALE_FACTOR 10 /* 2^10, carefully chosen */
-
-static inline void set_cyc2ns_scale(unsigned long cpu_khz)
-{
- cyc2ns_scale = (1000000 << CYC2NS_SCALE_FACTOR)/cpu_khz;
-}
-
-static inline unsigned long long cycles_2_ns(unsigned long long cyc)
-{
- return (cyc * cyc2ns_scale) >> CYC2NS_SCALE_FACTOR;
-}
-
-static unsigned long long monotonic_clock_hpet(void)
-{
- unsigned long long last_offset, this_offset, base;
- unsigned seq;
-
- /* atomically read monotonic base & last_offset */
- do {
- seq = read_seqbegin(&monotonic_lock);
- last_offset = ((unsigned long long)last_tsc_high<<32)|last_tsc_low;
- base = monotonic_base;
- } while (read_seqretry(&monotonic_lock, seq));
-
- /* Read the Time Stamp Counter */
- rdtscll(this_offset);
-
- /* return the value in ns */
- return base + cycles_2_ns(this_offset - last_offset);
-}
-
-static unsigned long get_offset_hpet(void)
-{
- register unsigned long eax, edx;
-
- eax = hpet_readl(HPET_COUNTER);
- eax -= hpet_last; /* hpet delta */
- eax = min(hpet_tick, eax);
- /*
- * Time offset = (hpet delta) * ( usecs per HPET clock )
- * = (hpet delta) * ( usecs per tick / HPET clocks per tick)
- * = (hpet delta) * ( hpet_usec_quotient ) / (2^32)
- *
- * Where,
- * hpet_usec_quotient = (2^32 * usecs per tick)/HPET clocks per tick
- *
- * Using a mull instead of a divl saves some cycles in critical path.
- */
- ASM_MUL64_REG(eax, edx, hpet_usec_quotient, eax);
-
- /* our adjusted time offset in microseconds */
- return edx;
-}
-
-static void mark_offset_hpet(void)
-{
- unsigned long long this_offset, last_offset;
- unsigned long offset;
-
- write_seqlock(&monotonic_lock);
- last_offset = ((unsigned long long)last_tsc_high<<32)|last_tsc_low;
- rdtsc(last_tsc_low, last_tsc_high);
-
- if (hpet_use_timer)
- offset = hpet_readl(HPET_T0_CMP) - hpet_tick;
- else
- offset = hpet_readl(HPET_COUNTER);
- if (unlikely(((offset - hpet_last) >= (2*hpet_tick)) && (hpet_last != 0))) {
- int lost_ticks = ((offset - hpet_last) / hpet_tick) - 1;
- jiffies_64 += lost_ticks;
- }
- hpet_last = offset;
-
- /* update the monotonic base value */
- this_offset = ((unsigned long long)last_tsc_high<<32)|last_tsc_low;
- monotonic_base += cycles_2_ns(this_offset - last_offset);
- write_sequnlock(&monotonic_lock);
-}
-
-static void delay_hpet(unsigned long loops)
-{
- unsigned long hpet_start, hpet_end;
- unsigned long eax;
-
- /* loops is the number of cpu cycles. Convert it to hpet clocks */
- ASM_MUL64_REG(eax, loops, tsc_hpet_quotient, loops);
-
- hpet_start = hpet_readl(HPET_COUNTER);
- do {
- rep_nop();
- hpet_end = hpet_readl(HPET_COUNTER);
- } while ((hpet_end - hpet_start) < (loops));
-}
-
-static struct timer_opts timer_hpet;
-
-static int __init init_hpet(char* override)
-{
- unsigned long result, remain;
-
- /* check clock override */
- if (override[0] && strncmp(override,"hpet",4))
- return -ENODEV;
-
- if (!is_hpet_enabled())
- return -ENODEV;
-
- printk("Using HPET for gettimeofday\n");
- if (cpu_has_tsc) {
- unsigned long tsc_quotient = calibrate_tsc_hpet(&tsc_hpet_quotient);
- if (tsc_quotient) {
- /* report CPU clock rate in Hz.
- * The formula is (10^6 * 2^32) / (2^32 * 1 / (clocks/us)) =
- * clock/second. Our precision is about 100 ppm.
- */
- { unsigned long eax=0, edx=1000;
- ASM_DIV64_REG(cpu_khz, edx, tsc_quotient,
- eax, edx);
- printk("Detected %u.%03u MHz processor.\n",
- cpu_khz / 1000, cpu_khz % 1000);
- }
- set_cyc2ns_scale(cpu_khz);
- }
- /* set this only when cpu_has_tsc */
- timer_hpet.read_timer = read_timer_tsc;
- }
-
- /*
- * Math to calculate hpet to usec multiplier
- * Look for the comments at get_offset_hpet()
- */
- ASM_DIV64_REG(result, remain, hpet_tick, 0, KERNEL_TICK_USEC);
- if (remain > (hpet_tick >> 1))
- result++; /* rounding the result */
- hpet_usec_quotient = result;
-
- return 0;
-}
-
-static int hpet_resume(void)
-{
- write_seqlock(&monotonic_lock);
- /* Assume this is the last mark offset time */
- rdtsc(last_tsc_low, last_tsc_high);
-
- if (hpet_use_timer)
- hpet_last = hpet_readl(HPET_T0_CMP) - hpet_tick;
- else
- hpet_last = hpet_readl(HPET_COUNTER);
- write_sequnlock(&monotonic_lock);
- return 0;
-}
-/************************************************************/
-
-/* tsc timer_opts struct */
-static struct timer_opts timer_hpet __read_mostly = {
- .name = "hpet",
- .mark_offset = mark_offset_hpet,
- .get_offset = get_offset_hpet,
- .monotonic_clock = monotonic_clock_hpet,
- .delay = delay_hpet,
- .resume = hpet_resume,
-};
-
-struct init_timer_opts __initdata timer_hpet_init = {
- .init = init_hpet,
- .opts = &timer_hpet,
-};
diff -ruN tod-mm1/arch/i386/kernel/timers/timer_none.c tod-mm2/arch/i386/kernel/timers/timer_none.c
--- tod-mm1/arch/i386/kernel/timers/timer_none.c 2005-12-01 18:12:56.000000000 -0800
+++ tod-mm2/arch/i386/kernel/timers/timer_none.c 1969-12-31 16:00:00.000000000 -0800
@@ -1,39 +0,0 @@
-#include <linux/init.h>
-#include <asm/timer.h>
-
-static void mark_offset_none(void)
-{
- /* nothing needed */
-}
-
-static unsigned long get_offset_none(void)
-{
- return 0;
-}
-
-static unsigned long long monotonic_clock_none(void)
-{
- return 0;
-}
-
-static void delay_none(unsigned long loops)
-{
- int d0;
- __asm__ __volatile__(
- "\tjmp 1f\n"
- ".align 16\n"
- "1:\tjmp 2f\n"
- ".align 16\n"
- "2:\tdecl %0\n\tjns 2b"
- :"=&a" (d0)
- :"0" (loops));
-}
-
-/* none timer_opts struct */
-struct timer_opts timer_none = {
- .name = "none",
- .mark_offset = mark_offset_none,
- .get_offset = get_offset_none,
- .monotonic_clock = monotonic_clock_none,
- .delay = delay_none,
-};
diff -ruN tod-mm1/arch/i386/kernel/timers/timer_pit.c tod-mm2/arch/i386/kernel/timers/timer_pit.c
--- tod-mm1/arch/i386/kernel/timers/timer_pit.c 2005-12-01 18:24:31.000000000 -0800
+++ tod-mm2/arch/i386/kernel/timers/timer_pit.c 1969-12-31 16:00:00.000000000 -0800
@@ -1,164 +0,0 @@
-/*
- * This code largely moved from arch/i386/kernel/time.c.
- * See comments there for proper credits.
- */
-
-#include <linux/spinlock.h>
-#include <linux/module.h>
-#include <linux/device.h>
-#include <linux/sysdev.h>
-#include <linux/timex.h>
-#include <asm/delay.h>
-#include <asm/mpspec.h>
-#include <asm/timer.h>
-#include <asm/smp.h>
-#include <asm/io.h>
-#include <asm/arch_hooks.h>
-#include <asm/i8253.h>
-
-#include "do_timer.h"
-#include "io_ports.h"
-
-static int count_p; /* counter in get_offset_pit() */
-
-static int __init init_pit(char* override)
-{
- /* check clock override */
- if (override[0] && strncmp(override,"pit",3))
- printk(KERN_ERR "Warning: clock= override failed. Defaulting "
- "to PIT\n");
- init_cpu_khz();
- count_p = LATCH;
- return 0;
-}
-
-static void mark_offset_pit(void)
-{
- /* nothing needed */
-}
-
-static unsigned long long monotonic_clock_pit(void)
-{
- return 0;
-}
-
-static void delay_pit(unsigned long loops)
-{
- int d0;
- __asm__ __volatile__(
- "\tjmp 1f\n"
- ".align 16\n"
- "1:\tjmp 2f\n"
- ".align 16\n"
- "2:\tdecl %0\n\tjns 2b"
- :"=&a" (d0)
- :"0" (loops));
-}
-
-
-/* This function must be called with xtime_lock held.
- * It was inspired by Steve McCanne's microtime-i386 for BSD. -- jrs
- *
- * However, the pc-audio speaker driver changes the divisor so that
- * it gets interrupted rather more often - it loads 64 into the
- * counter rather than 11932! This has an adverse impact on
- * do_gettimeoffset() -- it stops working! What is also not
- * good is that the interval that our timer function gets called
- * is no longer 10.0002 ms, but 9.9767 ms. To get around this
- * would require using a different timing source. Maybe someone
- * could use the RTC - I know that this can interrupt at frequencies
- * ranging from 8192Hz to 2Hz. If I had the energy, I'd somehow fix
- * it so that at startup, the timer code in sched.c would select
- * using either the RTC or the 8253 timer. The decision would be
- * based on whether there was any other device around that needed
- * to trample on the 8253. I'd set up the RTC to interrupt at 1024 Hz,
- * and then do some jiggery to have a version of do_timer that
- * advanced the clock by 1/1024 s. Every time that reached over 1/100
- * of a second, then do all the old code. If the time was kept correct
- * then do_gettimeoffset could just return 0 - there is no low order
- * divider that can be accessed.
- *
- * Ideally, you would be able to use the RTC for the speaker driver,
- * but it appears that the speaker driver really needs interrupt more
- * often than every 120 us or so.
- *
- * Anyway, this needs more thought.... pjsg (1993-08-28)
- *
- * If you are really that interested, you should be reading
- * comp.protocols.time.ntp!
- */
-
-static unsigned long get_offset_pit(void)
-{
- int count;
- unsigned long flags;
- static unsigned long jiffies_p = 0;
-
- /*
- * cache volatile jiffies temporarily; we have xtime_lock.
- */
- unsigned long jiffies_t;
-
- spin_lock_irqsave(&i8253_lock, flags);
- /* timer count may underflow right here */
- outb_p(0x00, PIT_MODE); /* latch the count ASAP */
-
- count = inb_p(PIT_CH0); /* read the latched count */
-
- /*
- * We do this guaranteed double memory access instead of a _p
- * postfix in the previous port access. Wheee, hackady hack
- */
- jiffies_t = jiffies;
-
- count |= inb_p(PIT_CH0) << 8;
-
- /* VIA686a test code... reset the latch if count > max + 1 */
- if (count > LATCH) {
- outb_p(0x34, PIT_MODE);
- outb_p(LATCH & 0xff, PIT_CH0);
- outb(LATCH >> 8, PIT_CH0);
- count = LATCH - 1;
- }
-
- /*
- * avoiding timer inconsistencies (they are rare, but they happen)...
- * there are two kinds of problems that must be avoided here:
- * 1. the timer counter underflows
- * 2. hardware problem with the timer, not giving us continuous time,
- * the counter does small "jumps" upwards on some Pentium systems,
- * (see c't 95/10 page 335 for Neptun bug.)
- */
-
- if( jiffies_t == jiffies_p ) {
- if( count > count_p ) {
- /* the nutcase */
- count = do_timer_overflow(count);
- }
- } else
- jiffies_p = jiffies_t;
-
- count_p = count;
-
- spin_unlock_irqrestore(&i8253_lock, flags);
-
- count = ((LATCH-1) - count) * TICK_SIZE;
- count = (count + LATCH/2) / LATCH;
-
- return count;
-}
-
-
-/* tsc timer_opts struct */
-struct timer_opts timer_pit = {
- .name = "pit",
- .mark_offset = mark_offset_pit,
- .get_offset = get_offset_pit,
- .monotonic_clock = monotonic_clock_pit,
- .delay = delay_pit,
-};
-
-struct init_timer_opts __initdata timer_pit_init = {
- .init = init_pit,
- .opts = &timer_pit,
-};
diff -ruN tod-mm1/arch/i386/kernel/timers/timer_pm.c tod-mm2/arch/i386/kernel/timers/timer_pm.c
--- tod-mm1/arch/i386/kernel/timers/timer_pm.c 2005-12-01 18:12:56.000000000 -0800
+++ tod-mm2/arch/i386/kernel/timers/timer_pm.c 1969-12-31 16:00:00.000000000 -0800
@@ -1,268 +0,0 @@
-/*
- * (C) Dominik Brodowski <[email protected]> 2003
- *
- * Driver to use the Power Management Timer (PMTMR) available in some
- * southbridges as primary timing source for the Linux kernel.
- *
- * Based on parts of linux/drivers/acpi/hardware/hwtimer.c, timer_pit.c,
- * timer_hpet.c, and on Arjan van de Ven's implementation for 2.4.
- *
- * This file is licensed under the GPL v2.
- */
-
-
-#include <linux/kernel.h>
-#include <linux/module.h>
-#include <linux/device.h>
-#include <linux/init.h>
-#include <asm/types.h>
-#include <asm/timer.h>
-#include <asm/smp.h>
-#include <asm/io.h>
-#include <asm/arch_hooks.h>
-
-#include <linux/timex.h>
-#include "mach_timer.h"
-
-/* Number of PMTMR ticks expected during calibration run */
-#define PMTMR_TICKS_PER_SEC 3579545
-#define PMTMR_EXPECTED_RATE \
- ((CALIBRATE_LATCH * (PMTMR_TICKS_PER_SEC >> 10)) / (CLOCK_TICK_RATE>>10))
-
-
-/* The I/O port the PMTMR resides at.
- * The location is detected during setup_arch(),
- * in arch/i386/acpi/boot.c */
-u32 pmtmr_ioport = 0;
-
-
-/* value of the Power timer at last timer interrupt */
-static u32 offset_tick;
-static u32 offset_delay;
-
-static unsigned long long monotonic_base;
-static seqlock_t monotonic_lock = SEQLOCK_UNLOCKED;
-
-#define ACPI_PM_MASK 0xFFFFFF /* limit it to 24 bits */
-
-/*helper function to safely read acpi pm timesource*/
-static inline u32 read_pmtmr(void)
-{
- u32 v1=0,v2=0,v3=0;
- /* It has been reported that because of various broken
- * chipsets (ICH4, PIIX4 and PIIX4E) where the ACPI PM time
- * source is not latched, so you must read it multiple
- * times to insure a safe value is read.
- */
- do {
- v1 = inl(pmtmr_ioport);
- v2 = inl(pmtmr_ioport);
- v3 = inl(pmtmr_ioport);
- } while ((v1 > v2 && v1 < v3) || (v2 > v3 && v2 < v1)
- || (v3 > v1 && v3 < v2));
-
- /* mask the output to 24 bits */
- return v2 & ACPI_PM_MASK;
-}
-
-
-/*
- * Some boards have the PMTMR running way too fast. We check
- * the PMTMR rate against PIT channel 2 to catch these cases.
- */
-static int verify_pmtmr_rate(void)
-{
- u32 value1, value2;
- unsigned long count, delta;
-
- mach_prepare_counter();
- value1 = read_pmtmr();
- mach_countup(&count);
- value2 = read_pmtmr();
- delta = (value2 - value1) & ACPI_PM_MASK;
-
- /* Check that the PMTMR delta is within 5% of what we expect */
- if (delta < (PMTMR_EXPECTED_RATE * 19) / 20 ||
- delta > (PMTMR_EXPECTED_RATE * 21) / 20) {
- printk(KERN_INFO "PM-Timer running at invalid rate: %lu%% of normal - aborting.\n", 100UL * delta / PMTMR_EXPECTED_RATE);
- return -1;
- }
-
- return 0;
-}
-
-
-static int init_pmtmr(char* override)
-{
- u32 value1, value2;
- unsigned int i;
-
- if (override[0] && strncmp(override,"pmtmr",5))
- return -ENODEV;
-
- if (!pmtmr_ioport)
- return -ENODEV;
-
- /* we use the TSC for delay_pmtmr, so make sure it exists */
- if (!cpu_has_tsc)
- return -ENODEV;
-
- /* "verify" this timing source */
- value1 = read_pmtmr();
- for (i = 0; i < 10000; i++) {
- value2 = read_pmtmr();
- if (value2 == value1)
- continue;
- if (value2 > value1)
- goto pm_good;
- if ((value2 < value1) && ((value2) < 0xFFF))
- goto pm_good;
- printk(KERN_INFO "PM-Timer had inconsistent results: 0x%#x, 0x%#x - aborting.\n", value1, value2);
- return -EINVAL;
- }
- printk(KERN_INFO "PM-Timer had no reasonable result: 0x%#x - aborting.\n", value1);
- return -ENODEV;
-
-pm_good:
- if (verify_pmtmr_rate() != 0)
- return -ENODEV;
-
- init_cpu_khz();
- return 0;
-}
-
-static inline u32 cyc2us(u32 cycles)
-{
- /* The Power Management Timer ticks at 3.579545 ticks per microsecond.
- * 1 / PM_TIMER_FREQUENCY == 0.27936511 =~ 286/1024 [error: 0.024%]
- *
- * Even with HZ = 100, delta is at maximum 35796 ticks, so it can
- * easily be multiplied with 286 (=0x11E) without having to fear
- * u32 overflows.
- */
- cycles *= 286;
- return (cycles >> 10);
-}
-
-/*
- * this gets called during each timer interrupt
- * - Called while holding the writer xtime_lock
- */
-static void mark_offset_pmtmr(void)
-{
- u32 lost, delta, last_offset;
- static int first_run = 1;
- last_offset = offset_tick;
-
- write_seqlock(&monotonic_lock);
-
- offset_tick = read_pmtmr();
-
- /* calculate tick interval */
- delta = (offset_tick - last_offset) & ACPI_PM_MASK;
-
- /* convert to usecs */
- delta = cyc2us(delta);
-
- /* update the monotonic base value */
- monotonic_base += delta * NSEC_PER_USEC;
- write_sequnlock(&monotonic_lock);
-
- /* convert to ticks */
- delta += offset_delay;
- lost = delta / (USEC_PER_SEC / HZ);
- offset_delay = delta % (USEC_PER_SEC / HZ);
-
-
- /* compensate for lost ticks */
- if (lost >= 2)
- jiffies_64 += lost - 1;
-
- /* don't calculate delay for first run,
- or if we've got less then a tick */
- if (first_run || (lost < 1)) {
- first_run = 0;
- offset_delay = 0;
- }
-}
-
-static int pmtmr_resume(void)
-{
- write_seqlock(&monotonic_lock);
- /* Assume this is the last mark offset time */
- offset_tick = read_pmtmr();
- write_sequnlock(&monotonic_lock);
- return 0;
-}
-
-static unsigned long long monotonic_clock_pmtmr(void)
-{
- u32 last_offset, this_offset;
- unsigned long long base, ret;
- unsigned seq;
-
-
- /* atomically read monotonic base & last_offset */
- do {
- seq = read_seqbegin(&monotonic_lock);
- last_offset = offset_tick;
- base = monotonic_base;
- } while (read_seqretry(&monotonic_lock, seq));
-
- /* Read the pmtmr */
- this_offset = read_pmtmr();
-
- /* convert to nanoseconds */
- ret = (this_offset - last_offset) & ACPI_PM_MASK;
- ret = base + (cyc2us(ret) * NSEC_PER_USEC);
- return ret;
-}
-
-static void delay_pmtmr(unsigned long loops)
-{
- unsigned long bclock, now;
-
- rdtscl(bclock);
- do
- {
- rep_nop();
- rdtscl(now);
- } while ((now-bclock) < loops);
-}
-
-
-/*
- * get the offset (in microseconds) from the last call to mark_offset()
- * - Called holding a reader xtime_lock
- */
-static unsigned long get_offset_pmtmr(void)
-{
- u32 now, offset, delta = 0;
-
- offset = offset_tick;
- now = read_pmtmr();
- delta = (now - offset)&ACPI_PM_MASK;
-
- return (unsigned long) offset_delay + cyc2us(delta);
-}
-
-
-/* acpi timer_opts struct */
-static struct timer_opts timer_pmtmr = {
- .name = "pmtmr",
- .mark_offset = mark_offset_pmtmr,
- .get_offset = get_offset_pmtmr,
- .monotonic_clock = monotonic_clock_pmtmr,
- .delay = delay_pmtmr,
- .read_timer = read_timer_tsc,
- .resume = pmtmr_resume,
-};
-
-struct init_timer_opts __initdata timer_pmtmr_init = {
- .init = init_pmtmr,
- .opts = &timer_pmtmr,
-};
-
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Dominik Brodowski <[email protected]>");
-MODULE_DESCRIPTION("Power Management Timer (PMTMR) as primary timing source for x86");
diff -ruN tod-mm1/arch/i386/kernel/timers/timer_tsc.c tod-mm2/arch/i386/kernel/timers/timer_tsc.c
--- tod-mm1/arch/i386/kernel/timers/timer_tsc.c 2005-12-01 18:25:04.000000000 -0800
+++ tod-mm2/arch/i386/kernel/timers/timer_tsc.c 1969-12-31 16:00:00.000000000 -0800
@@ -1,388 +0,0 @@
-/*
- * This code largely moved from arch/i386/kernel/time.c.
- * See comments there for proper credits.
- *
- * 2004-06-25 Jesper Juhl
- * moved mark_offset_tsc below cpufreq_delayed_get to avoid gcc 3.4
- * failing to inline.
- */
-
-#include <linux/spinlock.h>
-#include <linux/init.h>
-#include <linux/timex.h>
-#include <linux/errno.h>
-#include <linux/cpufreq.h>
-#include <linux/string.h>
-#include <linux/jiffies.h>
-
-#include <asm/timer.h>
-#include <asm/io.h>
-/* processor.h for distable_tsc flag */
-#include <asm/processor.h>
-
-#include "io_ports.h"
-#include "mach_timer.h"
-
-#include <asm/hpet.h>
-#include <asm/i8253.h>
-
-#ifdef CONFIG_HPET_TIMER
-static unsigned long hpet_usec_quotient;
-static unsigned long hpet_last;
-static struct timer_opts timer_tsc;
-#endif
-
-static int use_tsc;
-/* Number of usecs that the last interrupt was delayed */
-static int delay_at_last_interrupt;
-
-static unsigned long last_tsc_low; /* lsb 32 bits of Time Stamp Counter */
-static unsigned long last_tsc_high; /* msb 32 bits of Time Stamp Counter */
-static unsigned long long monotonic_base;
-static seqlock_t monotonic_lock = SEQLOCK_UNLOCKED;
-
-static int count2; /* counter for mark_offset_tsc() */
-
-/* Cached *multiplier* to convert TSC counts to microseconds.
- * (see the equation below).
- * Equal to 2^32 * (1 / (clocks per usec) ).
- * Initialized in time_init.
- */
-static unsigned long fast_gettimeoffset_quotient;
-
-static unsigned long get_offset_tsc(void)
-{
- register unsigned long eax, edx;
-
- /* Read the Time Stamp Counter */
-
- rdtsc(eax,edx);
-
- /* .. relative to previous jiffy (32 bits is enough) */
- eax -= last_tsc_low; /* tsc_low delta */
-
- /*
- * Time offset = (tsc_low delta) * fast_gettimeoffset_quotient
- * = (tsc_low delta) * (usecs_per_clock)
- * = (tsc_low delta) * (usecs_per_jiffy / clocks_per_jiffy)
- *
- * Using a mull instead of a divl saves up to 31 clock cycles
- * in the critical path.
- */
-
- __asm__("mull %2"
- :"=a" (eax), "=d" (edx)
- :"rm" (fast_gettimeoffset_quotient),
- "0" (eax));
-
- /* our adjusted time offset in microseconds */
- return delay_at_last_interrupt + edx;
-}
-
-static unsigned long long monotonic_clock_tsc(void)
-{
- unsigned long long last_offset, this_offset, base;
- unsigned seq;
-
- /* atomically read monotonic base & last_offset */
- do {
- seq = read_seqbegin(&monotonic_lock);
- last_offset = ((unsigned long long)last_tsc_high<<32)|last_tsc_low;
- base = monotonic_base;
- } while (read_seqretry(&monotonic_lock, seq));
-
- /* Read the Time Stamp Counter */
- rdtscll(this_offset);
-
- /* return the value in ns */
- return base + cycles_2_ns(this_offset - last_offset);
-}
-
-
-static void delay_tsc(unsigned long loops)
-{
- unsigned long bclock, now;
-
- rdtscl(bclock);
- do
- {
- rep_nop();
- rdtscl(now);
- } while ((now-bclock) < loops);
-}
-
-#ifdef CONFIG_HPET_TIMER
-static void mark_offset_tsc_hpet(void)
-{
- unsigned long long this_offset, last_offset;
- unsigned long offset, temp, hpet_current;
-
- write_seqlock(&monotonic_lock);
- last_offset = ((unsigned long long)last_tsc_high<<32)|last_tsc_low;
- /*
- * It is important that these two operations happen almost at
- * the same time. We do the RDTSC stuff first, since it's
- * faster. To avoid any inconsistencies, we need interrupts
- * disabled locally.
- */
- /*
- * Interrupts are just disabled locally since the timer irq
- * has the SA_INTERRUPT flag set. -arca
- */
- /* read Pentium cycle counter */
-
- hpet_current = hpet_readl(HPET_COUNTER);
- rdtsc(last_tsc_low, last_tsc_high);
-
- /* lost tick compensation */
- offset = hpet_readl(HPET_T0_CMP) - hpet_tick;
- if (unlikely(((offset - hpet_last) > hpet_tick) && (hpet_last != 0))) {
- int lost_ticks = (offset - hpet_last) / hpet_tick;
- jiffies_64 += lost_ticks;
- }
- hpet_last = hpet_current;
-
- /* update the monotonic base value */
- this_offset = ((unsigned long long)last_tsc_high<<32)|last_tsc_low;
- monotonic_base += cycles_2_ns(this_offset - last_offset);
- write_sequnlock(&monotonic_lock);
-
- /* calculate delay_at_last_interrupt */
- /*
- * Time offset = (hpet delta) * ( usecs per HPET clock )
- * = (hpet delta) * ( usecs per tick / HPET clocks per tick)
- * = (hpet delta) * ( hpet_usec_quotient ) / (2^32)
- * Where,
- * hpet_usec_quotient = (2^32 * usecs per tick)/HPET clocks per tick
- */
- delay_at_last_interrupt = hpet_current - offset;
- ASM_MUL64_REG(temp, delay_at_last_interrupt,
- hpet_usec_quotient, delay_at_last_interrupt);
-}
-#endif
-
-
-
-static void mark_offset_tsc(void)
-{
- unsigned long lost,delay;
- unsigned long delta = last_tsc_low;
- int count;
- int countmp;
- static int count1 = 0;
- unsigned long long this_offset, last_offset;
- static int lost_count = 0;
-
- write_seqlock(&monotonic_lock);
- last_offset = ((unsigned long long)last_tsc_high<<32)|last_tsc_low;
- /*
- * It is important that these two operations happen almost at
- * the same time. We do the RDTSC stuff first, since it's
- * faster. To avoid any inconsistencies, we need interrupts
- * disabled locally.
- */
-
- /*
- * Interrupts are just disabled locally since the timer irq
- * has the SA_INTERRUPT flag set. -arca
- */
-
- /* read Pentium cycle counter */
-
- rdtsc(last_tsc_low, last_tsc_high);
-
- spin_lock(&i8253_lock);
- outb_p(0x00, PIT_MODE); /* latch the count ASAP */
-
- count = inb_p(PIT_CH0); /* read the latched count */
- count |= inb(PIT_CH0) << 8;
-
- /*
- * VIA686a test code... reset the latch if count > max + 1
- * from timer_pit.c - cjb
- */
- if (count > LATCH) {
- outb_p(0x34, PIT_MODE);
- outb_p(LATCH & 0xff, PIT_CH0);
- outb(LATCH >> 8, PIT_CH0);
- count = LATCH - 1;
- }
-
- spin_unlock(&i8253_lock);
-
- if (pit_latch_buggy) {
- /* get center value of last 3 time lutch */
- if ((count2 >= count && count >= count1)
- || (count1 >= count && count >= count2)) {
- count2 = count1; count1 = count;
- } else if ((count1 >= count2 && count2 >= count)
- || (count >= count2 && count2 >= count1)) {
- countmp = count;count = count2;
- count2 = count1;count1 = countmp;
- } else {
- count2 = count1; count1 = count; count = count1;
- }
- }
-
- /* lost tick compensation */
- delta = last_tsc_low - delta;
- {
- register unsigned long eax, edx;
- eax = delta;
- __asm__("mull %2"
- :"=a" (eax), "=d" (edx)
- :"rm" (fast_gettimeoffset_quotient),
- "0" (eax));
- delta = edx;
- }
- delta += delay_at_last_interrupt;
- lost = delta/(1000000/HZ);
- delay = delta%(1000000/HZ);
- if (lost >= 2) {
- jiffies_64 += lost-1;
-
- /* sanity check to ensure we're not always losing ticks */
- if (lost_count++ > 100) {
- printk(KERN_WARNING "Losing too many ticks!\n");
- printk(KERN_WARNING "TSC cannot be used as a timesource. \n");
- printk(KERN_WARNING "Possible reasons for this are:\n");
- printk(KERN_WARNING " You're running with Speedstep,\n");
- printk(KERN_WARNING " You don't have DMA enabled for your hard disk (see hdparm),\n");
- printk(KERN_WARNING " Incorrect TSC synchronization on an SMP system (see dmesg).\n");
- printk(KERN_WARNING "Falling back to a sane timesource now.\n");
-
- clock_fallback();
- }
- /* ... but give the TSC a fair chance */
- if (lost_count > 25)
- cpufreq_delayed_get();
- } else
- lost_count = 0;
- /* update the monotonic base value */
- this_offset = ((unsigned long long)last_tsc_high<<32)|last_tsc_low;
- monotonic_base += cycles_2_ns(this_offset - last_offset);
- write_sequnlock(&monotonic_lock);
-
- /* calculate delay_at_last_interrupt */
- count = ((LATCH-1) - count) * TICK_SIZE;
- delay_at_last_interrupt = (count + LATCH/2) / LATCH;
-
- /* catch corner case where tick rollover occured
- * between tsc and pit reads (as noted when
- * usec delta is > 90% # of usecs/tick)
- */
- if (lost && abs(delay - delay_at_last_interrupt) > (900000/HZ))
- jiffies_64++;
-}
-
-static int __init init_tsc(char* override)
-{
-
- /* check clock override */
- if (override[0] && strncmp(override,"tsc",3)) {
-#ifdef CONFIG_HPET_TIMER
- if (is_hpet_enabled()) {
- printk(KERN_ERR "Warning: clock= override failed. Defaulting to tsc\n");
- } else
-#endif
- {
- return -ENODEV;
- }
- }
-
- /*
- * If we have APM enabled or the CPU clock speed is variable
- * (CPU stops clock on HLT or slows clock to save power)
- * then the TSC timestamps may diverge by up to 1 jiffy from
- * 'real time' but nothing will break.
- * The most frequent case is that the CPU is "woken" from a halt
- * state by the timer interrupt itself, so we get 0 error. In the
- * rare cases where a driver would "wake" the CPU and request a
- * timestamp, the maximum error is < 1 jiffy. But timestamps are
- * still perfectly ordered.
- * Note that the TSC counter will be reset if APM suspends
- * to disk; this won't break the kernel, though, 'cuz we're
- * smart. See arch/i386/kernel/apm.c.
- */
- /*
- * Firstly we have to do a CPU check for chips with
- * a potentially buggy TSC. At this point we haven't run
- * the ident/bugs checks so we must run this hook as it
- * may turn off the TSC flag.
- *
- * NOTE: this doesn't yet handle SMP 486 machines where only
- * some CPU's have a TSC. Thats never worked and nobody has
- * moaned if you have the only one in the world - you fix it!
- */
-
- count2 = LATCH; /* initialize counter for mark_offset_tsc() */
-
- if (cpu_has_tsc) {
- unsigned long tsc_quotient;
-#ifdef CONFIG_HPET_TIMER
- if (is_hpet_enabled() && hpet_use_timer) {
- unsigned long result, remain;
- printk("Using TSC for gettimeofday\n");
- tsc_quotient = calibrate_tsc_hpet(NULL);
- timer_tsc.mark_offset = &mark_offset_tsc_hpet;
- /*
- * Math to calculate hpet to usec multiplier
- * Look for the comments at get_offset_tsc_hpet()
- */
- ASM_DIV64_REG(result, remain, hpet_tick,
- 0, KERNEL_TICK_USEC);
- if (remain > (hpet_tick >> 1))
- result++; /* rounding the result */
-
- hpet_usec_quotient = result;
- } else
-#endif
- {
- tsc_quotient = calibrate_tsc();
- }
-
- if (tsc_quotient) {
- fast_gettimeoffset_quotient = tsc_quotient;
- use_tsc = 1;
- /*
- * We could be more selective here I suspect
- * and just enable this for the next intel chips ?
- */
- /* report CPU clock rate in Hz.
- * The formula is (10^6 * 2^32) / (2^32 * 1 / (clocks/us)) =
- * clock/second. Our precision is about 100 ppm.
- */
- { unsigned long eax=0, edx=1000;
- __asm__("divl %2"
- :"=a" (cpu_khz), "=d" (edx)
- :"r" (tsc_quotient),
- "0" (eax), "1" (edx));
- printk("Detected %u.%03u MHz processor.\n",
- cpu_khz / 1000, cpu_khz % 1000);
- }
- set_cyc2ns_scale(cpu_khz);
- return 0;
- }
- }
- return -ENODEV;
-}
-
-
-
-/************************************************************/
-
-/* tsc timer_opts struct */
-static struct timer_opts timer_tsc = {
- .name = "tsc",
- .mark_offset = mark_offset_tsc,
- .get_offset = get_offset_tsc,
- .monotonic_clock = monotonic_clock_tsc,
- .delay = delay_tsc,
- .read_timer = read_timer_tsc,
- .resume = tsc_resume,
-};
-
-struct init_timer_opts __initdata timer_tsc_init = {
- .init = init_tsc,
- .opts = &timer_tsc,
-};
john stultz wrote:
> All,
> Here is the second of two patches which try to minimize my ntp rework
> patches.
>
> This patch further changes the interrupt time NTP code, breaking out the
> leapsecond processing and introduces an accessor to a shifted ppm
In a discusson aroung the leapsecond and how to disable it (some folks
don't want the time jump) it came to light that, for the most part,
this is unused code. It requires that time be kept in UST to be
useful and, from what I can tell, most folks keep time in their local
timezone, thus, effectively, disableing the usage of the leapsecond
correction (ntp figures this out and just says "no"). Possibly it is
time to ask if we should keep this in the kernel at all.
--
George Anzinger [email protected]
HRT (High-res-timers): http://sourceforge.net/projects/high-res-timers/
On Fri, 2005-12-02 at 16:19 -0800, George Anzinger wrote:
> john stultz wrote:
> > All,
> > Here is the second of two patches which try to minimize my ntp rework
> > patches.
> >
> > This patch further changes the interrupt time NTP code, breaking out the
> > leapsecond processing and introduces an accessor to a shifted ppm
>
> In a discusson aroung the leapsecond and how to disable it (some folks
> don't want the time jump) it came to light that, for the most part,
> this is unused code. It requires that time be kept in UST to be
> useful and, from what I can tell, most folks keep time in their local
> timezone, thus, effectively, disableing the usage of the leapsecond
> correction (ntp figures this out and just says "no"). Possibly it is
> time to ask if we should keep this in the kernel at all.
Well, I'm trying explicitly to not change the kernel's current NTP
behavior. Instead I'm trying to allow the current NTP behavior to apply
to a continuous clocksource to avoid interpolation error and other
problems with irregular timer interrupts.
Although Its an interesting point, and I'm not opposed to discussing it,
its just not really relevant to the patch. So you might want to start a
new thread.
Also, I think the debate has been brought up a number of times on
comp.protocols.time.ntp, so you might want to check there first.
thanks
-john
On 2 Dec 2005 at 16:19, George Anzinger wrote:
> john stultz wrote:
> > All,
> > Here is the second of two patches which try to minimize my ntp rework
> > patches.
> >
> > This patch further changes the interrupt time NTP code, breaking out the
> > leapsecond processing and introduces an accessor to a shifted ppm
>
> In a discusson aroung the leapsecond and how to disable it (some folks
> don't want the time jump) it came to light that, for the most part,
> this is unused code. It requires that time be kept in UST to be
> useful and, from what I can tell, most folks keep time in their local
> timezone, thus, effectively, disableing the usage of the leapsecond
> correction (ntp figures this out and just says "no"). Possibly it is
> time to ask if we should keep this in the kernel at all.
I think this is not a question at all whether people like leap seconds or not:
Either they want to have the current official time, or they do not. If they do
not, they won't care about NTP; if they do they'd use it.
If they don't like leap seconds, they'd go into politics to forbid them by law.
Regards,
Ulrich
Hi,
On Fri, 2 Dec 2005, George Anzinger wrote:
> In a discusson aroung the leapsecond and how to disable it (some folks don't
> want the time jump) it came to light that, for the most part, this is unused
> code. It requires that time be kept in UST to be useful and, from what I can
> tell, most folks keep time in their local timezone, thus, effectively,
> disableing the usage of the leapsecond correction (ntp figures this out and
> just says "no"). Possibly it is time to ask if we should keep this in the
> kernel at all.
I'm thinking about moving the leap second handling to a timer, with the
new timer system it would be easy to set a timer for e.g. 23:59.59 and
then set the time. This way it would be gone from the common path and it
wouldn't matter that much anymore whether it's used or not.
bye, Roman
On 5 Dec 2005 at 11:35, Roman Zippel wrote:
> Hi,
>
> On Fri, 2 Dec 2005, George Anzinger wrote:
>
> > In a discusson aroung the leapsecond and how to disable it (some folks don't
> > want the time jump) it came to light that, for the most part, this is unused
> > code. It requires that time be kept in UST to be useful and, from what I can
> > tell, most folks keep time in their local timezone, thus, effectively,
> > disableing the usage of the leapsecond correction (ntp figures this out and
> > just says "no"). Possibly it is time to ask if we should keep this in the
> > kernel at all.
>
> I'm thinking about moving the leap second handling to a timer, with the
> new timer system it would be easy to set a timer for e.g. 23:59.59 and
> then set the time. This way it would be gone from the common path and it
> wouldn't matter that much anymore whether it's used or not.
Will the timer solution guarantee consistent and exact updates?
Regards,
Ulrich
* Ulrich Windl <[email protected]> wrote:
> > I'm thinking about moving the leap second handling to a timer, with the
> > new timer system it would be easy to set a timer for e.g. 23:59.59 and
> > then set the time. This way it would be gone from the common path and it
> > wouldn't matter that much anymore whether it's used or not.
>
> Will the timer solution guarantee consistent and exact updates?
it would still be dependent on system-load situations. It's an
interesting idea to use a timer for that, but there is no strict
synchronization between "get time of day" and "timer execution", so any
timer-based leap-second handling would be fundamentally asynchronous. I
dont think we want that, leap second handling should be a synchronous
property of 'time'.
i think the very first step should be the cleanups i did to the NTP
portions of timer.c. That made all the code (including leap second
handling) more readable. I think a portion of the inner desire to
rewrite the NTP code comes from the current spaghetti that accumulated
over the years.
Ingo
Hi,
On Tue, 6 Dec 2005, Ingo Molnar wrote:
> > > I'm thinking about moving the leap second handling to a timer, with the
> > > new timer system it would be easy to set a timer for e.g. 23:59.59 and
> > > then set the time. This way it would be gone from the common path and it
> > > wouldn't matter that much anymore whether it's used or not.
> >
> > Will the timer solution guarantee consistent and exact updates?
>
> it would still be dependent on system-load situations.
Interrupt-load, actually.
> It's an
> interesting idea to use a timer for that, but there is no strict
> synchronization between "get time of day" and "timer execution", so any
> timer-based leap-second handling would be fundamentally asynchronous. I
> dont think we want that, leap second handling should be a synchronous
> property of 'time'.
I'm not really sure what you're talking about. Could you please elaborate
on "fundamentally asynchronous" and "synchronous property of 'time'"?
bye, Roman
* Roman Zippel <[email protected]> wrote:
> On Tue, 6 Dec 2005, Ingo Molnar wrote:
>
> > > > I'm thinking about moving the leap second handling to a timer, with the
> > > > new timer system it would be easy to set a timer for e.g. 23:59.59 and
> > > > then set the time. This way it would be gone from the common path and it
> > > > wouldn't matter that much anymore whether it's used or not.
> > >
> > > Will the timer solution guarantee consistent and exact updates?
> >
> > it would still be dependent on system-load situations.
>
> Interrupt-load, actually.
yeah. To a smaller degree it would be dependent on generic system-load
too. E.g. if some code keeps interrupts disabled for too long. But the
main delay potential is from other interrupts, or from having too many
timers to process.
> > It's an
> > interesting idea to use a timer for that, but there is no strict
> > synchronization between "get time of day" and "timer execution", so any
> > timer-based leap-second handling would be fundamentally asynchronous. I
> > dont think we want that, leap second handling should be a synchronous
> > property of 'time'.
>
> I'm not really sure what you're talking about. [...]
doh, it was too early in the morning. Of course xtime is driven by
interrupts just as much, so time updates are already 'asynchronous' and
subject to interrupt delays. So your idea is perfectly fine.
Ingo
Ulrich Windl wrote:
> On 2 Dec 2005 at 16:19, George Anzinger wrote:
>
>
>>john stultz wrote:
>>
>>>All,
>>> Here is the second of two patches which try to minimize my ntp rework
>>>patches.
>>>
>>>This patch further changes the interrupt time NTP code, breaking out the
>>>leapsecond processing and introduces an accessor to a shifted ppm
>>
>>In a discusson aroung the leapsecond and how to disable it (some folks
>>don't want the time jump) it came to light that, for the most part,
>>this is unused code. It requires that time be kept in UST to be
>>useful and, from what I can tell, most folks keep time in their local
>>timezone, thus, effectively, disableing the usage of the leapsecond
>>correction (ntp figures this out and just says "no"). Possibly it is
>>time to ask if we should keep this in the kernel at all.
>
>
> I think this is not a question at all whether people like leap seconds or not:
> Either they want to have the current official time, or they do not. If they do
> not, they won't care about NTP; if they do they'd use it.
>
> If they don't like leap seconds, they'd go into politics to forbid them by law.
I don't think that is what happens now. Rather the leapsecond is not
requested by ntp and either a) ntp sets the clock at the required time
or b) it "creeps" it ahead or back by one second over a somewhat
longer time. It is behavior b) that I have found some folks want. In
no case do I see anyone wanting to drop the leapsecond, they just
don't want the discontinuity it introduces and are willing to be a
second (or if done properly, half a second) away from the correct time
for a period of time around the official leapsecond.
--
George Anzinger [email protected]
HRT (High-res-timers): http://sourceforge.net/projects/high-res-timers/
On 6 Dec 2005 at 11:35, Roman Zippel wrote:
> Hi,
>
> On Tue, 6 Dec 2005, Ingo Molnar wrote:
>
> > > > I'm thinking about moving the leap second handling to a timer, with the
> > > > new timer system it would be easy to set a timer for e.g. 23:59.59 and
> > > > then set the time. This way it would be gone from the common path and it
> > > > wouldn't matter that much anymore whether it's used or not.
> > >
> > > Will the timer solution guarantee consistent and exact updates?
> >
> > it would still be dependent on system-load situations.
>
> Interrupt-load, actually.
>
> > It's an
> > interesting idea to use a timer for that, but there is no strict
> > synchronization between "get time of day" and "timer execution", so any
> > timer-based leap-second handling would be fundamentally asynchronous. I
> > dont think we want that, leap second handling should be a synchronous
> > property of 'time'.
>
> I'm not really sure what you're talking about. Could you please elaborate
> on "fundamentally asynchronous" and "synchronous property of 'time'"?
It's always the same: A process busily reads time, and it wants to have it smooth
(low jitter, preferrably constant jitter, small time increments):
/*
* This program can be used to calibrate the clock reading jitter of a
* particular CPU and operating system. It first tickles every element
* of an array, in order to force pages into memory, then repeatedly calls
* gettimeofday() and, finally, writes out the time values for later
* analysis. From this you can determine the jitter and if the clock ever
* runs backwards.
*/
#include <sys/time.h>
#include <stdio.h>
#define NBUF 20002
void
main()
{
struct timeval ts, tr;
struct timezone tzp;
long temp, j, i, gtod[NBUF];
gettimeofday(&ts, &tzp);
/*
* Force pages into memory
*/
for (i = 0; i < NBUF; i ++)
gtod[i] = 0;
/*
* Construct gtod array
*/
for (i = 0; i < NBUF; i ++) {
gettimeofday(&tr, &tzp);
gtod[i] = (tr.tv_sec - ts.tv_sec) * 1000000 + tr.tv_usec;
}
/*
* Write out gtod array for later processing with S
*/
for (i = 0; i < NBUF - 2; i++) {
/*
printf("%lu\n", gtod[i]);
*/
gtod[i] = gtod[i + 1] - gtod[i];
printf("%lu\n", gtod[i]);
}
/*
* Sort the gtod array and display deciles
*/
for (i = 0; i < NBUF - 2; i++) {
for (j = 0; j <= i; j++) {
if (gtod[j] > gtod[i]) {
temp = gtod[j];
gtod[j] = gtod[i];
gtod[i] = temp;
}
}
}
fprintf(stderr, "First rank\n");
for (i = 0; i < 10; i++)
fprintf(stderr, "%10ld%10ld\n", i, gtod[i]);
fprintf(stderr, "Last rank\n");
for (i = NBUF - 12; i < NBUF - 2; i++)
fprintf(stderr, "%10ld%10ld\n", i, gtod[i]);
}
(Code taken from some 10 year old xntp source tree)
On 6 Dec 2005 at 12:26, George Anzinger wrote:
[...]
> > I think this is not a question at all whether people like leap seconds or not:
> > Either they want to have the current official time, or they do not. If they do
> > not, they won't care about NTP; if they do they'd use it.
> >
> > If they don't like leap seconds, they'd go into politics to forbid them by law.
>
> I don't think that is what happens now. Rather the leapsecond is not
Be aware of the following: If a recent NTP daemon detects an old NTP kernel
interface (like that of unpatched Linux), it will not use the NTP kernel interface
at all ("disable kernel"). Thus no leap second processing will happen (AFAIK).
> requested by ntp and either a) ntp sets the clock at the required time
> or b) it "creeps" it ahead or back by one second over a somewhat
> longer time. It is behavior b) that I have found some folks want. In
> no case do I see anyone wanting to drop the leapsecond, they just
> don't want the discontinuity it introduces and are willing to be a
> second (or if done properly, half a second) away from the correct time
> for a period of time around the official leapsecond.
Fight the real problem then: Update the kernel NTP interface to the current one
(which is several years old by now). It's named "nanokernel" by Dave Mills, and it
was exactly the reason why I had patched Linux 2.4 to provide a nanosecond clock
resolution (in theory), and I also thought that's the reason for having
nanoseconds with your patch set ;-)
Regards,
Ulrich