This patch series adds X86_FEATURE_TSC_KNOWN_FREQ flag and modifies
Intel Atom SoC related TSC codes to use the new TSC_KNOWN_FREQ flag
and the existed TSC_RELIABLE flag.
Bin Gao (4):
x86/tsc: add X86_FEATURE_TSC_KNOWN_FREQ flag
x86/tsc: mark TSC frequency determined by CPUID as known
x86/tsc: mark Intel ATOM_GOLDMONT TSC reliable
x86/tsc: set TSC_KNOWN_FREQ and TSC_RELIABLE flags on Intel Atom SoCs
arch/x86/include/asm/cpufeatures.h | 1 +
arch/x86/kernel/tsc.c | 25 ++++++++++++++++++++++---
arch/x86/kernel/tsc_msr.c | 18 ++++++++++++++++++
arch/x86/platform/intel-mid/mfld.c | 9 +++++++--
arch/x86/platform/intel-mid/mrfld.c | 8 ++++++--
5 files changed, 54 insertions(+), 7 deletions(-)
--
1.9.1
Unlike TSC calibration where we determin TSC frequency by another timer
with known frequency, CPUs/SoCs with CPUID leaf 0x15 come with a known
frequency and will report the frequency to software via CPUID
instruction. This hardware provided frequency is the "real" frequency
of TSC so we set the X86_FEATURE_TSC_KNOWN_FREQ flag to skip the whole
software calibration process.
We had a 24 hours test on one of the CPUID 0x15 capable platforms. With
PIT calibrated frequency, we got more than 3 seconds drift whereas with
CPUID determined frequency we only got less than 0.5 second drift. This
makes us believe that we should prefer CPUID determined frequency over
software calibrated frequency.
Signed-off-by: Bin Gao <[email protected]>
---
arch/x86/kernel/tsc.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 3ba146e..f1a7fb5 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -702,6 +702,13 @@ unsigned long native_calibrate_tsc(void)
}
}
+ /*
+ * TSC frequency determined by CPUID is a "hardware reported"
+ * frequency and is the most accurate one so far we have. This
+ * is considered a known frequency.
+ */
+ setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+
return crystal_khz * ebx_numerator / eax_denominator;
}
--
1.9.1
The X86_FEATURE_TSC_RELIABLE flag in Linux kernel implies both reliable
(at runtime) and trustable (at calibration). But reliable running and
trustable calibration are logically irrelevant. Per Thomas Gleixner's
suggestion we would like to split this flag into two separate flags:
X86_FEATURE_TSC_RELIABLE - running reliably
X86_FEATURE_TSC_KNOWN_FREQ - frequency is known (no calibration required)
These two flags allow Linux kernel to act differently based on
processor/SoC's capability, i.e. no watchdog on TSC if TSC is reliable,
and no calibration if TSC frequency is known.
Current Linux kernel already gurantees calibration is skipped for
processors that can report TSC frequency by CPUID or MSR. However, the
delayed calibration is still not skipped for these CPUID/MSR capable
processors. The new flag X86_FEATURE_TSC_KNOWN_FREQ added by this patch
will gurantee the delayed calibration is skipped.
Signed-off-by: Bin Gao <[email protected]>
---
arch/x86/include/asm/cpufeatures.h | 1 +
arch/x86/kernel/tsc.c | 11 ++++++++---
2 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index a396292..7f6a5f8 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -106,6 +106,7 @@
#define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */
#define X86_FEATURE_EAGER_FPU ( 3*32+29) /* "eagerfpu" Non lazy FPU restore */
#define X86_FEATURE_NONSTOP_TSC_S3 ( 3*32+30) /* TSC doesn't stop in S3 state */
+#define X86_FEATURE_TSC_KNOWN_FREQ ( 3*32+31) /* TSC has known frequency */
/* Intel-defined CPU features, CPUID level 0x00000001 (ecx), word 4 */
#define X86_FEATURE_XMM3 ( 4*32+ 0) /* "pni" SSE-3 */
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 46b2f41..3ba146e 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -1283,10 +1283,15 @@ static int __init init_tsc_clocksource(void)
clocksource_tsc.flags |= CLOCK_SOURCE_SUSPEND_NONSTOP;
/*
- * Trust the results of the earlier calibration on systems
- * exporting a reliable TSC.
+ * When TSC frequency is known (generally got by MSR or CPUID), we skip
+ * the refined calibration and directly register it as a clocksource.
+ *
+ * We still keep the TSC_RELIABLE flag here to avoid regression -
+ * it will be removed after all the conversion for other code paths
+ * connected to this flag is done.
*/
- if (boot_cpu_has(X86_FEATURE_TSC_RELIABLE)) {
+ if (boot_cpu_has(X86_FEATURE_TSC_RELIABLE) ||
+ boot_cpu_has(X86_FEATURE_TSC_KNOWN_FREQ)) {
clocksource_register_khz(&clocksource_tsc, tsc_khz);
return 0;
}
--
1.9.1
TSC on Intel Atom SoCs capable of determining TSC frequency by MSR is
reliable and the frequency is known (because it's provided by HW).
On these platforms PIT/HPET is generally not available so
calibration won't work at all and also TSC is the only reliable
clocksource. So we set both X86_FEATURE_TSC_KNOWN_FREQ and
X86_FEATURE_TSC_RELIABLE flags to make sure the calibration is
skipped and no watchdog on TSC.
Signed-off-by: Bin Gao <[email protected]>
---
arch/x86/kernel/tsc_msr.c | 18 ++++++++++++++++++
arch/x86/platform/intel-mid/mfld.c | 9 +++++++--
arch/x86/platform/intel-mid/mrfld.c | 8 ++++++--
3 files changed, 31 insertions(+), 4 deletions(-)
diff --git a/arch/x86/kernel/tsc_msr.c b/arch/x86/kernel/tsc_msr.c
index 0fe720d..c0f137c 100644
--- a/arch/x86/kernel/tsc_msr.c
+++ b/arch/x86/kernel/tsc_msr.c
@@ -100,5 +100,23 @@ unsigned long cpu_khz_from_msr(void)
#ifdef CONFIG_X86_LOCAL_APIC
lapic_timer_frequency = (freq * 1000) / HZ;
#endif
+
+ /*
+ * TSC frequency determined by MSR is always considered "known"
+ * because it is reported by HW.
+ * Another fact is that on MSR capable platforms, PIT/HPET is
+ * generally not available so calibration won't work at all.
+ */
+ setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+
+ /*
+ * Unfortunately there is no a HW way to report TSC is reliable.
+ * We were told by silicon design team that TSC on Atom SoCs are
+ * always "reliable". TSC is also the only reliable clocksource
+ * on these SoCs (HPET is either not present or not functional)
+ * so marke TSC reliable to avoid watchdog on it.
+ */
+ setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
+
return res;
}
diff --git a/arch/x86/platform/intel-mid/mfld.c b/arch/x86/platform/intel-mid/mfld.c
index 1eb47b6..e793fe5 100644
--- a/arch/x86/platform/intel-mid/mfld.c
+++ b/arch/x86/platform/intel-mid/mfld.c
@@ -49,8 +49,13 @@ static unsigned long __init mfld_calibrate_tsc(void)
fast_calibrate = ratio * fsb;
pr_debug("read penwell tsc %lu khz\n", fast_calibrate);
lapic_timer_frequency = fsb * 1000 / HZ;
- /* mark tsc clocksource as reliable */
- set_cpu_cap(&boot_cpu_data, X86_FEATURE_TSC_RELIABLE);
+
+ /*
+ * TSC on Intel Atom SoCs is reliable and of known frequency.
+ * See tsc_msr.c for details.
+ */
+ setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+ setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
return fast_calibrate;
}
diff --git a/arch/x86/platform/intel-mid/mrfld.c b/arch/x86/platform/intel-mid/mrfld.c
index 59253db..e0607c7 100644
--- a/arch/x86/platform/intel-mid/mrfld.c
+++ b/arch/x86/platform/intel-mid/mrfld.c
@@ -78,8 +78,12 @@ static unsigned long __init tangier_calibrate_tsc(void)
pr_debug("Setting lapic_timer_frequency = %d\n",
lapic_timer_frequency);
- /* mark tsc clocksource as reliable */
- set_cpu_cap(&boot_cpu_data, X86_FEATURE_TSC_RELIABLE);
+ /*
+ * TSC on Intel Atom SoCs is reliable and of known frequency.
+ * See tsc_msr.c for details.
+ */
+ setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+ setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
return fast_calibrate;
}
--
1.9.1
On Intel GOLDMONT Atom SoC TSC is the only reliable clocksource.
We mark TSC reliable to avoid watchdog on it.
Signed-off-by: Bin Gao <[email protected]>
---
arch/x86/kernel/tsc.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index f1a7fb5..354b302 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -709,6 +709,13 @@ unsigned long native_calibrate_tsc(void)
*/
setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+ /*
+ * For Atom SoCs TSC is the only reliable clocksource.
+ * Mark TSC reliable so no watchdog on it.
+ */
+ if (boot_cpu_data.x86_model == INTEL_FAM6_ATOM_GOLDMONT)
+ setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
+
return crystal_khz * ebx_numerator / eax_denominator;
}
--
1.9.1
* Bin Gao <[email protected]> wrote:
> TSC on Intel Atom SoCs capable of determining TSC frequency by MSR is
> reliable and the frequency is known (because it's provided by HW).
> On these platforms PIT/HPET is generally not available so
> calibration won't work at all and also TSC is the only reliable
> clocksource. So we set both X86_FEATURE_TSC_KNOWN_FREQ and
> X86_FEATURE_TSC_RELIABLE flags to make sure the calibration is
> skipped and no watchdog on TSC.
> + /*
> + * TSC frequency determined by MSR is always considered "known"
> + * because it is reported by HW.
> + * Another fact is that on MSR capable platforms, PIT/HPET is
> + * generally not available so calibration won't work at all.
> + */
> + setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
> +
> + /*
> + * Unfortunately there is no a HW way to report TSC is reliable.
> + * We were told by silicon design team that TSC on Atom SoCs are
> + * always "reliable". TSC is also the only reliable clocksource
> + * on these SoCs (HPET is either not present or not functional)
> + * so marke TSC reliable to avoid watchdog on it.
minor nit:
s/there is no a HW way/
there is no HW way
Thanks,
Ingo
Commit-ID: 47c95a46d0fae07762f0a38aa3709ae63f307048
Gitweb: http://git.kernel.org/tip/47c95a46d0fae07762f0a38aa3709ae63f307048
Author: Bin Gao <[email protected]>
AuthorDate: Tue, 15 Nov 2016 12:27:21 -0800
Committer: Thomas Gleixner <[email protected]>
CommitDate: Fri, 18 Nov 2016 10:58:30 +0100
x86/tsc: Add X86_FEATURE_TSC_KNOWN_FREQ flag
The X86_FEATURE_TSC_RELIABLE flag in Linux kernel implies both reliable
(at runtime) and trustable (at calibration). But reliable running and
trustable calibration independent of each other.
Add a new flag X86_FEATURE_TSC_KNOWN_FREQ, which denotes that the frequency
is known (via MSR/CPUID). This flag is only meant to skip the long term
calibration on systems which have a known frequency.
Add X86_FEATURE_TSC_KNOWN_FREQ to the skip the delayed calibration and
leave X86_FEATURE_TSC_RELIABLE in place.
After converting the existing users of X86_FEATURE_TSC_RELIABLE to use
either both flags or just X86_FEATURE_TSC_KNOWN_FREQ we can seperate the
functionality.
Suggested-by: Thomas Gleixner <[email protected]>
Signed-off-by: Bin Gao <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Thomas Gleixner <[email protected]>
---
arch/x86/include/asm/cpufeatures.h | 1 +
arch/x86/kernel/tsc.c | 11 ++++++++---
2 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index a396292..7f6a5f8 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -106,6 +106,7 @@
#define X86_FEATURE_APERFMPERF ( 3*32+28) /* APERFMPERF */
#define X86_FEATURE_EAGER_FPU ( 3*32+29) /* "eagerfpu" Non lazy FPU restore */
#define X86_FEATURE_NONSTOP_TSC_S3 ( 3*32+30) /* TSC doesn't stop in S3 state */
+#define X86_FEATURE_TSC_KNOWN_FREQ ( 3*32+31) /* TSC has known frequency */
/* Intel-defined CPU features, CPUID level 0x00000001 (ecx), word 4 */
#define X86_FEATURE_XMM3 ( 4*32+ 0) /* "pni" SSE-3 */
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index 46b2f41..d2c4ee4 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -1283,10 +1283,15 @@ static int __init init_tsc_clocksource(void)
clocksource_tsc.flags |= CLOCK_SOURCE_SUSPEND_NONSTOP;
/*
- * Trust the results of the earlier calibration on systems
- * exporting a reliable TSC.
+ * When TSC frequency is known (retrieved via MSR or CPUID), we skip
+ * the refined calibration and directly register it as a clocksource.
+ *
+ * We still keep the TSC_RELIABLE flag here to avoid regressions -
+ * it will be removed after all the conversion for other code paths
+ * connected to this flag is done.
*/
- if (boot_cpu_has(X86_FEATURE_TSC_RELIABLE)) {
+ if (boot_cpu_has(X86_FEATURE_TSC_RELIABLE) ||
+ boot_cpu_has(X86_FEATURE_TSC_KNOWN_FREQ)) {
clocksource_register_khz(&clocksource_tsc, tsc_khz);
return 0;
}
Commit-ID: 4ca4df0b7eb06df264b2919759957f6d6ea1822e
Gitweb: http://git.kernel.org/tip/4ca4df0b7eb06df264b2919759957f6d6ea1822e
Author: Bin Gao <[email protected]>
AuthorDate: Tue, 15 Nov 2016 12:27:22 -0800
Committer: Thomas Gleixner <[email protected]>
CommitDate: Fri, 18 Nov 2016 10:58:30 +0100
x86/tsc: Mark TSC frequency determined by CPUID as known
CPUs/SoCs with CPUID leaf 0x15 come with a known frequency and will report
the frequency to software via CPUID instruction. This hardware provided
frequency is the "real" frequency of TSC.
Set the X86_FEATURE_TSC_KNOWN_FREQ flag for such systems to skip the
software calibration process.
A 24 hours test on one of the CPUID 0x15 capable platforms was
conducted. PIT calibrated frequency resulted in more than 3 seconds drift
whereas the CPUID determined frequency showed less than 0.5 second
drift.
Signed-off-by: Bin Gao <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Thomas Gleixner <[email protected]>
---
arch/x86/kernel/tsc.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index d2c4ee4..e58c319 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -702,6 +702,13 @@ unsigned long native_calibrate_tsc(void)
}
}
+ /*
+ * TSC frequency determined by CPUID is a "hardware reported"
+ * frequency and is the most accurate one so far we have. This
+ * is considered a known frequency.
+ */
+ setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+
return crystal_khz * ebx_numerator / eax_denominator;
}
Commit-ID: f3a02ecebed7df7d5d68898628dea7a3bfcf03e3
Gitweb: http://git.kernel.org/tip/f3a02ecebed7df7d5d68898628dea7a3bfcf03e3
Author: Bin Gao <[email protected]>
AuthorDate: Tue, 15 Nov 2016 12:27:24 -0800
Committer: Thomas Gleixner <[email protected]>
CommitDate: Fri, 18 Nov 2016 10:58:31 +0100
x86/tsc: Set TSC_KNOWN_FREQ and TSC_RELIABLE flags on Intel Atom SoCs
TSC on Intel Atom SoCs capable of determining TSC frequency by MSR is
reliable and the frequency is known (provided by HW).
On these platforms PIT/HPET is generally not available so calibration won't
work at all and there is no other clocksource to act as a watchdog for the
TSC, so we have no other choice than to trust it.
Set both X86_FEATURE_TSC_KNOWN_FREQ and X86_FEATURE_TSC_RELIABLE flags to
make sure the calibration is skipped and no watchdog is required.
Signed-off-by: Bin Gao <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Thomas Gleixner <[email protected]>
---
arch/x86/kernel/tsc_msr.c | 19 +++++++++++++++++++
arch/x86/platform/intel-mid/mfld.c | 9 +++++++--
arch/x86/platform/intel-mid/mrfld.c | 8 ++++++--
3 files changed, 32 insertions(+), 4 deletions(-)
diff --git a/arch/x86/kernel/tsc_msr.c b/arch/x86/kernel/tsc_msr.c
index 0fe720d..19afdbd 100644
--- a/arch/x86/kernel/tsc_msr.c
+++ b/arch/x86/kernel/tsc_msr.c
@@ -100,5 +100,24 @@ unsigned long cpu_khz_from_msr(void)
#ifdef CONFIG_X86_LOCAL_APIC
lapic_timer_frequency = (freq * 1000) / HZ;
#endif
+
+ /*
+ * TSC frequency determined by MSR is always considered "known"
+ * because it is reported by HW.
+ * Another fact is that on MSR capable platforms, PIT/HPET is
+ * generally not available so calibration won't work at all.
+ */
+ setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+
+ /*
+ * Unfortunately there is no way for hardware to tell whether the
+ * TSC is reliable. We were told by silicon design team that TSC
+ * on Atom SoCs are always "reliable". TSC is also the only
+ * reliable clocksource on these SoCs (HPET is either not present
+ * or not functional) so mark TSC reliable which removes the
+ * requirement for a watchdog clocksource.
+ */
+ setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
+
return res;
}
diff --git a/arch/x86/platform/intel-mid/mfld.c b/arch/x86/platform/intel-mid/mfld.c
index 1eb47b6..e793fe5 100644
--- a/arch/x86/platform/intel-mid/mfld.c
+++ b/arch/x86/platform/intel-mid/mfld.c
@@ -49,8 +49,13 @@ static unsigned long __init mfld_calibrate_tsc(void)
fast_calibrate = ratio * fsb;
pr_debug("read penwell tsc %lu khz\n", fast_calibrate);
lapic_timer_frequency = fsb * 1000 / HZ;
- /* mark tsc clocksource as reliable */
- set_cpu_cap(&boot_cpu_data, X86_FEATURE_TSC_RELIABLE);
+
+ /*
+ * TSC on Intel Atom SoCs is reliable and of known frequency.
+ * See tsc_msr.c for details.
+ */
+ setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+ setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
return fast_calibrate;
}
diff --git a/arch/x86/platform/intel-mid/mrfld.c b/arch/x86/platform/intel-mid/mrfld.c
index 59253db..e0607c7 100644
--- a/arch/x86/platform/intel-mid/mrfld.c
+++ b/arch/x86/platform/intel-mid/mrfld.c
@@ -78,8 +78,12 @@ static unsigned long __init tangier_calibrate_tsc(void)
pr_debug("Setting lapic_timer_frequency = %d\n",
lapic_timer_frequency);
- /* mark tsc clocksource as reliable */
- set_cpu_cap(&boot_cpu_data, X86_FEATURE_TSC_RELIABLE);
+ /*
+ * TSC on Intel Atom SoCs is reliable and of known frequency.
+ * See tsc_msr.c for details.
+ */
+ setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+ setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
return fast_calibrate;
}
Commit-ID: 4635fdc696a8e89eead3ea1712ae6ada38538d40
Gitweb: http://git.kernel.org/tip/4635fdc696a8e89eead3ea1712ae6ada38538d40
Author: Bin Gao <[email protected]>
AuthorDate: Tue, 15 Nov 2016 12:27:23 -0800
Committer: Thomas Gleixner <[email protected]>
CommitDate: Fri, 18 Nov 2016 10:58:30 +0100
x86/tsc: Mark Intel ATOM_GOLDMONT TSC reliable
On Intel GOLDMONT Atom SoC TSC is the only available clocksource, so there
is no way to do software calibration or have a watchdog clocksource for it.
Software calibration is already disabled via the TSC_KNOWN_FREQ flag, but
the watchdog requirement still persists, so such systems cannot switch to
high resolution/nohz mode.
Mark it reliable, so it becomes usable. Hardware teams confirmed that this
is safe on that SoC.
Signed-off-by: Bin Gao <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Thomas Gleixner <[email protected]>
---
arch/x86/kernel/tsc.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
index e58c319..f4dfdaa 100644
--- a/arch/x86/kernel/tsc.c
+++ b/arch/x86/kernel/tsc.c
@@ -709,6 +709,13 @@ unsigned long native_calibrate_tsc(void)
*/
setup_force_cpu_cap(X86_FEATURE_TSC_KNOWN_FREQ);
+ /*
+ * For Atom SoCs TSC is the only reliable clocksource.
+ * Mark TSC reliable so no watchdog on it.
+ */
+ if (boot_cpu_data.x86_model == INTEL_FAM6_ATOM_GOLDMONT)
+ setup_force_cpu_cap(X86_FEATURE_TSC_RELIABLE);
+
return crystal_khz * ebx_numerator / eax_denominator;
}