2016-03-23 20:49:44

by Kani, Toshimitsu

Subject: [PATCH v3 0/7] Enhance PAT init to fix Xorg crashes

A Xorg failure on qemu32 was reported as a regression [1] caused by
'commit 9cd25aac1f44 ("x86/mm/pat: Emulate PAT when it is disabled")'.
This patch-set fixes the regression.

Negative effects of this regression were two failures [2] in Xorg on
QEMU with QEMU CPU model "qemu32" (-cpu qemu32), which were triggered
by the fact that its virtual CPU does not support MTRR.
#1. copy_process() failed in the check in reserve_pfn_range()
#2. error path in copy_process() then hit WARN_ON_ONCE in
untrack_pfn().

These negative effects are caused by two separate bugs, but they can be
addressed in separate patches. This patch-set addresses the root cause,
a long-standing PAT initialization issue.

Please see the changelog in patch 4/7 for the details of the issue.

- Patch 1-2 make necessary enhancement to PAT for the fix without
breaking Xen.
- Patch 3 is cleanup.
- Patch 4 fixes the regression.
- Patch 5 fixes an MTRR issue related with PAT init.
- Patch 6 removes PAT init code from Xen.
- Patch 7 adds PAT init to documentation.

[1]: https://lkml.org/lkml/2016/3/3/828
[2]: https://lkml.org/lkml/2016/3/4/775

I'd appreciate if someone can test this patch-set on Xen to verify that
there is no change in "x86/PAT: Configuration [0-7] .." message in dmesg.

---
v3:
- Change a new func name to init_cache_modes(). (Borislav Petkov)
- Add check with __pat_enabled, and use WARN_ONCE() for a bug check
in pat_disable(). (Borislav Petkov)
- Update changelog, comments, and doc per review. (Borislav Petkov)

v2:
- Divide patch-set into a single change. (Borislav Petkov)
- Xen's case must be handled properly. (Luis R. Rodriguez)
- Change changelog and title to describe the issue. (Ingo Molnar)
- Update an error message. (Robert Elliott, Borislav Petkov)

---
Toshi Kani (7):
1/7 x86/mm/pat: Add support of non-default PAT MSR setting
2/7 x86/mm/pat: Add pat_disable() interface
3/7 x86/mm/pat: Replace cpu_has_pat with boot_cpu_has
4/7 x86/mtrr: Fix Xorg crashes in Qemu sessions
5/7 x86/mtrr: Fix PAT init handling when MTRR is disabled
6/7 x86/xen,pat: Remove PAT table init code from Xen
7/7 x86/pat: Document PAT initialization

---
Documentation/x86/pat.txt | 32 ++++++++++++++
arch/x86/include/asm/mtrr.h | 6 ++-
arch/x86/include/asm/pat.h | 2 +-
arch/x86/kernel/cpu/mtrr/generic.c | 24 +++++-----
arch/x86/kernel/cpu/mtrr/main.c | 13 +++++-
arch/x86/kernel/cpu/mtrr/mtrr.h | 1 +
arch/x86/mm/pat.c | 90 ++++++++++++++++++++++++++++----------
arch/x86/xen/enlighten.c | 9 ----
8 files changed, 132 insertions(+), 45 deletions(-)

2016-03-23 20:49:49

by Kani, Toshimitsu

Subject: [PATCH v3 1/7] x86/mm/pat: Add support of non-default PAT MSR setting

In preparation for fixing a regression caused by 'commit 9cd25aac1f44
("x86/mm/pat: Emulate PAT when it is disabled")', PAT needs to
support a case that PAT MSR is initialized with a non-default
value.

When pat_init() is called and PAT is disabled, it initializes
PAT table with the BIOS default value. Xen, however, sets PAT MSR
with a non-default value to enable WC. This causes inconsistency
between PAT table and PAT MSR when PAT is set to disable on Xen.

Change pat_init() to handle the PAT disable cases properly. Add
init_cache_modes() to handle two cases when PAT is set to disable.
1. CPU supports PAT: Set PAT table to be consistent with PAT MSR.
2. CPU does not support PAT: Set PAT table to be consistent with
PWT and PCD bits in a PTE.

Note, __init_cache_modes(), renamed from pat_init_cache_modes(),
will be changed to a static function in a later patch.

Signed-off-by: Toshi Kani <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Luis R. Rodriguez <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Thomas Gleixner <[email protected]>
---
arch/x86/include/asm/pat.h | 2 +
arch/x86/mm/pat.c | 73 ++++++++++++++++++++++++++++++++------------
arch/x86/xen/enlighten.c | 2 +
3 files changed, 55 insertions(+), 22 deletions(-)

diff --git a/arch/x86/include/asm/pat.h b/arch/x86/include/asm/pat.h
index ca6c228..97ea55b 100644
--- a/arch/x86/include/asm/pat.h
+++ b/arch/x86/include/asm/pat.h
@@ -6,7 +6,7 @@

bool pat_enabled(void);
extern void pat_init(void);
-void pat_init_cache_modes(u64);
+void __init_cache_modes(u64);

extern int reserve_memtype(u64 start, u64 end,
enum page_cache_mode req_pcm, enum page_cache_mode *ret_pcm);
diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 04e2e71..1da55a5 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -181,7 +181,7 @@ static enum page_cache_mode pat_get_cache_mode(unsigned pat_val, char *msg)
* configuration.
* Using lower indices is preferred, so we start with highest index.
*/
-void pat_init_cache_modes(u64 pat)
+void __init_cache_modes(u64 pat)
{
enum page_cache_mode cache;
char pat_msg[33];
@@ -207,9 +207,6 @@ static void pat_bsp_init(u64 pat)
return;
}

- if (!pat_enabled())
- goto done;
-
rdmsrl(MSR_IA32_CR_PAT, tmp_pat);
if (!tmp_pat) {
pat_disable("PAT MSR is 0, disabled.");
@@ -218,15 +215,11 @@ static void pat_bsp_init(u64 pat)

wrmsrl(MSR_IA32_CR_PAT, pat);

-done:
- pat_init_cache_modes(pat);
+ __init_cache_modes(pat);
}

static void pat_ap_init(u64 pat)
{
- if (!pat_enabled())
- return;
-
if (!cpu_has_pat) {
/*
* If this happens we are on a secondary CPU, but switched to
@@ -238,18 +231,32 @@ static void pat_ap_init(u64 pat)
wrmsrl(MSR_IA32_CR_PAT, pat);
}

-void pat_init(void)
+static void init_cache_modes(void)
{
- u64 pat;
- struct cpuinfo_x86 *c = &boot_cpu_data;
+ u64 pat = 0;
+ static int init_cm_done;

- if (!pat_enabled()) {
+ if (init_cm_done)
+ return;
+
+ if (boot_cpu_has(X86_FEATURE_PAT)) {
+ /*
+ * CPU supports PAT. Set PAT table to be consistent with
+ * PAT MSR. This case supports "nopat" boot option, and
+ * virtual machine environments which support PAT without
+ * MTRRs. In specific, Xen has unique setup to PAT MSR.
+ *
+ * If PAT MSR returns 0, it is considered invalid and emulates
+ * as No PAT.
+ */
+ rdmsrl(MSR_IA32_CR_PAT, pat);
+ }
+
+ if (!pat) {
/*
* No PAT. Emulate the PAT table that corresponds to the two
- * cache bits, PWT (Write Through) and PCD (Cache Disable). This
- * setup is the same as the BIOS default setup when the system
- * has PAT but the "nopat" boot option has been specified. This
- * emulated PAT table is used when MSR_IA32_CR_PAT returns 0.
+ * cache bits, PWT (Write Through) and PCD (Cache Disable).
+ * This setup is also the same as the BIOS default setup.
*
* PTE encoding:
*
@@ -266,10 +273,36 @@ void pat_init(void)
*/
pat = PAT(0, WB) | PAT(1, WT) | PAT(2, UC_MINUS) | PAT(3, UC) |
PAT(4, WB) | PAT(5, WT) | PAT(6, UC_MINUS) | PAT(7, UC);
+ }
+
+ __init_cache_modes(pat);
+
+ init_cm_done = 1;
+}
+
+/**
+ * pat_init - Initialize PAT MSR and PAT table
+ *
+ * This function initializes PAT MSR and PAT table with an OS-defined value
+ * to enable additional cache attributes, WC and WT.
+ *
+ * This function must be called on all CPUs using the specific sequence of
+ * operations defined in Intel SDM. mtrr_rendezvous_handler() provides this
+ * procedure for PAT.
+ */
+void pat_init(void)
+{
+ u64 pat;
+ struct cpuinfo_x86 *c = &boot_cpu_data;
+
+ if (!pat_enabled()) {
+ init_cache_modes();
+ return;
+ }

- } else if ((c->x86_vendor == X86_VENDOR_INTEL) &&
- (((c->x86 == 0x6) && (c->x86_model <= 0xd)) ||
- ((c->x86 == 0xf) && (c->x86_model <= 0x6)))) {
+ if ((c->x86_vendor == X86_VENDOR_INTEL) &&
+ (((c->x86 == 0x6) && (c->x86_model <= 0xd)) ||
+ ((c->x86 == 0xf) && (c->x86_model <= 0x6)))) {
/*
* PAT support with the lower four entries. Intel Pentium 2,
* 3, M, and 4 are affected by PAT errata, which makes the
diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index 2379a5a..f4296b6 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -1623,7 +1623,7 @@ asmlinkage __visible void __init xen_start_kernel(void)
* configuration.
*/
rdmsrl(MSR_IA32_CR_PAT, pat);
- pat_init_cache_modes(pat);
+ __init_cache_modes(pat);

/* keep using Xen gdt for now; no urgent need to change it */

2016-03-23 20:50:01

by Kani, Toshimitsu

Subject: [PATCH v3 7/7] x86/pat: Document PAT initialization

Update PAT documentation to describe how PAT is initialized under
various configurations.

Signed-off-by: Toshi Kani <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Luis R. Rodriguez <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Thomas Gleixner <[email protected]>
---
Documentation/x86/pat.txt | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)

diff --git a/Documentation/x86/pat.txt b/Documentation/x86/pat.txt
index 54944c7..8ccc0fc 100644
--- a/Documentation/x86/pat.txt
+++ b/Documentation/x86/pat.txt
@@ -196,3 +196,35 @@ Another, more verbose way of getting PAT related debug messages is with
"debugpat" boot parameter. With this parameter, various debug messages are
printed to dmesg log.

+PAT Initialization
+------------------
+
+The following table describes how PAT is initialized under various
+configurations. PAT MSR must be updated by Linux in order to support WC
+and WT attributes. Otherwise, the PAT MSR has the value programmed in it
+by the firmware. Note, Xen enables WC attribute in the PAT MSR for guests.
+
+ MTRR PAT Call Sequence PAT State PAT MSR
+ =========================================================
+ E E MTRR -> PAT init Enabled OS
+ E D MTRR -> PAT init Disabled -
+ D E MTRR -> PAT disable Disabled BIOS
+ D D MTRR -> PAT disable Disabled -
+ - np/E PAT -> PAT disable Disabled BIOS
+ - np/D PAT -> PAT disable Disabled -
+ E !P/E MTRR -> PAT init Disabled BIOS
+ D !P/E MTRR -> PAT disable Disabled BIOS
+ !M !P/E MTRR stub -> PAT disable Disabled BIOS
+
+ Legend
+ ------------------------------------------------
+ E Feature enabled in CPU
+ D Feature disabled/unsupported in CPU
+ np "nopat" boot option specified
+ !P CONFIG_X86_PAT option unset
+ !M CONFIG_MTRR option unset
+ Enabled PAT state set to enable
+ Disabled PAT state set to disable
+ OS PAT initializes PAT MSR with OS setting
+ BIOS PAT keeps PAT MSR with BIOS setting
+

2016-03-23 20:49:53

by Kani, Toshimitsu

Subject: [PATCH v3 3/7] x86/mm/pat: Replace cpu_has_pat with boot_cpu_has

Borislav Petkov wrote:
> Please use on init paths boot_cpu_has(X86_FEATURE_PAT) and on fast
> paths static_cpu_has(X86_FEATURE_PAT). No more of that cpu_has_XXX
> ugliness.

Replace the use of cpu_has_pat on init paths with boot_cpu_has().

Suggested-by: Borislav Petkov <[email protected]>
Signed-off-by: Toshi Kani <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Luis R. Rodriguez <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Robert Elliott <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Thomas Gleixner <[email protected]>
---
arch/x86/mm/pat.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 3c08a27..3aea1ab 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -213,7 +213,7 @@ static void pat_bsp_init(u64 pat)
{
u64 tmp_pat;

- if (!cpu_has_pat) {
+ if (!boot_cpu_has(X86_FEATURE_PAT)) {
pat_disable("PAT not supported by CPU.");
return;
}
@@ -231,7 +231,7 @@ static void pat_bsp_init(u64 pat)

static void pat_ap_init(u64 pat)
{
- if (!cpu_has_pat) {
+ if (!boot_cpu_has(X86_FEATURE_PAT)) {
/*
* If this happens we are on a secondary CPU, but switched to
* PAT on the boot CPU. We have no way to undo PAT.

2016-03-23 20:49:59

by Kani, Toshimitsu

Subject: [PATCH v3 5/7] x86/mtrr: Fix PAT init handling when MTRR is disabled

get_mtrr_state() calls pat_init() on BSP even if MTRR is disabled.
This results in calling pat_init() on BSP only since APs do not call
pat_init() when MTRR is disabled. This inconsistency between BSP
and APs leads to undefined behavior.

Make BSP's calling condition to pat_init() consistent with AP's,
mtrr_ap_init() and mtrr_aps_init().

Signed-off-by: Toshi Kani <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Luis R. Rodriguez <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Thomas Gleixner <[email protected]>
---
arch/x86/kernel/cpu/mtrr/generic.c | 24 ++++++++++++++----------
arch/x86/kernel/cpu/mtrr/main.c | 3 +++
arch/x86/kernel/cpu/mtrr/mtrr.h | 1 +
3 files changed, 18 insertions(+), 10 deletions(-)

diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/generic.c
index fcbcb2f..a9d2e54 100644
--- a/arch/x86/kernel/cpu/mtrr/generic.c
+++ b/arch/x86/kernel/cpu/mtrr/generic.c
@@ -444,11 +444,24 @@ static void __init print_mtrr_state(void)
pr_debug("TOM2: %016llx aka %lldM\n", mtrr_tom2, mtrr_tom2>>20);
}

+/* PAT setup for BP. We need to go through sync steps here */
+void __init mtrr_bp_pat_init(void)
+{
+ unsigned long flags;
+
+ local_irq_save(flags);
+ prepare_set();
+
+ pat_init();
+
+ post_set();
+ local_irq_restore(flags);
+}
+
/* Grab all of the MTRR state for this CPU into *state */
bool __init get_mtrr_state(void)
{
struct mtrr_var_range *vrs;
- unsigned long flags;
unsigned lo, dummy;
unsigned int i;

@@ -481,15 +494,6 @@ bool __init get_mtrr_state(void)

mtrr_state_set = 1;

- /* PAT setup for BP. We need to go through sync steps here */
- local_irq_save(flags);
- prepare_set();
-
- pat_init();
-
- post_set();
- local_irq_restore(flags);
-
return !!(mtrr_state.enabled & MTRR_STATE_MTRR_ENABLED);
}

diff --git a/arch/x86/kernel/cpu/mtrr/main.c b/arch/x86/kernel/cpu/mtrr/main.c
index 8b1947b..7d393ec 100644
--- a/arch/x86/kernel/cpu/mtrr/main.c
+++ b/arch/x86/kernel/cpu/mtrr/main.c
@@ -752,6 +752,9 @@ void __init mtrr_bp_init(void)
/* BIOS may override */
__mtrr_enabled = get_mtrr_state();

+ if (mtrr_enabled())
+ mtrr_bp_pat_init();
+
if (mtrr_cleanup(phys_addr)) {
changed_by_mtrr_cleanup = 1;
mtrr_if->set_all();
diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.h b/arch/x86/kernel/cpu/mtrr/mtrr.h
index 951884d..6c7ced0 100644
--- a/arch/x86/kernel/cpu/mtrr/mtrr.h
+++ b/arch/x86/kernel/cpu/mtrr/mtrr.h
@@ -52,6 +52,7 @@ void set_mtrr_prepare_save(struct set_mtrr_context *ctxt);
void fill_mtrr_var_range(unsigned int index,
u32 base_lo, u32 base_hi, u32 mask_lo, u32 mask_hi);
bool get_mtrr_state(void);
+void mtrr_bp_pat_init(void);

extern void set_mtrr_ops(const struct mtrr_ops *ops);

2016-03-23 20:49:57

by Kani, Toshimitsu

Subject: [PATCH v3 4/6] x86/mtrr: Fix Xorg crashes in Qemu sessions

A Xorg failure on qemu32 was reported as a regression [1] caused by
'commit 9cd25aac1f44 ("x86/mm/pat: Emulate PAT when it is disabled")'.
This patch fixes this regression.

Negative effects of this regression were the following two failures [2]
in Xorg on QEMU with QEMU CPU model "qemu32" (-cpu qemu32), which were
triggered by the fact that its virtual CPU does not support MTRRs.

#1. copy_process() failed in the check in reserve_pfn_range()

copy_process
copy_mm
dup_mm
dup_mmap
copy_page_range
track_pfn_copy
reserve_pfn_range

A WC map request was tracked as WC in memtype, which set a PTE as
UC (pgprot) per __cachemode2pte_tbl[]. This led to this error in
reserve_pfn_range() called from track_pfn_copy(), which obtained
a pgprot from a PTE. It converts pgprot to page_cache_mode, which
does not necessarily result in the original page_cache_mode since
__cachemode2pte_tbl[] redirects multiple types to UC.

#2. error path in copy_process() then hit WARN_ON_ONCE in
untrack_pfn().

x86/PAT: Xorg:509 map pfn expected mapping type uncached-
minus for [mem 0xfd000000-0xfdffffff], got write-combining
Call Trace:
dump_stack
warn_slowpath_common
? untrack_pfn
? untrack_pfn
warn_slowpath_null
untrack_pfn
? __kunmap_atomic
unmap_single_vma
? pagevec_move_tail_fn
unmap_vmas
exit_mmap
mmput
copy_process.part.47
_do_fork
SyS_clone
do_syscall_32_irqs_on
entry_INT80_32

These negative effects are caused by two separate bugs, but they
can be addressed in separate patches. Fixing the pat_init() issue
described below addresses the root cause, and avoids Xorg to hit
these cases.

When the CPU does not support MTRRs, MTRR does not call pat_init(),
which leaves PAT enabled without initializing PAT. This pat_init()
issue is a long-standing issue, but manifested as issue #1 (and then
hit issue #2) with the above-mentioned commit because the memtype
now tracks cache attribute with 'page_cache_mode'.

This pat_init() issue existed before the commit, but we used pgprot
in memtype. Hence, we did not have issue #1 before. But WC request
resulted in WT in effect because WC pgrot is actually WT when PAT
is not initialized. This is not how it was designed to work. When
PAT is set to disable properly, WC is converted to UC. The use of
WT can result in a system crash if the target range does not support
WT. Fortunately, nobody ran into such issue before.

To fix this pat_init() issue, PAT code has been enhanced to provide
pat_disable() interface. Call this interface when MTRRs are disabled.
By setting PAT to disable properly, PAT bypasses the memtype check,
and avoids issue #1.

[1]: https://lkml.org/lkml/2016/3/3/828
[2]: https://lkml.org/lkml/2016/3/4/775
Signed-off-by: Toshi Kani <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Luis R. Rodriguez <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Thomas Gleixner <[email protected]>
---
arch/x86/include/asm/mtrr.h | 6 +++++-
arch/x86/kernel/cpu/mtrr/main.c | 10 +++++++++-
2 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/mtrr.h b/arch/x86/include/asm/mtrr.h
index b94f6f6..dbff145 100644
--- a/arch/x86/include/asm/mtrr.h
+++ b/arch/x86/include/asm/mtrr.h
@@ -24,6 +24,7 @@
#define _ASM_X86_MTRR_H

#include <uapi/asm/mtrr.h>
+#include <asm/pat.h>


/*
@@ -83,9 +84,12 @@ static inline int mtrr_trim_uncached_memory(unsigned long end_pfn)
static inline void mtrr_centaur_report_mcr(int mcr, u32 lo, u32 hi)
{
}
+static inline void mtrr_bp_init(void)
+{
+ pat_disable("MTRRs disabled, skipping PAT initialization too.");
+}

#define mtrr_ap_init() do {} while (0)
-#define mtrr_bp_init() do {} while (0)
#define set_mtrr_aps_delayed_init() do {} while (0)
#define mtrr_aps_init() do {} while (0)
#define mtrr_bp_restore() do {} while (0)
diff --git a/arch/x86/kernel/cpu/mtrr/main.c b/arch/x86/kernel/cpu/mtrr/main.c
index 10f8d47..8b1947b 100644
--- a/arch/x86/kernel/cpu/mtrr/main.c
+++ b/arch/x86/kernel/cpu/mtrr/main.c
@@ -759,8 +759,16 @@ void __init mtrr_bp_init(void)
}
}

- if (!mtrr_enabled())
+ if (!mtrr_enabled()) {
pr_info("MTRR: Disabled\n");
+
+ /*
+ * PAT initialization relies on MTRR's rendezvous handler.
+ * Skip PAT init until the handler can initialize both
+ * features independently.
+ */
+ pat_disable("MTRRs disabled, skipping PAT initialization too.");
+ }
}

void mtrr_ap_init(void)

2016-03-23 20:50:53

by Kani, Toshimitsu

Subject: [PATCH v3 6/7] x86/xen,pat: Remove PAT table init code from Xen

Xen supports PAT without MTRRs for its guests. In order to
enable WC attribute, it was necessary for xen_start_kernel()
to call pat_init_cache_modes() to update PAT table before
starting guest kernel.

Now that the kernel initializes PAT table to the BIOS handoff
state when MTRR is disabled, this Xen-specific PAT init code
is no longer necessary. Delete it from xen_start_kernel().

Also change __init_cache_modes() to a static function since
PAT table should not be tweaked by other modules.

Signed-off-by: Toshi Kani <[email protected]>
Acked-by: Juergen Gross <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Luis R. Rodriguez <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Thomas Gleixner <[email protected]>
---
arch/x86/include/asm/pat.h | 1 -
arch/x86/mm/pat.c | 2 +-
arch/x86/xen/enlighten.c | 9 ---------
3 files changed, 1 insertion(+), 11 deletions(-)

diff --git a/arch/x86/include/asm/pat.h b/arch/x86/include/asm/pat.h
index 0ad356c..0b1ff4c 100644
--- a/arch/x86/include/asm/pat.h
+++ b/arch/x86/include/asm/pat.h
@@ -7,7 +7,6 @@
bool pat_enabled(void);
void pat_disable(const char *reason);
extern void pat_init(void);
-void __init_cache_modes(u64);

extern int reserve_memtype(u64 start, u64 end,
enum page_cache_mode req_pcm, enum page_cache_mode *ret_pcm);
diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 3aea1ab..9db6915 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -192,7 +192,7 @@ static enum page_cache_mode pat_get_cache_mode(unsigned pat_val, char *msg)
* configuration.
* Using lower indices is preferred, so we start with highest index.
*/
-void __init_cache_modes(u64 pat)
+static void __init_cache_modes(u64 pat)
{
enum page_cache_mode cache;
char pat_msg[33];
diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index f4296b6..d5f172d 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -75,7 +75,6 @@
#include <asm/mach_traps.h>
#include <asm/mwait.h>
#include <asm/pci_x86.h>
-#include <asm/pat.h>
#include <asm/cpu.h>

#ifdef CONFIG_ACPI
@@ -1511,7 +1510,6 @@ asmlinkage __visible void __init xen_start_kernel(void)
{
struct physdev_set_iopl set_iopl;
unsigned long initrd_start = 0;
- u64 pat;
int rc;

if (!xen_start_info)
@@ -1618,13 +1616,6 @@ asmlinkage __visible void __init xen_start_kernel(void)
xen_start_info->nr_pages);
xen_reserve_special_pages();

- /*
- * Modify the cache mode translation tables to match Xen's PAT
- * configuration.
- */
- rdmsrl(MSR_IA32_CR_PAT, pat);
- __init_cache_modes(pat);
-
/* keep using Xen gdt for now; no urgent need to change it */

#ifdef CONFIG_X86_32

2016-03-23 20:49:51

by Kani, Toshimitsu

Subject: [PATCH v3 2/7] x86/mm/pat: Add pat_disable() interface

In preparation for fixing a regression caused by 'commit 9cd25aac1f44
("x86/mm/pat: Emulate PAT when it is disabled")', PAT needs to
provide an interface that prevents the OS from initializing the
PAT MSR.

PAT MSR initialization must be done on all CPUs using the specific
sequence of operations defined in Intel SDM. This requires MTRRs
to be enabled since pat_init() is called as part of MTRR init
from mtrr_rendezvous_handler().

Make pat_disable() as the interface that prevents the OS from
initializing the PAT MSR. MTRR will call this interface when it
cannot provide the SDM-defined sequence to initialize PAT.

This also assures pat_disable() called from pat_bsp_init() to
set PAT table properly when CPU does not support PAT.

Signed-off-by: Toshi Kani <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Luis R. Rodriguez <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Robert Elliott <[email protected]>
Cc: Ingo Molnar <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Thomas Gleixner <[email protected]>
---
arch/x86/include/asm/pat.h | 1 +
arch/x86/mm/pat.c | 13 ++++++++++++-
2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/pat.h b/arch/x86/include/asm/pat.h
index 97ea55b..0ad356c 100644
--- a/arch/x86/include/asm/pat.h
+++ b/arch/x86/include/asm/pat.h
@@ -5,6 +5,7 @@
#include <asm/pgtable_types.h>

bool pat_enabled(void);
+void pat_disable(const char *reason);
extern void pat_init(void);
void __init_cache_modes(u64);

diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 1da55a5..3c08a27 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -40,11 +40,22 @@
static bool boot_cpu_done;

static int __read_mostly __pat_enabled = IS_ENABLED(CONFIG_X86_PAT);
+static void init_cache_modes(void);

-static inline void pat_disable(const char *reason)
+void pat_disable(const char *reason)
{
+ if (!__pat_enabled)
+ return;
+
+ if (boot_cpu_done) {
+ WARN_ONCE(1, "x86/PAT: PAT cannot be disabled after initialization\n");
+ return;
+ }
+
__pat_enabled = 0;
pr_info("x86/PAT: %s\n", reason);
+
+ init_cache_modes();
}

static int __init nopat(char *str)

2016-03-29 10:34:51

by Ingo Molnar

Subject: Re: [PATCH v3 0/7] Enhance PAT init to fix Xorg crashes

* Toshi Kani <[email protected]> wrote:

> A Xorg failure on qemu32 was reported as a regression [1] caused by
> 'commit 9cd25aac1f44 ("x86/mm/pat: Emulate PAT when it is disabled")'.
> This patch-set fixes the regression.
>
> Negative effects of this regression were two failures [2] in Xorg on
> QEMU with QEMU CPU model "qemu32" (-cpu qemu32), which were triggered
> by the fact that its virtual CPU does not support MTRR.
> #1. copy_process() failed in the check in reserve_pfn_range()
> #2. error path in copy_process() then hit WARN_ON_ONCE in
> untrack_pfn().
>
> These negative effects are caused by two separate bugs, but they can be
> addressed in separate patches. This patch-set addresses the root cause,
> a long-standing PAT initialization issue.
>
> Please see the changelog in patch 4/7 for the details of the issue.
>
> - Patch 1-2 make necessary enhancement to PAT for the fix without
> breaking Xen.
> - Patch 3 is cleanup.
> - Patch 4 fixes the regression.
> - Patch 5 fixes an MTRR issue related with PAT init.
> - Patch 6 removes PAT init code from Xen.
> - Patch 7 adds PAT init to documentation.
>
> [1]: https://lkml.org/lkml/2016/3/3/828
> [2]: https://lkml.org/lkml/2016/3/4/775

The changelogs are much improved, I've applied these patches to tip:x86/mm,
thanks Toshi!

> I'd appreciate if someone can test this patch-set on Xen to verify that
> there is no change in "x86/PAT: Configuration [0-7] .." message in dmesg.

So I don't have a Xen setup, but hopefully such testing will happen once these
changes show up in linux-next, tomorrow or so.

Thanks,

Ingo

2016-03-29 10:39:45

by tip-bot for Vasyl Gomonovych

Subject: [tip:x86/mm] x86/mm/pat: Add support of non-default PAT MSR setting

Commit-ID: 02f037d641dc6672be5cfe7875a48ab99b95b154
Gitweb: http://git.kernel.org/tip/02f037d641dc6672be5cfe7875a48ab99b95b154
Author: Toshi Kani <[email protected]>
AuthorDate: Wed, 23 Mar 2016 15:41:57 -0600
Committer: Ingo Molnar <[email protected]>
CommitDate: Tue, 29 Mar 2016 12:23:25 +0200

x86/mm/pat: Add support of non-default PAT MSR setting

In preparation for fixing a regression caused by:

9cd25aac1f44 ("x86/mm/pat: Emulate PAT when it is disabled")'

... PAT needs to support a case that PAT MSR is initialized with a
non-default value.

When pat_init() is called and PAT is disabled, it initializes the
PAT table with the BIOS default value. Xen, however, sets PAT MSR
with a non-default value to enable WC. This causes inconsistency
between the PAT table and PAT MSR when PAT is set to disable on Xen.

Change pat_init() to handle the PAT disable cases properly. Add
init_cache_modes() to handle two cases when PAT is set to disable.

1. CPU supports PAT: Set PAT table to be consistent with PAT MSR.
2. CPU does not support PAT: Set PAT table to be consistent with
PWT and PCD bits in a PTE.

Note, __init_cache_modes(), renamed from pat_init_cache_modes(),
will be changed to a static function in a later patch.

Signed-off-by: Toshi Kani <[email protected]>
Reviewed-by: Thomas Gleixner <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: Andy Lutomirski <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Brian Gerst <[email protected]>
Cc: Denys Vlasenko <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Luis R. Rodriguez <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Toshi Kani <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
---
arch/x86/include/asm/pat.h | 2 +-
arch/x86/mm/pat.c | 73 +++++++++++++++++++++++++++++++++-------------
arch/x86/xen/enlighten.c | 2 +-
3 files changed, 55 insertions(+), 22 deletions(-)

diff --git a/arch/x86/include/asm/pat.h b/arch/x86/include/asm/pat.h
index ca6c228..97ea55b 100644
--- a/arch/x86/include/asm/pat.h
+++ b/arch/x86/include/asm/pat.h
@@ -6,7 +6,7 @@

bool pat_enabled(void);
extern void pat_init(void);
-void pat_init_cache_modes(u64);
+void __init_cache_modes(u64);

extern int reserve_memtype(u64 start, u64 end,
enum page_cache_mode req_pcm, enum page_cache_mode *ret_pcm);
diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index faec01e..b4663885 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -181,7 +181,7 @@ static enum page_cache_mode pat_get_cache_mode(unsigned pat_val, char *msg)
* configuration.
* Using lower indices is preferred, so we start with highest index.
*/
-void pat_init_cache_modes(u64 pat)
+void __init_cache_modes(u64 pat)
{
enum page_cache_mode cache;
char pat_msg[33];
@@ -207,9 +207,6 @@ static void pat_bsp_init(u64 pat)
return;
}

- if (!pat_enabled())
- goto done;
-
rdmsrl(MSR_IA32_CR_PAT, tmp_pat);
if (!tmp_pat) {
pat_disable("PAT MSR is 0, disabled.");
@@ -218,15 +215,11 @@ static void pat_bsp_init(u64 pat)

wrmsrl(MSR_IA32_CR_PAT, pat);

-done:
- pat_init_cache_modes(pat);
+ __init_cache_modes(pat);
}

static void pat_ap_init(u64 pat)
{
- if (!pat_enabled())
- return;
-
if (!cpu_has_pat) {
/*
* If this happens we are on a secondary CPU, but switched to
@@ -238,18 +231,32 @@ static void pat_ap_init(u64 pat)
wrmsrl(MSR_IA32_CR_PAT, pat);
}

-void pat_init(void)
+static void init_cache_modes(void)
{
- u64 pat;
- struct cpuinfo_x86 *c = &boot_cpu_data;
+ u64 pat = 0;
+ static int init_cm_done;

- if (!pat_enabled()) {
+ if (init_cm_done)
+ return;
+
+ if (boot_cpu_has(X86_FEATURE_PAT)) {
+ /*
+ * CPU supports PAT. Set PAT table to be consistent with
+ * PAT MSR. This case supports "nopat" boot option, and
+ * virtual machine environments which support PAT without
+ * MTRRs. In specific, Xen has unique setup to PAT MSR.
+ *
+ * If PAT MSR returns 0, it is considered invalid and emulates
+ * as No PAT.
+ */
+ rdmsrl(MSR_IA32_CR_PAT, pat);
+ }
+
+ if (!pat) {
/*
* No PAT. Emulate the PAT table that corresponds to the two
- * cache bits, PWT (Write Through) and PCD (Cache Disable). This
- * setup is the same as the BIOS default setup when the system
- * has PAT but the "nopat" boot option has been specified. This
- * emulated PAT table is used when MSR_IA32_CR_PAT returns 0.
+ * cache bits, PWT (Write Through) and PCD (Cache Disable).
+ * This setup is also the same as the BIOS default setup.
*
* PTE encoding:
*
@@ -266,10 +273,36 @@ void pat_init(void)
*/
pat = PAT(0, WB) | PAT(1, WT) | PAT(2, UC_MINUS) | PAT(3, UC) |
PAT(4, WB) | PAT(5, WT) | PAT(6, UC_MINUS) | PAT(7, UC);
+ }
+
+ __init_cache_modes(pat);
+
+ init_cm_done = 1;
+}
+
+/**
+ * pat_init - Initialize PAT MSR and PAT table
+ *
+ * This function initializes PAT MSR and PAT table with an OS-defined value
+ * to enable additional cache attributes, WC and WT.
+ *
+ * This function must be called on all CPUs using the specific sequence of
+ * operations defined in Intel SDM. mtrr_rendezvous_handler() provides this
+ * procedure for PAT.
+ */
+void pat_init(void)
+{
+ u64 pat;
+ struct cpuinfo_x86 *c = &boot_cpu_data;
+
+ if (!pat_enabled()) {
+ init_cache_modes();
+ return;
+ }

- } else if ((c->x86_vendor == X86_VENDOR_INTEL) &&
- (((c->x86 == 0x6) && (c->x86_model <= 0xd)) ||
- ((c->x86 == 0xf) && (c->x86_model <= 0x6)))) {
+ if ((c->x86_vendor == X86_VENDOR_INTEL) &&
+ (((c->x86 == 0x6) && (c->x86_model <= 0xd)) ||
+ ((c->x86 == 0xf) && (c->x86_model <= 0x6)))) {
/*
* PAT support with the lower four entries. Intel Pentium 2,
* 3, M, and 4 are affected by PAT errata, which makes the
diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index 880862c..c469a7c 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -1623,7 +1623,7 @@ asmlinkage __visible void __init xen_start_kernel(void)
* configuration.
*/
rdmsrl(MSR_IA32_CR_PAT, pat);
- pat_init_cache_modes(pat);
+ __init_cache_modes(pat);

/* keep using Xen gdt for now; no urgent need to change it */

2016-03-29 10:39:43

by tip-bot for Vasyl Gomonovych

Subject: [tip:x86/mm] x86/mm/pat: Add pat_disable() interface

Commit-ID: 224bb1e5d67ba0f2872c98002d6a6f991ac6fd4a
Gitweb: http://git.kernel.org/tip/224bb1e5d67ba0f2872c98002d6a6f991ac6fd4a
Author: Toshi Kani <[email protected]>
AuthorDate: Wed, 23 Mar 2016 15:41:58 -0600
Committer: Ingo Molnar <[email protected]>
CommitDate: Tue, 29 Mar 2016 12:23:25 +0200

x86/mm/pat: Add pat_disable() interface

In preparation for fixing a regression caused by:

9cd25aac1f44 ("x86/mm/pat: Emulate PAT when it is disabled")

... PAT needs to provide an interface that prevents the OS from
initializing the PAT MSR.

PAT MSR initialization must be done on all CPUs using the specific
sequence of operations defined in the Intel SDM. This requires MTRRs
to be enabled since pat_init() is called as part of MTRR init
from mtrr_rendezvous_handler().

Make pat_disable() as the interface that prevents the OS from
initializing the PAT MSR. MTRR will call this interface when it
cannot provide the SDM-defined sequence to initialize PAT.

This also assures that pat_disable() called from pat_bsp_init()
will set the PAT table properly when CPU does not support PAT.

Signed-off-by: Toshi Kani <[email protected]>
Reviewed-by: Thomas Gleixner <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: Andy Lutomirski <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Brian Gerst <[email protected]>
Cc: Denys Vlasenko <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Luis R. Rodriguez <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Robert Elliott <[email protected]>
Cc: Toshi Kani <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
---
arch/x86/include/asm/pat.h | 1 +
arch/x86/mm/pat.c | 13 ++++++++++++-
2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/arch/x86/include/asm/pat.h b/arch/x86/include/asm/pat.h
index 97ea55b..0ad356c 100644
--- a/arch/x86/include/asm/pat.h
+++ b/arch/x86/include/asm/pat.h
@@ -5,6 +5,7 @@
#include <asm/pgtable_types.h>

bool pat_enabled(void);
+void pat_disable(const char *reason);
extern void pat_init(void);
void __init_cache_modes(u64);

diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index b4663885..1cc1d37 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -40,11 +40,22 @@
static bool boot_cpu_done;

static int __read_mostly __pat_enabled = IS_ENABLED(CONFIG_X86_PAT);
+static void init_cache_modes(void);

-static inline void pat_disable(const char *reason)
+void pat_disable(const char *reason)
{
+ if (!__pat_enabled)
+ return;
+
+ if (boot_cpu_done) {
+ WARN_ONCE(1, "x86/PAT: PAT cannot be disabled after initialization\n");
+ return;
+ }
+
__pat_enabled = 0;
pr_info("x86/PAT: %s\n", reason);
+
+ init_cache_modes();
}

static int __init nopat(char *str)

2016-03-29 10:40:24

by tip-bot for Vasyl Gomonovych

Subject: [tip:x86/mm] x86/mm/pat: Replace cpu_has_pat with boot_cpu_has()

Commit-ID: d63dcf49cf5ae5605f4d14229e3888e104f294b1
Gitweb: http://git.kernel.org/tip/d63dcf49cf5ae5605f4d14229e3888e104f294b1
Author: Toshi Kani <[email protected]>
AuthorDate: Wed, 23 Mar 2016 15:41:59 -0600
Committer: Ingo Molnar <[email protected]>
CommitDate: Tue, 29 Mar 2016 12:23:26 +0200

x86/mm/pat: Replace cpu_has_pat with boot_cpu_has()

Borislav Petkov suggested:

> Please use on init paths boot_cpu_has(X86_FEATURE_PAT) and on fast
> paths static_cpu_has(X86_FEATURE_PAT). No more of that cpu_has_XXX
> ugliness.

Replace the use of cpu_has_pat on init paths with boot_cpu_has().

Suggested-by: Borislav Petkov <[email protected]>
Signed-off-by: Toshi Kani <[email protected]>
Reviewed-by: Thomas Gleixner <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: Andy Lutomirski <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Brian Gerst <[email protected]>
Cc: Denys Vlasenko <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Luis R. Rodriguez <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Robert Elliott <[email protected]>
Cc: Toshi Kani <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
---
arch/x86/mm/pat.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 1cc1d37..59ec038 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -213,7 +213,7 @@ static void pat_bsp_init(u64 pat)
{
u64 tmp_pat;

- if (!cpu_has_pat) {
+ if (!boot_cpu_has(X86_FEATURE_PAT)) {
pat_disable("PAT not supported by CPU.");
return;
}
@@ -231,7 +231,7 @@ static void pat_bsp_init(u64 pat)

static void pat_ap_init(u64 pat)
{
- if (!cpu_has_pat) {
+ if (!boot_cpu_has(X86_FEATURE_PAT)) {
/*
* If this happens we are on a secondary CPU, but switched to
* PAT on the boot CPU. We have no way to undo PAT.

2016-03-29 10:41:00

by tip-bot for Vasyl Gomonovych

Subject: [tip:x86/mm] x86/mtrr: Fix PAT init handling when MTRR is disabled

Commit-ID: ad025a73f0e9344ac73ffe1b74c184033e08e7d5
Gitweb: http://git.kernel.org/tip/ad025a73f0e9344ac73ffe1b74c184033e08e7d5
Author: Toshi Kani <[email protected]>
AuthorDate: Wed, 23 Mar 2016 15:42:01 -0600
Committer: Ingo Molnar <[email protected]>
CommitDate: Tue, 29 Mar 2016 12:23:26 +0200

x86/mtrr: Fix PAT init handling when MTRR is disabled

get_mtrr_state() calls pat_init() on BSP even if MTRR is disabled.
This results in calling pat_init() on BSP only since APs do not call
pat_init() when MTRR is disabled. This inconsistency between BSP
and APs leads to undefined behavior.

Make BSP's calling condition to pat_init() consistent with AP's,
mtrr_ap_init() and mtrr_aps_init().

Signed-off-by: Toshi Kani <[email protected]>
Reviewed-by: Thomas Gleixner <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: Andy Lutomirski <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Brian Gerst <[email protected]>
Cc: Denys Vlasenko <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Luis R. Rodriguez <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Toshi Kani <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
---
arch/x86/kernel/cpu/mtrr/generic.c | 24 ++++++++++++++----------
arch/x86/kernel/cpu/mtrr/main.c | 3 +++
arch/x86/kernel/cpu/mtrr/mtrr.h | 1 +
3 files changed, 18 insertions(+), 10 deletions(-)

diff --git a/arch/x86/kernel/cpu/mtrr/generic.c b/arch/x86/kernel/cpu/mtrr/generic.c
index 19f5736..8d7a29e 100644
--- a/arch/x86/kernel/cpu/mtrr/generic.c
+++ b/arch/x86/kernel/cpu/mtrr/generic.c
@@ -444,11 +444,24 @@ static void __init print_mtrr_state(void)
pr_debug("TOM2: %016llx aka %lldM\n", mtrr_tom2, mtrr_tom2>>20);
}

+/* PAT setup for BP. We need to go through sync steps here */
+void __init mtrr_bp_pat_init(void)
+{
+ unsigned long flags;
+
+ local_irq_save(flags);
+ prepare_set();
+
+ pat_init();
+
+ post_set();
+ local_irq_restore(flags);
+}
+
/* Grab all of the MTRR state for this CPU into *state */
bool __init get_mtrr_state(void)
{
struct mtrr_var_range *vrs;
- unsigned long flags;
unsigned lo, dummy;
unsigned int i;

@@ -481,15 +494,6 @@ bool __init get_mtrr_state(void)

mtrr_state_set = 1;

- /* PAT setup for BP. We need to go through sync steps here */
- local_irq_save(flags);
- prepare_set();
-
- pat_init();
-
- post_set();
- local_irq_restore(flags);
-
return !!(mtrr_state.enabled & MTRR_STATE_MTRR_ENABLED);
}

diff --git a/arch/x86/kernel/cpu/mtrr/main.c b/arch/x86/kernel/cpu/mtrr/main.c
index 8b1947b..7d393ec 100644
--- a/arch/x86/kernel/cpu/mtrr/main.c
+++ b/arch/x86/kernel/cpu/mtrr/main.c
@@ -752,6 +752,9 @@ void __init mtrr_bp_init(void)
/* BIOS may override */
__mtrr_enabled = get_mtrr_state();

+ if (mtrr_enabled())
+ mtrr_bp_pat_init();
+
if (mtrr_cleanup(phys_addr)) {
changed_by_mtrr_cleanup = 1;
mtrr_if->set_all();
diff --git a/arch/x86/kernel/cpu/mtrr/mtrr.h b/arch/x86/kernel/cpu/mtrr/mtrr.h
index 951884d..6c7ced0 100644
--- a/arch/x86/kernel/cpu/mtrr/mtrr.h
+++ b/arch/x86/kernel/cpu/mtrr/mtrr.h
@@ -52,6 +52,7 @@ void set_mtrr_prepare_save(struct set_mtrr_context *ctxt);
void fill_mtrr_var_range(unsigned int index,
u32 base_lo, u32 base_hi, u32 mask_lo, u32 mask_hi);
bool get_mtrr_state(void);
+void mtrr_bp_pat_init(void);

extern void set_mtrr_ops(const struct mtrr_ops *ops);

2016-03-29 10:41:36

by tip-bot for Vasyl Gomonovych

Subject: [tip:x86/mm] x86/xen, pat: Remove PAT table init code from Xen

Commit-ID: 88ba281108ed0c25c9d292b48bd3f272fcb90dd0
Gitweb: http://git.kernel.org/tip/88ba281108ed0c25c9d292b48bd3f272fcb90dd0
Author: Toshi Kani <[email protected]>
AuthorDate: Wed, 23 Mar 2016 15:42:02 -0600
Committer: Ingo Molnar <[email protected]>
CommitDate: Tue, 29 Mar 2016 12:23:27 +0200

x86/xen, pat: Remove PAT table init code from Xen

Xen supports PAT without MTRRs for its guests. In order to
enable WC attribute, it was necessary for xen_start_kernel()
to call pat_init_cache_modes() to update PAT table before
starting guest kernel.

Now that the kernel initializes PAT table to the BIOS handoff
state when MTRR is disabled, this Xen-specific PAT init code
is no longer necessary. Delete it from xen_start_kernel().

Also change __init_cache_modes() to a static function since
PAT table should not be tweaked by other modules.

Signed-off-by: Toshi Kani <[email protected]>
Reviewed-by: Thomas Gleixner <[email protected]>
Acked-by: Juergen Gross <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: Andy Lutomirski <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Brian Gerst <[email protected]>
Cc: Denys Vlasenko <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Konrad Rzeszutek Wilk <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Luis R. Rodriguez <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Toshi Kani <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
---
arch/x86/include/asm/pat.h | 1 -
arch/x86/mm/pat.c | 2 +-
arch/x86/xen/enlighten.c | 9 ---------
3 files changed, 1 insertion(+), 11 deletions(-)

diff --git a/arch/x86/include/asm/pat.h b/arch/x86/include/asm/pat.h
index 0ad356c..0b1ff4c 100644
--- a/arch/x86/include/asm/pat.h
+++ b/arch/x86/include/asm/pat.h
@@ -7,7 +7,6 @@
bool pat_enabled(void);
void pat_disable(const char *reason);
extern void pat_init(void);
-void __init_cache_modes(u64);

extern int reserve_memtype(u64 start, u64 end,
enum page_cache_mode req_pcm, enum page_cache_mode *ret_pcm);
diff --git a/arch/x86/mm/pat.c b/arch/x86/mm/pat.c
index 59ec038..c4c3ddc 100644
--- a/arch/x86/mm/pat.c
+++ b/arch/x86/mm/pat.c
@@ -192,7 +192,7 @@ static enum page_cache_mode pat_get_cache_mode(unsigned pat_val, char *msg)
* configuration.
* Using lower indices is preferred, so we start with highest index.
*/
-void __init_cache_modes(u64 pat)
+static void __init_cache_modes(u64 pat)
{
enum page_cache_mode cache;
char pat_msg[33];
diff --git a/arch/x86/xen/enlighten.c b/arch/x86/xen/enlighten.c
index c469a7c..d8cca75 100644
--- a/arch/x86/xen/enlighten.c
+++ b/arch/x86/xen/enlighten.c
@@ -75,7 +75,6 @@
#include <asm/mach_traps.h>
#include <asm/mwait.h>
#include <asm/pci_x86.h>
-#include <asm/pat.h>
#include <asm/cpu.h>

#ifdef CONFIG_ACPI
@@ -1511,7 +1510,6 @@ asmlinkage __visible void __init xen_start_kernel(void)
{
struct physdev_set_iopl set_iopl;
unsigned long initrd_start = 0;
- u64 pat;
int rc;

if (!xen_start_info)
@@ -1618,13 +1616,6 @@ asmlinkage __visible void __init xen_start_kernel(void)
xen_start_info->nr_pages);
xen_reserve_special_pages();

- /*
- * Modify the cache mode translation tables to match Xen's PAT
- * configuration.
- */
- rdmsrl(MSR_IA32_CR_PAT, pat);
- __init_cache_modes(pat);
-
/* keep using Xen gdt for now; no urgent need to change it */

#ifdef CONFIG_X86_32

2016-03-29 10:41:47

by tip-bot for Vasyl Gomonovych

Subject: [tip:x86/mm] x86/mtrr: Fix Xorg crashes in Qemu sessions

Commit-ID: edfe63ec97ed8d4496225f7ba54c9ce4207c5431
Gitweb: http://git.kernel.org/tip/edfe63ec97ed8d4496225f7ba54c9ce4207c5431
Author: Toshi Kani <[email protected]>
AuthorDate: Wed, 23 Mar 2016 15:42:00 -0600
Committer: Ingo Molnar <[email protected]>
CommitDate: Tue, 29 Mar 2016 12:23:26 +0200

x86/mtrr: Fix Xorg crashes in Qemu sessions

A Xorg failure on qemu32 was reported as a regression [1] caused by
commit 9cd25aac1f44 ("x86/mm/pat: Emulate PAT when it is disabled").

This patch fixes the Xorg crash.

Negative effects of this regression were the following two failures [2]
in Xorg on QEMU with QEMU CPU model "qemu32" (-cpu qemu32), which were
triggered by the fact that its virtual CPU does not support MTRRs.

#1. copy_process() failed in the check in reserve_pfn_range()

copy_process
copy_mm
dup_mm
dup_mmap
copy_page_range
track_pfn_copy
reserve_pfn_range

A WC map request was tracked as WC in memtype, which set a PTE as
UC (pgprot) per __cachemode2pte_tbl[]. This led to this error in
reserve_pfn_range() called from track_pfn_copy(), which obtained
a pgprot from a PTE. It converts pgprot to page_cache_mode, which
does not necessarily result in the original page_cache_mode since
__cachemode2pte_tbl[] redirects multiple types to UC.

#2. error path in copy_process() then hit WARN_ON_ONCE in
untrack_pfn().

x86/PAT: Xorg:509 map pfn expected mapping type uncached-
minus for [mem 0xfd000000-0xfdffffff], got write-combining
Call Trace:
dump_stack
warn_slowpath_common
? untrack_pfn
? untrack_pfn
warn_slowpath_null
untrack_pfn
? __kunmap_atomic
unmap_single_vma
? pagevec_move_tail_fn
unmap_vmas
exit_mmap
mmput
copy_process.part.47
_do_fork
SyS_clone
do_syscall_32_irqs_on
entry_INT80_32

These negative effects are caused by two separate bugs, but they
can be addressed in separate patches. Fixing the pat_init() issue
described below addresses the root cause, and avoids Xorg to hit
these cases.

When the CPU does not support MTRRs, MTRR does not call pat_init(),
which leaves PAT enabled without initializing PAT. This pat_init()
issue is a long-standing issue, but manifested as issue #1 (and then
hit issue #2) with the above-mentioned commit because the memtype
now tracks cache attribute with 'page_cache_mode'.

This pat_init() issue existed before the commit, but we used pgprot
in memtype. Hence, we did not have issue #1 before. But WC request
resulted in WT in effect because WC pgrot is actually WT when PAT
is not initialized. This is not how it was designed to work. When
PAT is set to disable properly, WC is converted to UC. The use of
WT can result in a system crash if the target range does not support
WT. Fortunately, nobody ran into such issue before.

To fix this pat_init() issue, PAT code has been enhanced to provide
pat_disable() interface. Call this interface when MTRRs are disabled.
By setting PAT to disable properly, PAT bypasses the memtype check,
and avoids issue #1.

[1]: https://lkml.org/lkml/2016/3/3/828
[2]: https://lkml.org/lkml/2016/3/4/775

Signed-off-by: Toshi Kani <[email protected]>
Reviewed-by: Thomas Gleixner <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: Andy Lutomirski <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Brian Gerst <[email protected]>
Cc: Denys Vlasenko <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Luis R. Rodriguez <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Toshi Kani <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
---
arch/x86/include/asm/mtrr.h | 6 +++++-
arch/x86/kernel/cpu/mtrr/main.c | 10 +++++++++-
2 files changed, 14 insertions(+), 2 deletions(-)

diff --git a/arch/x86/include/asm/mtrr.h b/arch/x86/include/asm/mtrr.h
index b94f6f6..dbff145 100644
--- a/arch/x86/include/asm/mtrr.h
+++ b/arch/x86/include/asm/mtrr.h
@@ -24,6 +24,7 @@
#define _ASM_X86_MTRR_H

#include <uapi/asm/mtrr.h>
+#include <asm/pat.h>


/*
@@ -83,9 +84,12 @@ static inline int mtrr_trim_uncached_memory(unsigned long end_pfn)
static inline void mtrr_centaur_report_mcr(int mcr, u32 lo, u32 hi)
{
}
+static inline void mtrr_bp_init(void)
+{
+ pat_disable("MTRRs disabled, skipping PAT initialization too.");
+}

#define mtrr_ap_init() do {} while (0)
-#define mtrr_bp_init() do {} while (0)
#define set_mtrr_aps_delayed_init() do {} while (0)
#define mtrr_aps_init() do {} while (0)
#define mtrr_bp_restore() do {} while (0)
diff --git a/arch/x86/kernel/cpu/mtrr/main.c b/arch/x86/kernel/cpu/mtrr/main.c
index 10f8d47..8b1947b 100644
--- a/arch/x86/kernel/cpu/mtrr/main.c
+++ b/arch/x86/kernel/cpu/mtrr/main.c
@@ -759,8 +759,16 @@ void __init mtrr_bp_init(void)
}
}

- if (!mtrr_enabled())
+ if (!mtrr_enabled()) {
pr_info("MTRR: Disabled\n");
+
+ /*
+ * PAT initialization relies on MTRR's rendezvous handler.
+ * Skip PAT init until the handler can initialize both
+ * features independently.
+ */
+ pat_disable("MTRRs disabled, skipping PAT initialization too.");
+ }
}

void mtrr_ap_init(void)

2016-03-29 10:41:45

by tip-bot for Vasyl Gomonovych

Subject: [tip:x86/mm] x86/pat: Document the PAT initialization sequence

Commit-ID: b6350c21cfe8aa9d65e189509a23c0ea4b8362c2
Gitweb: http://git.kernel.org/tip/b6350c21cfe8aa9d65e189509a23c0ea4b8362c2
Author: Toshi Kani <[email protected]>
AuthorDate: Wed, 23 Mar 2016 15:42:03 -0600
Committer: Ingo Molnar <[email protected]>
CommitDate: Tue, 29 Mar 2016 12:23:27 +0200

x86/pat: Document the PAT initialization sequence

Update PAT documentation to describe how PAT is initialized under
various configurations.

Signed-off-by: Toshi Kani <[email protected]>
Reviewed-by: Thomas Gleixner <[email protected]>
Cc: Andrew Morton <[email protected]>
Cc: Andy Lutomirski <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Borislav Petkov <[email protected]>
Cc: Brian Gerst <[email protected]>
Cc: Denys Vlasenko <[email protected]>
Cc: H. Peter Anvin <[email protected]>
Cc: Juergen Gross <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Luis R. Rodriguez <[email protected]>
Cc: Peter Zijlstra <[email protected]>
Cc: Toshi Kani <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Link: http://lkml.kernel.org/r/[email protected]
Signed-off-by: Ingo Molnar <[email protected]>
---
Documentation/x86/pat.txt | 32 ++++++++++++++++++++++++++++++++
1 file changed, 32 insertions(+)

diff --git a/Documentation/x86/pat.txt b/Documentation/x86/pat.txt
index 54944c7..2a4ee63 100644
--- a/Documentation/x86/pat.txt
+++ b/Documentation/x86/pat.txt
@@ -196,3 +196,35 @@ Another, more verbose way of getting PAT related debug messages is with
"debugpat" boot parameter. With this parameter, various debug messages are
printed to dmesg log.

+PAT Initialization
+------------------
+
+The following table describes how PAT is initialized under various
+configurations. The PAT MSR must be updated by Linux in order to support WC
+and WT attributes. Otherwise, the PAT MSR has the value programmed in it
+by the firmware. Note, Xen enables WC attribute in the PAT MSR for guests.
+
+ MTRR PAT Call Sequence PAT State PAT MSR
+ =========================================================
+ E E MTRR -> PAT init Enabled OS
+ E D MTRR -> PAT init Disabled -
+ D E MTRR -> PAT disable Disabled BIOS
+ D D MTRR -> PAT disable Disabled -
+ - np/E PAT -> PAT disable Disabled BIOS
+ - np/D PAT -> PAT disable Disabled -
+ E !P/E MTRR -> PAT init Disabled BIOS
+ D !P/E MTRR -> PAT disable Disabled BIOS
+ !M !P/E MTRR stub -> PAT disable Disabled BIOS
+
+ Legend
+ ------------------------------------------------
+ E Feature enabled in CPU
+ D Feature disabled/unsupported in CPU
+ np "nopat" boot option specified
+ !P CONFIG_X86_PAT option unset
+ !M CONFIG_MTRR option unset
+ Enabled PAT state set to enabled
+ Disabled PAT state set to disabled
+ OS PAT initializes PAT MSR with OS setting
+ BIOS PAT keeps PAT MSR with BIOS setting
+

2016-03-29 13:27:16

by Kani, Toshimitsu

Subject: Re: [PATCH v3 0/7] Enhance PAT init to fix Xorg crashes

On Tue, 2016-03-29 at 12:34 +0200, Ingo Molnar wrote:
> * Toshi Kani <[email protected]> wrote:
>
> > A Xorg failure on qemu32 was reported as a regression [1] caused by
> > 'commit 9cd25aac1f44 ("x86/mm/pat: Emulate PAT when it is disabled")'.
> > This patch-set fixes the regression.
> >
> > Negative effects of this regression were two failures [2] in Xorg on
> > QEMU with QEMU CPU model "qemu32" (-cpu qemu32), which were triggered
> > by the fact that its virtual CPU does not support MTRR.
> >  #1. copy_process() failed in the check in reserve_pfn_range()
> >  #2. error path in copy_process() then hit WARN_ON_ONCE in
> >      untrack_pfn().
> >
> > These negative effects are caused by two separate bugs, but they can be
> > addressed in separate patches.  This patch-set addresses the root
> > cause, a long-standing PAT initialization issue.
> >
> > Please see the changelog in patch 4/7 for the details of the issue.
> >
> > - Patch 1-2 make necessary enhancement to PAT for the fix without
> >   breaking Xen.
> > - Patch 3 is cleanup.
> > - Patch 4 fixes the regression.
> > - Patch 5 fixes an MTRR issue related with PAT init.
> > - Patch 6 removes PAT init code from Xen.
> > - Patch 7 adds PAT init to documentation.
> >
> > [1]: https://lkml.org/lkml/2016/3/3/828
> > [2]: https://lkml.org/lkml/2016/3/4/775
>
> The changelogs are much improved, I've applied these patches to
> tip:x86/mm, thanks Toshi!

Great! Thanks for all valuable comments!

> > I'd appreciate if someone can test this patch-set on Xen to verify that
> > there is no change in "x86/PAT: Configuration [0-7] .." message in
> > dmesg.
>
> So I don't have a Xen setup, but hopefully such testing will happen once
> these changes show up in linux-next, tomorrow or so.

I will address if any issue is found in testing.

-Toshi

2016-03-29 14:47:25

by Boris Ostrovsky

Subject: Re: [Xen-devel] [PATCH v3 0/7] Enhance PAT init to fix Xorg crashes

On 03/29/2016 10:19 AM, Toshi Kani wrote:
> On Tue, 2016-03-29 at 12:34 +0200, Ingo Molnar wrote:
>
>>> I'd appreciate if someone can test this patch-set on Xen to verify that
>>> there is no change in "x86/PAT: Configuration [0-7] .." message in
>>> dmesg.
>> So I don't have a Xen setup, but hopefully such testing will happen once
>> these changes show up in linux-next, tomorrow or so.
> I will address if any issue is found in testing.

I ran a subset of out nightly test. Nobody died.

So this all looks good. (It actually may have also fixed another bug
that was reported recently by Olaf, copied here)

-boris

2016-03-29 14:57:16

by Kani, Toshimitsu

Subject: Re: [Xen-devel] [PATCH v3 0/7] Enhance PAT init to fix Xorg crashes

On Tue, 2016-03-29 at 10:46 -0400, Boris Ostrovsky wrote:
> On 03/29/2016 10:19 AM, Toshi Kani wrote:
> > On Tue, 2016-03-29 at 12:34 +0200, Ingo Molnar wrote:
> >
> > > > I'd appreciate if someone can test this patch-set on Xen to verify
> > > > that there is no change in "x86/PAT: Configuration [0-7] .."
> > > > message in dmesg.
> > > So I don't have a Xen setup, but hopefully such testing will happen
> > > once these changes show up in linux-next, tomorrow or so.
> > I will address if any issue is found in testing.
>
> I ran a subset of out nightly test. Nobody died.
>
> So this all looks good. (It actually may have also fixed another bug 
> that was reported recently by Olaf, copied here)

Cool! Thanks Boris!
-Toshi

2016-03-30 18:02:35

by Luis Chamberlain

Subject: Re: [Xen-devel] [PATCH v3 0/7] Enhance PAT init to fix Xorg crashes

On Tue, Mar 29, 2016 at 8:49 AM, Toshi Kani <[email protected]> wrote:
> On Tue, 2016-03-29 at 10:46 -0400, Boris Ostrovsky wrote:
>> On 03/29/2016 10:19 AM, Toshi Kani wrote:
>> > On Tue, 2016-03-29 at 12:34 +0200, Ingo Molnar wrote:
>> >
>> > > > I'd appreciate if someone can test this patch-set on Xen to verify
>> > > > that there is no change in "x86/PAT: Configuration [0-7] .."
>> > > > message in dmesg.
>> > > So I don't have a Xen setup, but hopefully such testing will happen
>> > > once these changes show up in linux-next, tomorrow or so.
>> > I will address if any issue is found in testing.
>>
>> I ran a subset of out nightly test. Nobody died.
>>
>> So this all looks good. (It actually may have also fixed another bug
>> that was reported recently by Olaf, copied here)
>
> Cool! Thanks Boris!

Hoping Boris or someone on the Xen front would test this prior to
merging helps but it also slows us down, a while ago we discussed the
possibility of getting Linux Xen guests automatically tested as part
of 0-day, this way then when a developer (in this case Toshi) pushes
to his own tree, he'd be able to just sit and wait for the results,
without having to hope Boris or someone goes out and tests.

Its a major undertaking to get Linux Xen guests boot strapped into
0-day, however such prospects were raised a while ago and it seems we
may be able to get there. Just wanted to send a reminder about this
possibility, and highlight this patch set as an ideal candidate where
a proactive test could have helped as well. I proactively caught the
original Xen issue, but git logs shows we are not doing so great in
this area, so any help on the Xen side as well to help with 0-day
integration would be appreciated.

I'll follow up on another thread on that.

Luis