Two users have reported [1] that they have an "extremely unlikely" system
with more than MAX_PA/2 memory and L1TF mitigation is not effective. In fact
it's a CPU with 36bits phys limit (64GB) and 32GB memory, but due to holes
in the e820 map, the main region is almost 500MB over the 32GB limit:
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000081effffff] usable
Suggestions to use 'mem=32G' to prefer L1TF mitigation while losing the 500MB
revealed, that there's an off-by-one error in the check in
l1tf_select_mitigation(). l1tf_pfn_limit() returns the last usable pfn
(inclusive), but it's more common and hopefully less error-prone to return the
first pfn that's over limit, so this patch changes that and updates the other
callers.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
Reported-by: George Anchev <[email protected]>
Reported-by: Christopher Snowhill <[email protected]>
Fixes: 17dbca119312 ("x86/speculation/l1tf: Add sysfs reporting for l1tf")
Cc: [email protected]
Signed-off-by: Vlastimil Babka <[email protected]>
---
arch/x86/include/asm/processor.h | 2 +-
arch/x86/mm/init.c | 2 +-
arch/x86/mm/mmap.c | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index a0a52274cb4a..c24297268ebc 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -183,7 +183,7 @@ extern void cpu_detect(struct cpuinfo_x86 *c);
static inline unsigned long long l1tf_pfn_limit(void)
{
- return BIT_ULL(boot_cpu_data.x86_phys_bits - 1 - PAGE_SHIFT) - 1;
+ return BIT_ULL(boot_cpu_data.x86_phys_bits - 1 - PAGE_SHIFT);
}
extern void early_cpu_init(void);
diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c
index 02de3d6065c4..63a6f9fcaf20 100644
--- a/arch/x86/mm/init.c
+++ b/arch/x86/mm/init.c
@@ -923,7 +923,7 @@ unsigned long max_swapfile_size(void)
if (boot_cpu_has_bug(X86_BUG_L1TF)) {
/* Limit the swap file size to MAX_PA/2 for L1TF workaround */
- unsigned long long l1tf_limit = l1tf_pfn_limit() + 1;
+ unsigned long long l1tf_limit = l1tf_pfn_limit();
/*
* We encode swap offsets also with 3 bits below those for pfn
* which makes the usable limit higher.
diff --git a/arch/x86/mm/mmap.c b/arch/x86/mm/mmap.c
index f40ab8185d94..1e95d57760cf 100644
--- a/arch/x86/mm/mmap.c
+++ b/arch/x86/mm/mmap.c
@@ -257,7 +257,7 @@ bool pfn_modify_allowed(unsigned long pfn, pgprot_t prot)
/* If it's real memory always allow */
if (pfn_valid(pfn))
return true;
- if (pfn > l1tf_pfn_limit() && !capable(CAP_SYS_ADMIN))
+ if (pfn >= l1tf_pfn_limit() && !capable(CAP_SYS_ADMIN))
return false;
return true;
}
--
2.18.0
On Thu 23-08-18 15:44:18, Vlastimil Babka wrote:
> Two users have reported [1] that they have an "extremely unlikely" system
> with more than MAX_PA/2 memory and L1TF mitigation is not effective. In fact
> it's a CPU with 36bits phys limit (64GB) and 32GB memory, but due to holes
> in the e820 map, the main region is almost 500MB over the 32GB limit:
>
> [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000081effffff] usable
>
> Suggestions to use 'mem=32G' to prefer L1TF mitigation while losing the 500MB
> revealed, that there's an off-by-one error in the check in
> l1tf_select_mitigation(). l1tf_pfn_limit() returns the last usable pfn
> (inclusive), but it's more common and hopefully less error-prone to return the
> first pfn that's over limit, so this patch changes that and updates the other
> callers.
>
> [1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
>
> Reported-by: George Anchev <[email protected]>
> Reported-by: Christopher Snowhill <[email protected]>
> Fixes: 17dbca119312 ("x86/speculation/l1tf: Add sysfs reporting for l1tf")
> Cc: [email protected]
> Signed-off-by: Vlastimil Babka <[email protected]>
Yes this should be less error prone.
Acked-by: Michal Hocko <[email protected]>
> ---
> arch/x86/include/asm/processor.h | 2 +-
> arch/x86/mm/init.c | 2 +-
> arch/x86/mm/mmap.c | 2 +-
> 3 files changed, 3 insertions(+), 3 deletions(-)
>
> diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
> index a0a52274cb4a..c24297268ebc 100644
> --- a/arch/x86/include/asm/processor.h
> +++ b/arch/x86/include/asm/processor.h
> @@ -183,7 +183,7 @@ extern void cpu_detect(struct cpuinfo_x86 *c);
>
> static inline unsigned long long l1tf_pfn_limit(void)
> {
> - return BIT_ULL(boot_cpu_data.x86_phys_bits - 1 - PAGE_SHIFT) - 1;
> + return BIT_ULL(boot_cpu_data.x86_phys_bits - 1 - PAGE_SHIFT);
> }
>
> extern void early_cpu_init(void);
> diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c
> index 02de3d6065c4..63a6f9fcaf20 100644
> --- a/arch/x86/mm/init.c
> +++ b/arch/x86/mm/init.c
> @@ -923,7 +923,7 @@ unsigned long max_swapfile_size(void)
>
> if (boot_cpu_has_bug(X86_BUG_L1TF)) {
> /* Limit the swap file size to MAX_PA/2 for L1TF workaround */
> - unsigned long long l1tf_limit = l1tf_pfn_limit() + 1;
> + unsigned long long l1tf_limit = l1tf_pfn_limit();
> /*
> * We encode swap offsets also with 3 bits below those for pfn
> * which makes the usable limit higher.
> diff --git a/arch/x86/mm/mmap.c b/arch/x86/mm/mmap.c
> index f40ab8185d94..1e95d57760cf 100644
> --- a/arch/x86/mm/mmap.c
> +++ b/arch/x86/mm/mmap.c
> @@ -257,7 +257,7 @@ bool pfn_modify_allowed(unsigned long pfn, pgprot_t prot)
> /* If it's real memory always allow */
> if (pfn_valid(pfn))
> return true;
> - if (pfn > l1tf_pfn_limit() && !capable(CAP_SYS_ADMIN))
> + if (pfn >= l1tf_pfn_limit() && !capable(CAP_SYS_ADMIN))
> return false;
> return true;
> }
> --
> 2.18.0
--
Michal Hocko
SUSE Labs
Two users have reported [1] that they have an "extremely unlikely" system
with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's
make the warning more helpful by suggesting the proper mem=X kernel boot param,
a rough calculation of how much RAM can be lost (not precise if there's holes
between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document
to help decide if the mitigation is worth the unusable RAM.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
Suggested-by: Michal Hocko <[email protected]>
Cc: [email protected]
Signed-off-by: Vlastimil Babka <[email protected]>
---
arch/x86/kernel/cpu/bugs.c | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index cb4a16292aa7..4b820bb6c504 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -702,6 +702,11 @@ static void __init l1tf_select_mitigation(void)
half_pa = (u64)l1tf_pfn_limit() << PAGE_SHIFT;
if (e820__mapped_any(half_pa, ULLONG_MAX - half_pa, E820_TYPE_RAM)) {
pr_warn("System has more than MAX_PA/2 memory. L1TF mitigation not effective.\n");
+ pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n",
+ half_pa);
+ pr_info("However, doing so will make up to %llu bytes of RAM unusable.\n",
+ ((u64) max_pfn << PAGE_SHIFT) - half_pa);
+ pr_info("Reading Documentation/admin-guide/l1tf.rst might help you decide.");
return;
}
--
2.18.0
On Thu, Aug 23, 2018 at 03:44:18PM +0200, Vlastimil Babka wrote:
> Two users have reported [1] that they have an "extremely unlikely" system
> with more than MAX_PA/2 memory and L1TF mitigation is not effective. In fact
> it's a CPU with 36bits phys limit (64GB) and 32GB memory, but due to holes
> in the e820 map, the main region is almost 500MB over the 32GB limit:
Ah I see it's a client part with very large DIMMs and someone being
very brave and using that much memory without ECC.
>
> [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000081effffff] usable
>
> Suggestions to use 'mem=32G' to prefer L1TF mitigation while losing the 500MB
> revealed, that there's an off-by-one error in the check in
> l1tf_select_mitigation(). l1tf_pfn_limit() returns the last usable pfn
> (inclusive), but it's more common and hopefully less error-prone to return the
> first pfn that's over limit, so this patch changes that and updates the other
> callers.
I can see the off by one, but does it really cause the user's problem?
They will be still over the limit in any case, with or without off-by-one.
So the description has nothing to do with the fix. Or do I miss something?
-Andi
On Thu, Aug 23, 2018 at 04:28:12PM +0200, Vlastimil Babka wrote:
> Two users have reported [1] that they have an "extremely unlikely" system
> with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's
> make the warning more helpful by suggesting the proper mem=X kernel boot param,
> a rough calculation of how much RAM can be lost (not precise if there's holes
> between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document
> to help decide if the mitigation is worth the unusable RAM.
I'm not sure anyone would really do that. After all you probably prefer
your memory. And if it's really a non ECC client part they are are
already used to to live very dangerously because their undetected RAM bit error
rate will be significant. L1TF is probably one of your smaller problems
in this case...
-Andi
Hi Vlastimil,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on tip/auto-latest]
[also build test ERROR on next-20180822]
[cannot apply to v4.18]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Vlastimil-Babka/x86-speculation-l1tf-suggest-what-to-do-on-systems-with-too-much-RAM/20180824-013859
config: i386-randconfig-x077-201833 (attached as .config)
compiler: gcc-7 (Debian 7.3.0-16) 7.3.0
reproduce:
# save the attached .config to linux build tree
make ARCH=i386
All errors (new ones prefixed by >>):
In file included from include/linux/kernel.h:14:0,
from arch/x86/include/asm/percpu.h:45,
from arch/x86/include/asm/current.h:6,
from include/linux/sched.h:12,
from include/linux/utsname.h:6,
from arch/x86/kernel/cpu/bugs.c:12:
arch/x86/kernel/cpu/bugs.c: In function 'l1tf_select_mitigation':
>> arch/x86/kernel/cpu/bugs.c:709:12: error: 'max_pfn' undeclared (first use in this function); did you mean 'pgd_pfn'?
((u64) max_pfn << PAGE_SHIFT) - half_pa);
^
include/linux/printk.h:311:34: note: in definition of macro 'pr_info'
printk(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__)
^~~~~~~~~~~
arch/x86/kernel/cpu/bugs.c:709:12: note: each undeclared identifier is reported only once for each function it appears in
((u64) max_pfn << PAGE_SHIFT) - half_pa);
^
include/linux/printk.h:311:34: note: in definition of macro 'pr_info'
printk(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__)
^~~~~~~~~~~
vim +709 arch/x86/kernel/cpu/bugs.c
697
698 /*
699 * This is extremely unlikely to happen because almost all
700 * systems have far more MAX_PA/2 than RAM can be fit into
701 * DIMM slots.
702 */
703 half_pa = (u64)l1tf_pfn_limit() << PAGE_SHIFT;
704 if (e820__mapped_any(half_pa, ULLONG_MAX - half_pa, E820_TYPE_RAM)) {
705 pr_warn("System has more than MAX_PA/2 memory. L1TF mitigation not effective.\n");
706 pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n",
707 half_pa);
708 pr_info("However, doing so will make up to %llu bytes of RAM unusable.\n",
> 709 ((u64) max_pfn << PAGE_SHIFT) - half_pa);
710 pr_info("Reading Documentation/admin-guide/l1tf.rst might help you decide.");
711 return;
712 }
713
714 setup_force_cpu_cap(X86_FEATURE_L1TF_PTEINV);
715 }
716
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
On Thu 23-08-18 08:46:48, Andi Kleen wrote:
> On Thu, Aug 23, 2018 at 04:28:12PM +0200, Vlastimil Babka wrote:
> > Two users have reported [1] that they have an "extremely unlikely" system
> > with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's
> > make the warning more helpful by suggesting the proper mem=X kernel boot param,
> > a rough calculation of how much RAM can be lost (not precise if there's holes
> > between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document
> > to help decide if the mitigation is worth the unusable RAM.
>
> I'm not sure anyone would really do that. After all you probably prefer
> your memory. And if it's really a non ECC client part they are are
> already used to to live very dangerously because their undetected RAM bit error
> rate will be significant. L1TF is probably one of your smaller problems
> in this case...
There are people who care about L1TF mitigations. I am not going to
question their motivation. In any case a hint how to make the mitigation
active again sounds more useful than something that sounds as scary as
"you are vulnerable".
--
Michal Hocko
SUSE Labs
On Thu 23-08-18 16:28:12, Vlastimil Babka wrote:
> Two users have reported [1] that they have an "extremely unlikely" system
> with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's
> make the warning more helpful by suggesting the proper mem=X kernel boot param,
> a rough calculation of how much RAM can be lost (not precise if there's holes
> between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document
> to help decide if the mitigation is worth the unusable RAM.
>
> [1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
>
> Suggested-by: Michal Hocko <[email protected]>
> Cc: [email protected]
> Signed-off-by: Vlastimil Babka <[email protected]>
I wouldn't bother with max_pfn-half_pa part but other than that this is
much more useful than the original message.
Acked-by: Michal Hocko <[email protected]>
> ---
> arch/x86/kernel/cpu/bugs.c | 5 +++++
> 1 file changed, 5 insertions(+)
>
> diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
> index cb4a16292aa7..4b820bb6c504 100644
> --- a/arch/x86/kernel/cpu/bugs.c
> +++ b/arch/x86/kernel/cpu/bugs.c
> @@ -702,6 +702,11 @@ static void __init l1tf_select_mitigation(void)
> half_pa = (u64)l1tf_pfn_limit() << PAGE_SHIFT;
> if (e820__mapped_any(half_pa, ULLONG_MAX - half_pa, E820_TYPE_RAM)) {
> pr_warn("System has more than MAX_PA/2 memory. L1TF mitigation not effective.\n");
> + pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n",
> + half_pa);
> + pr_info("However, doing so will make up to %llu bytes of RAM unusable.\n",
> + ((u64) max_pfn << PAGE_SHIFT) - half_pa);
> + pr_info("Reading Documentation/admin-guide/l1tf.rst might help you decide.");
> return;
> }
>
> --
> 2.18.0
--
Michal Hocko
SUSE Labs
Hi Vlastimil,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on tip/auto-latest]
[also build test ERROR on next-20180822]
[cannot apply to v4.18]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Vlastimil-Babka/x86-speculation-l1tf-suggest-what-to-do-on-systems-with-too-much-RAM/20180824-013859
config: i386-randconfig-a1-201833 (attached as .config)
compiler: gcc-4.9 (Debian 4.9.4-2) 4.9.4
reproduce:
# save the attached .config to linux build tree
make ARCH=i386
All errors (new ones prefixed by >>):
In file included from include/linux/kernel.h:14:0,
from arch/x86/include/asm/percpu.h:45,
from arch/x86/include/asm/current.h:6,
from include/linux/sched.h:12,
from include/linux/utsname.h:6,
from arch/x86//kernel/cpu/bugs.c:12:
arch/x86//kernel/cpu/bugs.c: In function 'l1tf_select_mitigation':
>> arch/x86//kernel/cpu/bugs.c:709:12: error: 'max_pfn' undeclared (first use in this function)
((u64) max_pfn << PAGE_SHIFT) - half_pa);
^
include/linux/printk.h:311:34: note: in definition of macro 'pr_info'
printk(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__)
^
arch/x86//kernel/cpu/bugs.c:709:12: note: each undeclared identifier is reported only once for each function it appears in
((u64) max_pfn << PAGE_SHIFT) - half_pa);
^
include/linux/printk.h:311:34: note: in definition of macro 'pr_info'
printk(KERN_INFO pr_fmt(fmt), ##__VA_ARGS__)
^
vim +/max_pfn +709 arch/x86//kernel/cpu/bugs.c
697
698 /*
699 * This is extremely unlikely to happen because almost all
700 * systems have far more MAX_PA/2 than RAM can be fit into
701 * DIMM slots.
702 */
703 half_pa = (u64)l1tf_pfn_limit() << PAGE_SHIFT;
704 if (e820__mapped_any(half_pa, ULLONG_MAX - half_pa, E820_TYPE_RAM)) {
705 pr_warn("System has more than MAX_PA/2 memory. L1TF mitigation not effective.\n");
706 pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n",
707 half_pa);
708 pr_info("However, doing so will make up to %llu bytes of RAM unusable.\n",
> 709 ((u64) max_pfn << PAGE_SHIFT) - half_pa);
710 pr_info("Reading Documentation/admin-guide/l1tf.rst might help you decide.");
711 return;
712 }
713
714 setup_force_cpu_cap(X86_FEATURE_L1TF_PTEINV);
715 }
716
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
> There are people who care about L1TF mitigations. I am not going to
> question their motivation. In any case a hint how to make the mitigation
> active again sounds more useful than something that sounds as scary as
> "you are vulnerable".
FWIW an early version of these patches automatically limited the available
memory, but Linus pointed out that people likely prefer their memory.
I still think his reasoning was right, and likely applies to your
message too.
-Andi
On Thu 23-08-18 12:38:33, Andi Kleen wrote:
> > There are people who care about L1TF mitigations. I am not going to
> > question their motivation. In any case a hint how to make the mitigation
> > active again sounds more useful than something that sounds as scary as
> > "you are vulnerable".
>
> FWIW an early version of these patches automatically limited the available
> memory, but Linus pointed out that people likely prefer their memory.
Nobody is questioning that. The point is to give them a hint on how to
make the mitigation active again without going to call for help. The
message does tell them how to _enable_ it and point them to the
documentation on how to _decide_.
--
Michal Hocko
SUSE Labs
On 8/23/18 5:44 PM, Andi Kleen wrote:
> On Thu, Aug 23, 2018 at 03:44:18PM +0200, Vlastimil Babka wrote:
>> Two users have reported [1] that they have an "extremely unlikely" system
>> with more than MAX_PA/2 memory and L1TF mitigation is not effective. In fact
>> it's a CPU with 36bits phys limit (64GB) and 32GB memory, but due to holes
>> in the e820 map, the main region is almost 500MB over the 32GB limit:
>
> Ah I see it's a client part with very large DIMMs and someone being
> very brave and using that much memory without ECC.
Are you trying to mock the users and diminsh their report because of that?
>>
>> [ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000081effffff] usable
>>
>> Suggestions to use 'mem=32G' to prefer L1TF mitigation while losing the 500MB
>> revealed, that there's an off-by-one error in the check in
>> l1tf_select_mitigation(). l1tf_pfn_limit() returns the last usable pfn
>> (inclusive), but it's more common and hopefully less error-prone to return the
>> first pfn that's over limit, so this patch changes that and updates the other
>> callers.
>
> I can see the off by one, but does it really cause the user's problem?
>
> They will be still over the limit in any case, with or without off-by-one.
>
> So the description has nothing to do with the fix. Or do I miss something?
The off-by-one happens when 'mem=32G' is used to limit the amount
memory. It's easier to do "mem=32G" with fixed code than "mem=33554428k"
to workaround the error.
>
> -Andi
>
On Thu, Aug 23, 2018 at 10:05:20PM +0200, Michal Hocko wrote:
> On Thu 23-08-18 12:38:33, Andi Kleen wrote:
> > > There are people who care about L1TF mitigations. I am not going to
> > > question their motivation. In any case a hint how to make the mitigation
> > > active again sounds more useful than something that sounds as scary as
> > > "you are vulnerable".
> >
> > FWIW an early version of these patches automatically limited the available
> > memory, but Linus pointed out that people likely prefer their memory.
>
> Nobody is questioning that. The point is to give them a hint on how to
> make the mitigation active again without going to call for help. The
> message does tell them how to _enable_ it and point them to the
> documentation on how to _decide_.
On the message I guess there are two cases:
- either it's very little memory that is lost like in the 32GB + memory
hole case. In this case maybe it's better if we just limit automatically
if the overlap is small enough (<2GB perhaps?)
- Or it's a lot of memory then people are unlikely to want to lose their
memory and I don't think we really need the message either.
Also I checked the bug again and it looks like the reporter has an IvyBridge.
There is actually a better solution for those (anything Nehalem and newer)
because they internally have at least 44 bits in the cache, which
is good enough for the mitigation. Just need a quirk to override
the bit width in this case (will submit a patch)
-Andi
On 23. aug. 2018 17:44, Andi Kleen wrote:
> On Thu, Aug 23, 2018 at 03:44:18PM +0200, Vlastimil Babka wrote:
>> Two users have reported [1] that they have an "extremely unlikely" system
>> with more than MAX_PA/2 memory and L1TF mitigation is not effective. In fact
>> it's a CPU with 36bits phys limit (64GB) and 32GB memory, but due to holes
>> in the e820 map, the main region is almost 500MB over the 32GB limit:
>
> Ah I see it's a client part with very large DIMMs and someone being
> very brave and using that much memory without ECC.
FYI; It is also happening on Xeon E3v2 (Ivy Bridge generation) w/ 32GB
of ECC RAM here, a low-end server part that officially supports up to 32GB.
> [ 0.000000] microcode: microcode updated early to revision 0x20, date = 2018-04-10
> [ 0.029728] L1TF: System has more than MAX_PA/2 memory. L1TF mitigation not effective.
> [ 1.063155] microcode: sig=0x306a9, pf=0x2, revision=0x20
> processor : 7
> vendor_id : GenuineIntel
> cpu family : 6
> model : 58
> model name : Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
> stepping : 9
> microcode : 0x20
> cpu MHz : 3500.044
> cache size : 8192 KB
> physical id : 0
> siblings : 8
> core id : 3
> cpu cores : 4
> apicid : 7
> initial apicid : 7
> fpu : yes
> fpu_exception : yes
> cpuid level : 13
> wp : yes
> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts flush_l1d
> bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf
> bogomips : 6602.15
> clflush size : 64
> cache_alignment : 64
> address sizes : 36 bits physical, 48 bits virtual
> power management:
On Fri, Aug 24, 2018 at 04:22:57AM +0200, Andre Tomt wrote:
> On 23. aug. 2018 17:44, Andi Kleen wrote:
> > On Thu, Aug 23, 2018 at 03:44:18PM +0200, Vlastimil Babka wrote:
> > > Two users have reported [1] that they have an "extremely unlikely" system
> > > with more than MAX_PA/2 memory and L1TF mitigation is not effective. In fact
> > > it's a CPU with 36bits phys limit (64GB) and 32GB memory, but due to holes
> > > in the e820 map, the main region is almost 500MB over the 32GB limit:
> >
> > Ah I see it's a client part with very large DIMMs and someone being
> > very brave and using that much memory without ECC.
>
> FYI; It is also happening on Xeon E3v2 (Ivy Bridge generation) w/ 32GB of
> ECC RAM here, a low-end server part that officially supports up to 32GB.
Good point.
-andi
On 08/23/2018 09:27 PM, Michal Hocko wrote:
> On Thu 23-08-18 16:28:12, Vlastimil Babka wrote:
>> Two users have reported [1] that they have an "extremely unlikely" system
>> with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's
>> make the warning more helpful by suggesting the proper mem=X kernel boot param,
>> a rough calculation of how much RAM can be lost (not precise if there's holes
>> between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document
>> to help decide if the mitigation is worth the unusable RAM.
>>
>> [1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
>>
>> Suggested-by: Michal Hocko <[email protected]>
>> Cc: [email protected]
>> Signed-off-by: Vlastimil Babka <[email protected]>
>
> I wouldn't bother with max_pfn-half_pa part but other than that this is
> much more useful than the original message.
Right, and it causes build failures on some configs.
> Acked-by: Michal Hocko <[email protected]>
Thanks! Here's a v2:
----8<----
From 977c5db27fe35a84807850b947bc5678c4d467b3 Mon Sep 17 00:00:00 2001
From: Vlastimil Babka <[email protected]>
Date: Thu, 23 Aug 2018 16:21:29 +0200
Subject: [PATCH] x86/speculation/l1tf: suggest what to do on systems with too
much RAM
Two users have reported [1] that they have an "extremely unlikely" system
with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's
make the warning more helpful by suggesting the proper mem=X kernel boot param
to make it effective and a link to the L1TF document to help decide if the
mitigation is worth the unusable RAM.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
Suggested-by: Michal Hocko <[email protected]>
Acked-by: Michal Hocko <[email protected]>
Signed-off-by: Vlastimil Babka <[email protected]>
---
arch/x86/kernel/cpu/bugs.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index cb4a16292aa7..5c32b5006738 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -702,6 +702,10 @@ static void __init l1tf_select_mitigation(void)
half_pa = (u64)l1tf_pfn_limit() << PAGE_SHIFT;
if (e820__mapped_any(half_pa, ULLONG_MAX - half_pa, E820_TYPE_RAM)) {
pr_warn("System has more than MAX_PA/2 memory. L1TF mitigation not effective.\n");
+ pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n",
+ half_pa);
+ pr_info("However, doing so will make a part of your RAM unusable.\n");
+ pr_info("Reading Documentation/admin-guide/l1tf.rst might help you decide.\n");
return;
}
--
2.18.0
Commit-ID: b0a182f875689647b014bc01d36b340217792852
Gitweb: https://git.kernel.org/tip/b0a182f875689647b014bc01d36b340217792852
Author: Vlastimil Babka <[email protected]>
AuthorDate: Thu, 23 Aug 2018 15:44:18 +0200
Committer: Thomas Gleixner <[email protected]>
CommitDate: Fri, 24 Aug 2018 09:51:14 +0200
x86/speculation/l1tf: Fix off-by-one error when warning that system has too much RAM
Two users have reported [1] that they have an "extremely unlikely" system
with more than MAX_PA/2 memory and L1TF mitigation is not effective. In
fact it's a CPU with 36bits phys limit (64GB) and 32GB memory, but due to
holes in the e820 map, the main region is almost 500MB over the 32GB limit:
[ 0.000000] BIOS-e820: [mem 0x0000000100000000-0x000000081effffff] usable
Suggestions to use 'mem=32G' to enable the L1TF mitigation while losing the
500MB revealed, that there's an off-by-one error in the check in
l1tf_select_mitigation().
l1tf_pfn_limit() returns the last usable pfn (inclusive) and the range
check in the mitigation path does not take this into account.
Instead of amending the range check, make l1tf_pfn_limit() return the first
PFN which is over the limit which is less error prone. Adjust the other
users accordingly.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
Fixes: 17dbca119312 ("x86/speculation/l1tf: Add sysfs reporting for l1tf")
Reported-by: George Anchev <[email protected]>
Reported-by: Christopher Snowhill <[email protected]>
Signed-off-by: Vlastimil Babka <[email protected]>
Signed-off-by: Thomas Gleixner <[email protected]>
Cc: "H . Peter Anvin" <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Andi Kleen <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: Michal Hocko <[email protected]>
Cc: [email protected]
Link: https://lkml.kernel.org/r/[email protected]
---
arch/x86/include/asm/processor.h | 2 +-
arch/x86/mm/init.c | 2 +-
arch/x86/mm/mmap.c | 2 +-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/arch/x86/include/asm/processor.h b/arch/x86/include/asm/processor.h
index a0a52274cb4a..c24297268ebc 100644
--- a/arch/x86/include/asm/processor.h
+++ b/arch/x86/include/asm/processor.h
@@ -183,7 +183,7 @@ extern void cpu_detect(struct cpuinfo_x86 *c);
static inline unsigned long long l1tf_pfn_limit(void)
{
- return BIT_ULL(boot_cpu_data.x86_phys_bits - 1 - PAGE_SHIFT) - 1;
+ return BIT_ULL(boot_cpu_data.x86_phys_bits - 1 - PAGE_SHIFT);
}
extern void early_cpu_init(void);
diff --git a/arch/x86/mm/init.c b/arch/x86/mm/init.c
index 02de3d6065c4..63a6f9fcaf20 100644
--- a/arch/x86/mm/init.c
+++ b/arch/x86/mm/init.c
@@ -923,7 +923,7 @@ unsigned long max_swapfile_size(void)
if (boot_cpu_has_bug(X86_BUG_L1TF)) {
/* Limit the swap file size to MAX_PA/2 for L1TF workaround */
- unsigned long long l1tf_limit = l1tf_pfn_limit() + 1;
+ unsigned long long l1tf_limit = l1tf_pfn_limit();
/*
* We encode swap offsets also with 3 bits below those for pfn
* which makes the usable limit higher.
diff --git a/arch/x86/mm/mmap.c b/arch/x86/mm/mmap.c
index f40ab8185d94..1e95d57760cf 100644
--- a/arch/x86/mm/mmap.c
+++ b/arch/x86/mm/mmap.c
@@ -257,7 +257,7 @@ bool pfn_modify_allowed(unsigned long pfn, pgprot_t prot)
/* If it's real memory always allow */
if (pfn_valid(pfn))
return true;
- if (pfn > l1tf_pfn_limit() && !capable(CAP_SYS_ADMIN))
+ if (pfn >= l1tf_pfn_limit() && !capable(CAP_SYS_ADMIN))
return false;
return true;
}
Commit-ID: 4b81eae3d37dee69231592182e1e34706f149a6e
Gitweb: https://git.kernel.org/tip/4b81eae3d37dee69231592182e1e34706f149a6e
Author: Vlastimil Babka <[email protected]>
AuthorDate: Thu, 23 Aug 2018 16:21:29 +0200
Committer: Thomas Gleixner <[email protected]>
CommitDate: Fri, 24 Aug 2018 09:51:14 +0200
x86/speculation/l1tf: Suggest what to do on systems with too much RAM
Two users have reported [1] that they have an "extremely unlikely" system
with more than MAX_PA/2 memory and L1TF mitigation is not effective.
Make the warning more helpful by suggesting the proper mem=X kernel boot
parameter to make it effective and a link to the L1TF document to help
decide if the mitigation is worth the unusable RAM.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
Suggested-by: Michal Hocko <[email protected]>
Signed-off-by: Vlastimil Babka <[email protected]>
Acked-by: Michal Hocko <[email protected]>
Cc: "H . Peter Anvin" <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Andi Kleen <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: [email protected]
Link: https://lkml.kernel.org/r/[email protected]
---
arch/x86/kernel/cpu/bugs.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index cb4a16292aa7..5c32b5006738 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -702,6 +702,10 @@ static void __init l1tf_select_mitigation(void)
half_pa = (u64)l1tf_pfn_limit() << PAGE_SHIFT;
if (e820__mapped_any(half_pa, ULLONG_MAX - half_pa, E820_TYPE_RAM)) {
pr_warn("System has more than MAX_PA/2 memory. L1TF mitigation not effective.\n");
+ pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n",
+ half_pa);
+ pr_info("However, doing so will make a part of your RAM unusable.\n");
+ pr_info("Reading Documentation/admin-guide/l1tf.rst might help you decide.\n");
return;
}
On 08/24/2018 09:32 AM, Vlastimil Babka wrote:
> On 08/23/2018 09:27 PM, Michal Hocko wrote:
>> On Thu 23-08-18 16:28:12, Vlastimil Babka wrote:
>>> Two users have reported [1] that they have an "extremely unlikely" system
>>> with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's
>>> make the warning more helpful by suggesting the proper mem=X kernel boot param,
>>> a rough calculation of how much RAM can be lost (not precise if there's holes
>>> between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document
>>> to help decide if the mitigation is worth the unusable RAM.
>>>
>>> [1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
>>>
>>> Suggested-by: Michal Hocko <[email protected]>
>>> Cc: [email protected]
>>> Signed-off-by: Vlastimil Babka <[email protected]>
>>
>> I wouldn't bother with max_pfn-half_pa part but other than that this is
>> much more useful than the original message.
>
> Right, and it causes build failures on some configs.
>
>> Acked-by: Michal Hocko <[email protected]>
>
> Thanks! Here's a v2:
Just realized that kvm printk's refer to the online version at
https://www.kernel.org/doc/html/latest/admin-guide/l1tf.html
which should be easier for the users of distro kernels, should I change
that?
On 08/24/2018 12:36 PM, Vlastimil Babka wrote:
> On 08/24/2018 09:32 AM, Vlastimil Babka wrote:
>> On 08/23/2018 09:27 PM, Michal Hocko wrote:
>>> On Thu 23-08-18 16:28:12, Vlastimil Babka wrote:
>>>> Two users have reported [1] that they have an "extremely unlikely" system
>>>> with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's
>>>> make the warning more helpful by suggesting the proper mem=X kernel boot param,
>>>> a rough calculation of how much RAM can be lost (not precise if there's holes
>>>> between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document
>>>> to help decide if the mitigation is worth the unusable RAM.
>>>>
>>>> [1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
>>>>
>>>> Suggested-by: Michal Hocko <[email protected]>
>>>> Cc: [email protected]
>>>> Signed-off-by: Vlastimil Babka <[email protected]>
>>>
>>> I wouldn't bother with max_pfn-half_pa part but other than that this is
>>> much more useful than the original message.
>>
>> Right, and it causes build failures on some configs.
>>
>>> Acked-by: Michal Hocko <[email protected]>
>>
>> Thanks! Here's a v2:
>
> Just realized that kvm printk's refer to the online version at
> https://www.kernel.org/doc/html/latest/admin-guide/l1tf.html
> which should be easier for the users of distro kernels, should I change
> that?
FWIW, if it's not possible to amend anymore...
----8<----
From a650e6a4b989c6e029ac1ab4e0a68553e074ba7a Mon Sep 17 00:00:00 2001
From: Vlastimil Babka <[email protected]>
Date: Fri, 24 Aug 2018 14:05:45 +0200
Subject: [PATCH] x86/speculation/l1tf: replace Documentation path with
kernel.org URL
The URL might be easier to reach for users of distro kernels without unpacked
source, and be more up-to-date. It's also used in KVM warnings related to L1TF.
Signed-off-by: Vlastimil Babka <[email protected]>
---
arch/x86/kernel/cpu/bugs.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 5c32b5006738..4c2313d0b9ca 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -705,7 +705,7 @@ static void __init l1tf_select_mitigation(void)
pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n",
half_pa);
pr_info("However, doing so will make a part of your RAM unusable.\n");
- pr_info("Reading Documentation/admin-guide/l1tf.rst might help you decide.\n");
+ pr_info("Reading https://www.kernel.org/doc/html/latest/admin-guide/l1tf.html might help you decide.\n");
return;
}
--
2.18.0
On Fri 24-08-18 14:10:54, Vlastimil Babka wrote:
> On 08/24/2018 12:36 PM, Vlastimil Babka wrote:
> > On 08/24/2018 09:32 AM, Vlastimil Babka wrote:
> >> On 08/23/2018 09:27 PM, Michal Hocko wrote:
> >>> On Thu 23-08-18 16:28:12, Vlastimil Babka wrote:
> >>>> Two users have reported [1] that they have an "extremely unlikely" system
> >>>> with more than MAX_PA/2 memory and L1TF mitigation is not effective. Let's
> >>>> make the warning more helpful by suggesting the proper mem=X kernel boot param,
> >>>> a rough calculation of how much RAM can be lost (not precise if there's holes
> >>>> between MAX_PA/2 and max_pfn in the e820 map) and a link to the L1TF document
> >>>> to help decide if the mitigation is worth the unusable RAM.
> >>>>
> >>>> [1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
> >>>>
> >>>> Suggested-by: Michal Hocko <[email protected]>
> >>>> Cc: [email protected]
> >>>> Signed-off-by: Vlastimil Babka <[email protected]>
> >>>
> >>> I wouldn't bother with max_pfn-half_pa part but other than that this is
> >>> much more useful than the original message.
> >>
> >> Right, and it causes build failures on some configs.
> >>
> >>> Acked-by: Michal Hocko <[email protected]>
> >>
> >> Thanks! Here's a v2:
> >
> > Just realized that kvm printk's refer to the online version at
> > https://www.kernel.org/doc/html/latest/admin-guide/l1tf.html
> > which should be easier for the users of distro kernels, should I change
> > that?
>
> FWIW, if it's not possible to amend anymore...
> ----8<----
> >From a650e6a4b989c6e029ac1ab4e0a68553e074ba7a Mon Sep 17 00:00:00 2001
> From: Vlastimil Babka <[email protected]>
> Date: Fri, 24 Aug 2018 14:05:45 +0200
> Subject: [PATCH] x86/speculation/l1tf: replace Documentation path with
> kernel.org URL
>
> The URL might be easier to reach for users of distro kernels without unpacked
> source, and be more up-to-date. It's also used in KVM warnings related to L1TF.
Yes, URL is much more convenient than a reference to the kernel source
documentation. If the link below is meant to be long lived then sure.
> Signed-off-by: Vlastimil Babka <[email protected]>
Acked-by: Michal Hocko <[email protected]>
> ---
> arch/x86/kernel/cpu/bugs.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
> index 5c32b5006738..4c2313d0b9ca 100644
> --- a/arch/x86/kernel/cpu/bugs.c
> +++ b/arch/x86/kernel/cpu/bugs.c
> @@ -705,7 +705,7 @@ static void __init l1tf_select_mitigation(void)
> pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n",
> half_pa);
> pr_info("However, doing so will make a part of your RAM unusable.\n");
> - pr_info("Reading Documentation/admin-guide/l1tf.rst might help you decide.\n");
> + pr_info("Reading https://www.kernel.org/doc/html/latest/admin-guide/l1tf.html might help you decide.\n");
> return;
> }
>
> --
> 2.18.0
>
>
>
--
Michal Hocko
SUSE Labs
Commit-ID: 6a012288d6906fee1dbc244050ade1dafe4a9c8d
Gitweb: https://git.kernel.org/tip/6a012288d6906fee1dbc244050ade1dafe4a9c8d
Author: Vlastimil Babka <[email protected]>
AuthorDate: Thu, 23 Aug 2018 16:21:29 +0200
Committer: Thomas Gleixner <[email protected]>
CommitDate: Fri, 24 Aug 2018 15:55:17 +0200
x86/speculation/l1tf: Suggest what to do on systems with too much RAM
Two users have reported [1] that they have an "extremely unlikely" system
with more than MAX_PA/2 memory and L1TF mitigation is not effective.
Make the warning more helpful by suggesting the proper mem=X kernel boot
parameter to make it effective and a link to the L1TF document to help
decide if the mitigation is worth the unusable RAM.
[1] https://bugzilla.suse.com/show_bug.cgi?id=1105536
Suggested-by: Michal Hocko <[email protected]>
Signed-off-by: Vlastimil Babka <[email protected]>
Acked-by: Michal Hocko <[email protected]>
Cc: "H . Peter Anvin" <[email protected]>
Cc: Linus Torvalds <[email protected]>
Cc: Andi Kleen <[email protected]>
Cc: Dave Hansen <[email protected]>
Cc: [email protected]
Link: https://lkml.kernel.org/r/[email protected]
---
arch/x86/kernel/cpu/bugs.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index cb4a16292aa7..4c2313d0b9ca 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -702,6 +702,10 @@ static void __init l1tf_select_mitigation(void)
half_pa = (u64)l1tf_pfn_limit() << PAGE_SHIFT;
if (e820__mapped_any(half_pa, ULLONG_MAX - half_pa, E820_TYPE_RAM)) {
pr_warn("System has more than MAX_PA/2 memory. L1TF mitigation not effective.\n");
+ pr_info("You may make it effective by booting the kernel with mem=%llu parameter.\n",
+ half_pa);
+ pr_info("However, doing so will make a part of your RAM unusable.\n");
+ pr_info("Reading https://www.kernel.org/doc/html/latest/admin-guide/l1tf.html might help you decide.\n");
return;
}
On 08/23/2018 07:22 PM, Andre Tomt wrote:
> On 23. aug. 2018 17:44, Andi Kleen wrote:
>> On Thu, Aug 23, 2018 at 03:44:18PM +0200, Vlastimil Babka wrote:
>>> Two users have reported [1] that they have an "extremely unlikely"
>>> system
>>> with more than MAX_PA/2 memory and L1TF mitigation is not effective.
>>> In fact
>>> it's a CPU with 36bits phys limit (64GB) and 32GB memory, but due to
>>> holes
>>> in the e820 map, the main region is almost 500MB over the 32GB limit:
>>
>> Ah I see it's a client part with very large DIMMs and someone being
>> very brave and using that much memory without ECC.
>
> FYI; It is also happening on Xeon E3v2 (Ivy Bridge generation) w/ 32GB
> of ECC RAM here, a low-end server part that officially supports up to
> 32GB.
>
Indeed, I must be "very brave" to not have chucked this CPU and
motherboard and RAM in the bin, and bought a new board, Xeon CPU, and
ECC RAM. Maybe I'll consider that in the future, when I again have
$1000+ to buy new kit. Which will probably be never, at this rate.
>> [ 0.000000] microcode: microcode updated early to revision 0x20,
>> date = 2018-04-10
>> [ 0.029728] L1TF: System has more than MAX_PA/2 memory. L1TF
>> mitigation not effective.
>> [ 1.063155] microcode: sig=0x306a9, pf=0x2, revision=0x20
>
>
>> processor : 7
>> vendor_id : GenuineIntel
>> cpu family : 6
>> model : 58
>> model name : Intel(R) Xeon(R) CPU E3-1230 V2 @ 3.30GHz
>> stepping : 9
>> microcode : 0x20
>> cpu MHz : 3500.044
>> cache size : 8192 KB
>> physical id : 0
>> siblings : 8
>> core id : 3
>> cpu cores : 4
>> apicid : 7
>> initial apicid : 7
>> fpu : yes
>> fpu_exception : yes
>> cpuid level : 13
>> wp : yes
>> flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge
>> mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe
>> syscall nx rdtscp lm constant_tsc arch_perfmon pebs bts rep_good nopl
>> xtopology nonstop_tsc cpuid aperfmperf pni pclmulqdq dtes64 monitor
>> ds_cpl vmx smx est tm2 ssse3 cx16 xtpr pdcm pcid sse4_1 sse4_2 x2apic
>> popcnt tsc_deadline_timer aes xsave avx f16c rdrand lahf_lm
>> cpuid_fault epb pti ssbd ibrs ibpb stibp tpr_shadow vnmi flexpriority
>> ept vpid fsgsbase smep erms xsaveopt dtherm ida arat pln pts flush_l1d
>> bugs : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf
>> bogomips : 6602.15
>> clflush size : 64
>> cache_alignment : 64
>> address sizes : 36 bits physical, 48 bits virtual
>> power management:
>
>