2007-06-06 12:12:42

by Bodo Eggert

[permalink] [raw]
Subject: [RFC][PATCH] Re: 4Gb ram not showing up

Change the description of CONFIG_*HIGHMEM* to reflect "lost" memory due to
PCI space and the existence of the NX flag.

Signed-Off-By: Bodo Eggert <[email protected]>
---
I made this quick patch using the information from LKML as I remembered
it. Please verify.

--- 2.6.21/arch/i386/Kconfig.ori 2007-06-06 13:41:09.000000000 +0200
+++ 2.6.21/arch/i386/Kconfig 2007-06-06 14:07:40.000000000 +0200
@@ -495,8 +495,8 @@ config NOHIGHMEM
bool "off"
depends on !X86_NUMAQ
---help---
- Linux can use up to 64 Gigabytes of physical memory on x86 systems.
- However, the address space of 32-bit x86 processors is only 4
+ Linux can use up to 64 Gigabytes of physical address space on x86
+ systems. However, the address space of 32-bit x86 processors is only 4
Gigabytes large. That means that, if you have a large amount of
physical memory, not all of it can be "permanently mapped" by the
kernel. The physical memory that's not permanently mapped is called
@@ -510,8 +510,15 @@ config NOHIGHMEM
by the kernel to permanently map as much physical memory as
possible.

- If the machine has between 1 and 4 Gigabytes physical RAM, then
+
+ If the machine has between 1 and 3.5 Gigabytes physical RAM, then
answer "4GB" here.
+
+ The PCI address space will usurally take 512 MB or 1 GB of address
+ space. This address space is unavailable to RAM, but depending on the
+ chipset (and BIOS settings), memory overlapping the PCI address space
+ may be mapped beyond the 4 GB limit and be available using "64GB".
+

If more than 4 Gigabytes is used then answer "64GB" here. This
selection turns Intel PAE (Physical Address Extension) mode on.
@@ -520,6 +527,10 @@ config NOHIGHMEM
processors (Pentium Pro and better). NOTE: If you say "64GB" here,
then the kernel will not boot on CPUs that don't support PAE!

+ An additional benefit of the 64GB-Mode is the availability of the
+ no-execute-pageflag, which can be used to prevent some attacks from
+ injecting malicious code into applications.
+
The actual amount of total physical memory will either be
auto detected or can be forced by using a kernel command line option
such as "mem=256M". (Try "man bootparam" or see the documentation of
@@ -532,14 +543,14 @@ config HIGHMEM4G
bool "4GB"
depends on !X86_NUMAQ
help
- Select this if you have a 32-bit processor and between 1 and 4
+ Select this if you have a 32-bit processor and between 1 and 3.5
gigabytes of physical RAM.

config HIGHMEM64G
- bool "64GB"
+ bool "64GB (enables no-execute memory protection if available)"
depends on X86_CMPXCHG64
help
- Select this if you have a 32-bit processor and more than 4
+ Select this if you have a 32-bit processor and more than 3.5
gigabytes of physical RAM.

endchoice
--
RAM DISK is not an installation procedure!


2007-06-06 14:37:32

by Lennart Sorensen

[permalink] [raw]
Subject: Re: [RFC][PATCH] Re: 4Gb ram not showing up

On Wed, Jun 06, 2007 at 02:12:22PM +0200, Bodo Eggert wrote:
> Change the description of CONFIG_*HIGHMEM* to reflect "lost" memory due to
> PCI space and the existence of the NX flag.
>
> Signed-Off-By: Bodo Eggert <[email protected]>
> ---
> I made this quick patch using the information from LKML as I remembered
> it. Please verify.
>
> --- 2.6.21/arch/i386/Kconfig.ori 2007-06-06 13:41:09.000000000 +0200
> +++ 2.6.21/arch/i386/Kconfig 2007-06-06 14:07:40.000000000 +0200
> @@ -495,8 +495,8 @@ config NOHIGHMEM
> bool "off"
> depends on !X86_NUMAQ
> ---help---
> - Linux can use up to 64 Gigabytes of physical memory on x86 systems.
> - However, the address space of 32-bit x86 processors is only 4
> + Linux can use up to 64 Gigabytes of physical address space on x86
> + systems. However, the address space of 32-bit x86 processors is only 4
> Gigabytes large. That means that, if you have a large amount of
> physical memory, not all of it can be "permanently mapped" by the
> kernel. The physical memory that's not permanently mapped is called
> @@ -510,8 +510,15 @@ config NOHIGHMEM
> by the kernel to permanently map as much physical memory as
> possible.
>
> - If the machine has between 1 and 4 Gigabytes physical RAM, then
> +
> + If the machine has between 1 and 3.5 Gigabytes physical RAM, then
> answer "4GB" here.
> +
> + The PCI address space will usurally take 512 MB or 1 GB of address
usually

> + space. This address space is unavailable to RAM, but depending on the
> + chipset (and BIOS settings), memory overlapping the PCI address space
> + may be mapped beyond the 4 GB limit and be available using "64GB".
> +
>
> If more than 4 Gigabytes is used then answer "64GB" here. This
> selection turns Intel PAE (Physical Address Extension) mode on.
> @@ -520,6 +527,10 @@ config NOHIGHMEM
> processors (Pentium Pro and better). NOTE: If you say "64GB" here,
> then the kernel will not boot on CPUs that don't support PAE!
>
> + An additional benefit of the 64GB-Mode is the availability of the
> + no-execute-pageflag, which can be used to prevent some attacks from
> + injecting malicious code into applications.
> +
> The actual amount of total physical memory will either be
> auto detected or can be forced by using a kernel command line option
> such as "mem=256M". (Try "man bootparam" or see the documentation of
> @@ -532,14 +543,14 @@ config HIGHMEM4G
> bool "4GB"
> depends on !X86_NUMAQ
> help
> - Select this if you have a 32-bit processor and between 1 and 4
> + Select this if you have a 32-bit processor and between 1 and 3.5
> gigabytes of physical RAM.
>
> config HIGHMEM64G
> - bool "64GB"
> + bool "64GB (enables no-execute memory protection if available)"
> depends on X86_CMPXCHG64
> help
> - Select this if you have a 32-bit processor and more than 4
> + Select this if you have a 32-bit processor and more than 3.5
> gigabytes of physical RAM.
>
> endchoice

Seems like an improvement to me. To fully explain how it could be 3 or
3.5 or 3.25 or who knows how many GB you can actually use without PAE
would probably require writing a small novel. Certainly talking about
address space instead of amounts of physical memory is more correct.

--
Len Sorensen

2007-06-06 21:56:21

by Wakko Warner

[permalink] [raw]
Subject: Re: [RFC][PATCH] Re: 4Gb ram not showing up

Lennart Sorensen wrote:
> On Wed, Jun 06, 2007 at 02:12:22PM +0200, Bodo Eggert wrote:
> > Change the description of CONFIG_*HIGHMEM* to reflect "lost" memory due to
> > PCI space and the existence of the NX flag.
> >
> > Signed-Off-By: Bodo Eggert <[email protected]>
> > ---
> > I made this quick patch using the information from LKML as I remembered
> > it. Please verify.
> >
> > @@ -510,8 +510,15 @@ config NOHIGHMEM
> > by the kernel to permanently map as much physical memory as
> > possible.
> >
> > - If the machine has between 1 and 4 Gigabytes physical RAM, then
> > +
> > + If the machine has between 1 and 3.5 Gigabytes physical RAM, then
> > answer "4GB" here.
> > +

Might be better to say 3 Gigabytes instead of 3.5. I booted a 4gb kernel on
a machine with 4gb at work and only saw 3gb of memory.

Also, I've noticed that there's ~128mb of memory lost if you select 1gb when
you have 1gb of memory installed.

> > If more than 4 Gigabytes is used then answer "64GB" here. This
> > selection turns Intel PAE (Physical Address Extension) mode on.
> > @@ -520,6 +527,10 @@ config NOHIGHMEM
> > processors (Pentium Pro and better). NOTE: If you say "64GB" here,
> > then the kernel will not boot on CPUs that don't support PAE!
> >
> > + An additional benefit of the 64GB-Mode is the availability of the
> > + no-execute-pageflag, which can be used to prevent some attacks from
> > + injecting malicious code into applications.
> > +
> > The actual amount of total physical memory will either be
> > auto detected or can be forced by using a kernel command line option
> > such as "mem=256M". (Try "man bootparam" or see the documentation of
> > @@ -532,14 +543,14 @@ config HIGHMEM4G
> > bool "4GB"
> > depends on !X86_NUMAQ
> > help
> > - Select this if you have a 32-bit processor and between 1 and 4
> > + Select this if you have a 32-bit processor and between 1 and 3.5
> > gigabytes of physical RAM.
> >
> > config HIGHMEM64G
> > - bool "64GB"
> > + bool "64GB (enables no-execute memory protection if available)"
> > depends on X86_CMPXCHG64
> > help
> > - Select this if you have a 32-bit processor and more than 4
> > + Select this if you have a 32-bit processor and more than 3.5
> > gigabytes of physical RAM.

Same here

> Seems like an improvement to me. To fully explain how it could be 3 or
> 3.5 or 3.25 or who knows how many GB you can actually use without PAE
> would probably require writing a small novel. Certainly talking about
> address space instead of amounts of physical memory is more correct.

It's probably worth it to just say 3gb.

I've looked at the cpuinfo for several machines, all of which (pIII and
better, not all physically support even 2gb of memory) show pae in the
flags, so is there really a reason not to just specify 64gb always?

The oldest machine I have access to is a pII 333mhz machine, cpuinfo also
lists pae in the cpu flags.

--
Lab tests show that use of micro$oft causes cancer in lab animals
Got Gas???

2007-06-06 22:12:29

by H. Peter Anvin

[permalink] [raw]
Subject: Re: [RFC][PATCH] Re: 4Gb ram not showing up

Lennart Sorensen wrote:
>
> Seems like an improvement to me. To fully explain how it could be 3 or
> 3.5 or 3.25 or who knows how many GB you can actually use without PAE
> would probably require writing a small novel. Certainly talking about
> address space instead of amounts of physical memory is more correct.
>

On some machines it's even just 2 GB.

-hpa

2007-06-06 22:42:12

by Andrew Lyon

[permalink] [raw]
Subject: Re: [RFC][PATCH] Re: 4Gb ram not showing up

On 6/6/07, H. Peter Anvin <[email protected]> wrote:
> Lennart Sorensen wrote:
> >
> > Seems like an improvement to me. To fully explain how it could be 3 or
> > 3.5 or 3.25 or who knows how many GB you can actually use without PAE
> > would probably require writing a small novel. Certainly talking about
> > address space instead of amounts of physical memory is more correct.
> >
>
> On some machines it's even just 2 GB.
>
> -hpa
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>

Could this also cause a system to be unstable? my abit athlon64 at
work will not run x64 with more than 1gb ram, and i have a colo server
with supermicro & 2 x dual core xeons that will not run with more than
2gb.

Both systems have long uptimes but if i add ram they crash within
minutes of booting.

Tried several kernels up to 2.6.21 and gave up, I can send dmesg
output but the crashes are completely random.

andy

2007-06-07 00:09:58

by Robert Hancock

[permalink] [raw]
Subject: Re: [RFC][PATCH] Re: 4Gb ram not showing up

Andrew Lyon wrote:
> Could this also cause a system to be unstable? my abit athlon64 at
> work will not run x64 with more than 1gb ram, and i have a colo server
> with supermicro & 2 x dual core xeons that will not run with more than
> 2gb.
>
> Both systems have long uptimes but if i add ram they crash within
> minutes of booting.
>
> Tried several kernels up to 2.6.21 and gave up, I can send dmesg
> output but the crashes are completely random.

That is likely some other problem, like bad RAM or memory timing issues.

--
Robert Hancock Saskatoon, SK, Canada
To email, remove "nospam" from [email protected]
Home Page: http://www.roberthancock.com/

2007-06-07 13:49:18

by Tom Moore

[permalink] [raw]
Subject: Re: [RFC][PATCH] Re: 4Gb ram not showing up

This looks like it would have probably given me the hint I needed. Can
I also suggest that you make a change to arch/i386/kernel/setup.c, line 296:

if (max_pfn > MAX_NONPAE_PFN) {
max_pfn = MAX_NONPAE_PFN;
printk(KERN_WARNING "Warning only 4GB will be
used.\n");
- printk(KERN_WARNING "Use a PAE enabled kernel.\n");
+ printk(KERN_WARNING "Use a PAE enabled kernel
by selecting the HIGHMEM64G configuration option.\n");
}

It does not seem that PAE is a configuration option that is exposed
through make menuconfig. How else are people supposed to know that PAE
is a side effect of CONFIG_HIGHMEM64G unless they know the kbuild logic?

Tom

Bodo Eggert wrote:
> Change the description of CONFIG_*HIGHMEM* to reflect "lost" memory due to
> PCI space and the existence of the NX flag.
>
> Signed-Off-By: Bodo Eggert <[email protected]>
> ---
> I made this quick patch using the information from LKML as I remembered
> it. Please verify.
>
> --- 2.6.21/arch/i386/Kconfig.ori 2007-06-06 13:41:09.000000000 +0200
> +++ 2.6.21/arch/i386/Kconfig 2007-06-06 14:07:40.000000000 +0200
> @@ -495,8 +495,8 @@ config NOHIGHMEM
> bool "off"
> depends on !X86_NUMAQ
> ---help---
> - Linux can use up to 64 Gigabytes of physical memory on x86 systems.
> - However, the address space of 32-bit x86 processors is only 4
> + Linux can use up to 64 Gigabytes of physical address space on x86
> + systems. However, the address space of 32-bit x86 processors is only 4
> Gigabytes large. That means that, if you have a large amount of
> physical memory, not all of it can be "permanently mapped" by the
> kernel. The physical memory that's not permanently mapped is called
> @@ -510,8 +510,15 @@ config NOHIGHMEM
> by the kernel to permanently map as much physical memory as
> possible.
>
> - If the machine has between 1 and 4 Gigabytes physical RAM, then
> +
> + If the machine has between 1 and 3.5 Gigabytes physical RAM, then
> answer "4GB" here.
> +
> + The PCI address space will usurally take 512 MB or 1 GB of address
> + space. This address space is unavailable to RAM, but depending on the
> + chipset (and BIOS settings), memory overlapping the PCI address space
> + may be mapped beyond the 4 GB limit and be available using "64GB".
> +
>
> If more than 4 Gigabytes is used then answer "64GB" here. This
> selection turns Intel PAE (Physical Address Extension) mode on.
> @@ -520,6 +527,10 @@ config NOHIGHMEM
> processors (Pentium Pro and better). NOTE: If you say "64GB" here,
> then the kernel will not boot on CPUs that don't support PAE!
>
> + An additional benefit of the 64GB-Mode is the availability of the
> + no-execute-pageflag, which can be used to prevent some attacks from
> + injecting malicious code into applications.
> +
> The actual amount of total physical memory will either be
> auto detected or can be forced by using a kernel command line option
> such as "mem=256M". (Try "man bootparam" or see the documentation of
> @@ -532,14 +543,14 @@ config HIGHMEM4G
> bool "4GB"
> depends on !X86_NUMAQ
> help
> - Select this if you have a 32-bit processor and between 1 and 4
> + Select this if you have a 32-bit processor and between 1 and 3.5
> gigabytes of physical RAM.
>
> config HIGHMEM64G
> - bool "64GB"
> + bool "64GB (enables no-execute memory protection if available)"
> depends on X86_CMPXCHG64
> help
> - Select this if you have a 32-bit processor and more than 4
> + Select this if you have a 32-bit processor and more than 3.5
> gigabytes of physical RAM.
>
> endchoice
>

2007-06-07 16:19:13

by H. Peter Anvin

[permalink] [raw]
Subject: Re: [RFC][PATCH] Re: 4Gb ram not showing up

Andrew Lyon wrote:
>
> Could this also cause a system to be unstable? my abit athlon64 at
> work will not run x64 with more than 1gb ram, and i have a colo server
> with supermicro & 2 x dual core xeons that will not run with more than
> 2gb.
>
> Both systems have long uptimes but if i add ram they crash within
> minutes of booting.
>

That's much more likely to be caused by bad memory bus layout so that it
doesn't handle the higher loading.

Run memtest86+ on the expanded memory for some time.

-hpa

2007-06-07 17:37:30

by Andrew Lyon

[permalink] [raw]
Subject: Re: [RFC][PATCH] Re: 4Gb ram not showing up

On 6/7/07, H. Peter Anvin <[email protected]> wrote:
> Andrew Lyon wrote:
> >
> > Could this also cause a system to be unstable? my abit athlon64 at
> > work will not run x64 with more than 1gb ram, and i have a colo server
> > with supermicro & 2 x dual core xeons that will not run with more than
> > 2gb.
> >
> > Both systems have long uptimes but if i add ram they crash within
> > minutes of booting.
> >
>
> That's much more likely to be caused by bad memory bus layout so that it
> doesn't handle the higher loading.
>
> Run memtest86+ on the expanded memory for some time.
>
> -hpa
>

I have run memtest86+ for days at a time on both systems with no errors at all.

broken hardware i guess, i am not surprised about the abit board, ive
had nothing but trouble with abit motherboards and do not use them any
more, but the supermicro is a xeon server board and i was using
kingston ram, i wouldnt expect that to have issues.

latest bios on both.

Andy

2007-06-09 00:47:41

by H. Peter Anvin

[permalink] [raw]
Subject: Re: [RFC][PATCH] Re: 4Gb ram not showing up

Andrew Lyon wrote:
>
> I have run memtest86+ for days at a time on both systems with no errors
> at all.
>
> broken hardware i guess, i am not surprised about the abit board, ive
> had nothing but trouble with abit motherboards and do not use them any
> more, but the supermicro is a xeon server board and i was using
> kingston ram, i wouldnt expect that to have issues.
>
> latest bios on both.
>

And have you verified that it's specced for the memory load you're
putting on it? Not all platforms can use every slot with certain kinds
of DIMMs, depending on chip and rank count.

-hpa

2007-06-09 20:38:57

by Matt Keenan

[permalink] [raw]
Subject: Re: [RFC][PATCH] Re: 4Gb ram not showing up

Andrew Lyon wrote:
> On 6/7/07, H. Peter Anvin <[email protected]> wrote:
>> Andrew Lyon wrote:
>> >
>> > Could this also cause a system to be unstable? my abit athlon64 at
>> > work will not run x64 with more than 1gb ram, and i have a colo server
>> > with supermicro & 2 x dual core xeons that will not run with more than
>> > 2gb.
>> >
>> > Both systems have long uptimes but if i add ram they crash within
>> > minutes of booting.
>> >
>>
>> That's much more likely to be caused by bad memory bus layout so that it
>> doesn't handle the higher loading.
>>
>> Run memtest86+ on the expanded memory for some time.
>>
>> -hpa
>>
>
> I have run memtest86+ for days at a time on both systems with no
> errors at all.
>
> broken hardware i guess, i am not surprised about the abit board, ive
> had nothing but trouble with abit motherboards and do not use them any
> more, but the supermicro is a xeon server board and i was using
> kingston ram, i wouldnt expect that to have issues.
I have had problems with kingston ram and heavily populated boards
(being ASUS boards didn't help either). I have found through trial and
error that gigabyte and/or msi boards with crucial memory seems to work
the best, and for servers i always buy low latency ecc ram, even if it
is a slower frequency. I also had problems with machines that passed
memtest but failed in real life.. you may also want to check your power
supply and make sure you buy boards with multiple (6 or more) voltage
regulators.

Matt