Hello!
I encountered with problem: one process can not allocate more then 2Gb of
memory. Kernel compiled with CONFIG_HIGHMEM4G=y, CONFIG_HIGHMEM=y. Kernel is
2.4.0
As far as I know on i386 linux process has got 32 bit address space. It
means that actually about 3Gb of memory should be available.
I tried to call getrlimit(). It shows only 2G available memory and there is
no way to increase it.
Could you say me are there any solutions? Might be on i386 linux process can
not use more than 2Gb of memory de facto? But I don't see the reason for it:
there is unsigned long type uses everywhere in kernel sources for memory
allocation.
--
Regards,
Ivan Stepnikov.
In mailing-lists.linux-kernel, you wrote:
>I encountered with problem: one process can not allocate more then 2Gb of
>memory.
This is a problem that I have run into myself. I am no kernel expert,
but I think I understand how this issue. Here is how the standard
kernel maps the 4Gb 32-bit address space from the process's point of
view:
128MB program executable mapped (twice)
128MB + program heap
1GB mmap() starts here
3GB kernel
You can see this for yourself by looking at /proc/pid/maps, where pid
is the PID of the process in question.
Now glibc()'s malloc uses mmap() for 'large' allocations, so you get
2GB maximum memory. The way around this is to change the various
numbers in the left-hand column.
For example, you can try the patch per-process-3.5G-IA32-no-PAE-1, at
/pub/linux/kernel/people/andrea/patches/v2.4/2.4.0-test11-pre5/ on
ftp.kernel.org. This will make the kernel space start at 3.5G (so
that CONFIG_4G is required to use more than 384MB of physical RAM) and
the mmap() space start at 224MB, giving 3G288MB of address space for
mmap(). Note that only 96MB is then available for {two copies of your
executable plus your program heap}.
This is more or less the most you can do, but your needs may be best
suited by something in between. The above patch is quite short, so it
is easy to figure out how to do that. The only hidden restriction is
that the size of the kernel space must be a power of 2: 2GB, 1GB,
512MB, etc. As explained to me, this is so that the kernel can easily
test a pointer to see whether it is to kernel space or user space.
Cheers,
Wayne
[Ivan Stepnikov]
> I tried to call getrlimit(). It shows only 2G available memory and
> there is no way to increase it.
Right. Architectural limit. There needs to be some room in the
address space for kernel stuff, I/O, etc -- in Linux at least, having
to play with your page tables every single time you enter a system call
or IRQ handler would be considered a Bad Thing.
> Could you say me are there any solutions?
a) If you have that much memory, maybe you need a 64-bit CPU.
b) fork() and do IPC. It's the Unix Way.
Peter