All,
TASK_UNMAPPED_SIZE is defined in include/asm-i386/processor.h as:
#define TASK_UNMAPPED_SIZE (TASK_SIZE / 3)
The value of TASK_SIZE is defined as PAGE_OFFSET which is set to 0xC0000000
(page.h). This works out to be a value of 0x4000000.
The question is:
Are there any negative side effects in defining TASK_UNMAPPED_SIZE to 0x1000000?
By doing this we allow a process to access more memory. On Oracle it allows us
to grow our buffer size from 1.7 GB to 2.4 GB improving overall performance by
reducing I/O.
Thanks in advance,
--
=======================================================================
Josue Emmanuel Amaro [email protected]
Linux Products Manager Phone: 650.506.1239
Intel and Linux Technologies Group Fax: 650.413.0167
=======================================================================
On Fri, Nov 03, 2000 at 11:15:46AM -0800, Josue Emmanuel Amaro wrote:
> (page.h). This works out to be a value of 0x4000000.
^ one more zero here
> Are there any negative side effects in defining TASK_UNMAPPED_SIZE to 0x1000000?
I guess you mean 0x10000000. There's no risk in doing that. I also did another
patch that moves the kernel away and allows 3.5G per process on IA32 via plain
mmap or shmat, but it has the downside of reducing a lot the ZONE_NORMAL where
on IA32 buffercache and skb still lives.
Andrea
Andrea Arcangeli <[email protected]> writes:
> On Fri, Nov 03, 2000 at 11:15:46AM -0800, Josue Emmanuel Amaro wrote:
> > (page.h). This works out to be a value of 0x4000000.
> ^ one more zero here
> > Are there any negative side effects in defining TASK_UNMAPPED_SIZE to 0x1000000?
>
> I guess you mean 0x10000000. There's no risk in doing that. I also did another
> patch that moves the kernel away and allows 3.5G per process on IA32 via plain
> mmap or shmat, but it has the downside of reducing a lot the ZONE_NORMAL where
> on IA32 buffercache and skb still lives.
Is this available as a patch, or preferably as a compilation option to
the standard kernel?
The ASIC tools that we use wants all the memory they can grab, and the
extra few hundred megs can sometimes be _very_ critical. On Solaris,
Synopsys' Design Compiler can allocate 3.8Gig and we've hit that limit
hard when working with toplevel netlists and SDF databases.
Thanks in advance,
Kai
--
Kai Harrekilde-Petersen
On Fri, Nov 03, 2000 at 09:27:07PM +0100, Kai Harrekilde-Petersen wrote:
> Is this available as a patch, or preferably as a compilation option to
They're available here:
ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/patches/v2.2/2.2.14/bigmem-large-mapping-1.bz2
ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/patches/v2.2/2.2.14/bigmem-large-task-1.bz2
But they're against 2.2.x + bigmem. The first one is still valid (and it's
similar to the one discussed here). The second one doesn't apply to 2.4.x
and both vmlinux.lds and PAGE_OFFSET should be changed that way to
make it to work there.
Andrea
Andrea,
We will give it a try.
How difficult would it be to move that patch to 2.4?
It would be great if it could be a kernel configuration time option.
Regards,
Andrea Arcangeli wrote:
> On Fri, Nov 03, 2000 at 09:27:07PM +0100, Kai Harrekilde-Petersen wrote:
> > Is this available as a patch, or preferably as a compilation option to
>
> They're available here:
>
> ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/patches/v2.2/2.2.14/bigmem-large-mapping-1.bz2
> ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/patches/v2.2/2.2.14/bigmem-large-task-1.bz2
>
> But they're against 2.2.x + bigmem. The first one is still valid (and it's
> similar to the one discussed here). The second one doesn't apply to 2.4.x
> and both vmlinux.lds and PAGE_OFFSET should be changed that way to
> make it to work there.
>
> Andrea
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> Please read the FAQ at http://www.tux.org/lkml/
--
=======================================================================
Josue Emmanuel Amaro [email protected]
Linux Products Manager Phone: 650.506.1239
Intel and Linux Technologies Group Fax: 650.413.0167
=======================================================================
Hello,
On Fri, 3 Nov 2000, Josue Emmanuel Amaro wrote:
> It would be great if it could be a kernel configuration time option.
Something like the attached old patch for 2.2. It is very
optimistic in using 64MB as min value for TASK_UNMAPPED_BASE while
the real min is above 128MB (where malloc starts). May be needs tuning for
other archs. You still need to select good value for TASK_SIZE/PAGE_OFFSET
at compile time. I run the patch safely with
echo 195887104 > /proc/sys/kernel/task_unmapped_base
Regards
--
Julian Anastasov <[email protected]>
On Sat, Nov 04, 2000 at 01:09:42AM +0000, Julian Anastasov wrote:
> Something like the attached old patch for 2.2. It is very
It's not ok for 64bit archs.
Andrea
All,
Additional follow up.
The idea of modifying this variable is to increase the amount of memory that a
process can use. A database like Oracle can benefit from this because it allows
Oracle to create a bigger data buffer.
An obvious side effect of allocating more physical memory to a process is that there
is less available for other things like the kernel buffer cache. This brings up a
question. Would a write of a data block from the Oracle data buffer incur a "read
penalty" if the block is not in the buffer cache?
For example, a data block is read by Oracle into its buffer. Since it is read
through the file system, this block is now present in the buffer cache too. After a
while, and since we have allocated most of the memory to Oracle, that block is
removed/replaced from the cache.
Now a transaction modifies that block in Oracle's buffer and Oracle writes it to
disk. Does the kernel firsts reads that block into the buffer cache, "read
penalty", and then writes it to disk? Or does it just write through the buffer cache
now placing the block in the kernel buffer cache?
Regards,
--
=======================================================================
Josue Emmanuel Amaro [email protected]
Linux Products Manager Phone: 650.506.1239
Intel and Linux Technologies Group Fax: 650.413.0167
=======================================================================
Hello,
On Sat, 4 Nov 2000, Andrea Arcangeli wrote:
> On Sat, Nov 04, 2000 at 01:09:42AM +0000, Julian Anastasov wrote:
> > Something like the attached old patch for 2.2. It is very
>
> It's not ok for 64bit archs.
Agreed. I see very different definitions for TASK_UNMAPPED_BASE
and ELF_ET_DYN_BASE in all archs and I'm not a guru to make this patch
for all archs but everyone can see the idea. May be the signed int max
value is not suitable for users that need large brk allocations with
more than 2GB on 32bit, i.e. cur_task_unmapped_base is enough. Using
machines with this RAM size and more are already a common practice but
I'm not sure if this feature is so useful compared to the difficulties
to implement it for all archs.
> Andrea
Regards
--
Julian Anastasov <[email protected]>
On Fri, Nov 03, 2000 at 02:36:22PM -0800, Josue Emmanuel Amaro wrote:
> Andrea,
>
> We will give it a try.
>
> How difficult would it be to move that patch to 2.4?
I moved it to 2.4.0-test11-pre5 (should work with pre7 too):
ftp://ftp.us.kernel.org/pub/linux/kernel/people/andrea/patches/v2.4/2.4.0-test11-pre5/per-process-3.5G-IA32-no-PAE-1
It won't work with PAE enabled though (64G option). Making it to work with PAE
enabled isn't much more complicated and I'll address that later (but
in the meantime you can use it on <= 4GB RAM machines).
Andrea