Hi,
until OLS of this year, most people thought that Dave's shared page
table patch was effectively a database-only hack, and thus not all that
interesting...
however it's also possible to use shared page tables for shared
libraries, and thus the gain is MUCH wider in scope. Dave has been so
kind as to send me his latest patch, and I've done some measurements on
my new x86-64 test machine (which runs FC6-test2)...
and the result is interesting:
Just booting into runlevel 5 and logging into gnome (without starting
any apps) gets a sharing of 1284 pte pages! This means that five
megabytes (!!) of memory is saved, and countless pagefaults are avoided.
Now, as Dave discussed in Ottawa, there is one so-far unsolved gotcha
with shared pagetables: RSS accounting.
The problem is that if application A and B both map a file C, and end up
sharing the pagetable.. page faults that A gets are accounted as the rss
for A, but not in the RSS for B, even though the page is now in the
pagetables for application B.
One possible solution would be to upper-bound estimage the RSS cost of
shareable file mappings to the mapping size; another possible angle is
to dynamically calculate the RSS for file backed mappings (by storing
the rss count in the struct page of the pagetable, and then iterate over
these).
But.. I'm really hoping someone out here has some other smart idea; it'd
be really nice to get this working really well.. after all it's a memory
saving for everyone!
Greetings,
Arjan van de Ven
Notes about the measurement of the number 1284
[1] This only works on x86-64, not i386. For this to work the libraries
need to be mapped at a granularity of 2Mb; i386 uses 4Kb which is too
small
[2] This only works if you use a new enough binutils (such as FC6-test
has); older versions of binutils used 1Mb not 2Mb as alignment.
[3] It really helps to use prelink (as I did), prelink makes sure that a
library is on the same virtual address in all applications, which is
basically a requirement for sharing the pagetables
--
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Arjan van de Ven <[email protected]> writes:
> and the result is interesting:
> Just booting into runlevel 5 and logging into gnome (without starting
> any apps) gets a sharing of 1284 pte pages! This means that five
> megabytes (!!) of memory is saved, and countless pagefaults are avoided.
>
When I start SLES10 GNOME after boot with one firefox window and
one gnome terminal I only have ~5.3MB in total page tables according
to /proc/meminfo
You're saying you can share 5MB of those. Call me sceptical of your
numbers.
-Andi
On Tue, 2006-08-15 at 16:41 +0200, Andi Kleen wrote:
> Arjan van de Ven <[email protected]> writes:
> > and the result is interesting:
> > Just booting into runlevel 5 and logging into gnome (without starting
> > any apps) gets a sharing of 1284 pte pages! This means that five
> > megabytes (!!) of memory is saved, and countless pagefaults are avoided.
> >
>
> When I start SLES10 GNOME after boot with one firefox window and
> one gnome terminal I only have ~5.3MB in total page tables according
> to /proc/meminfo
>
> You're saying you can share 5MB of those. Call me sceptical of your
> numbers.
PageTables: 9836 kB
(this is after sharing, so before it was more)
maybe your gnome is not as bloated as my gnome ;)
--
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
El Tue, 15 Aug 2006 12:34:07 +0200,
Arjan van de Ven <[email protected]> escribi?:
> Just booting into runlevel 5 and logging into gnome (without starting
> any apps) gets a sharing of 1284 pte pages! This means that five
> megabytes (!!) of memory is saved, and countless pagefaults are avoided.
It's possible to get the patch to test it? (saving memory is one of those
things that makes people want to try patches ;) It'd be interesting to see
how this affects to other (mine) environments.
Diego Calleja wrote:
> El Tue, 15 Aug 2006 12:34:07 +0200,
> Arjan van de Ven <[email protected]> escribi?:
>
>> Just booting into runlevel 5 and logging into gnome (without starting
> It's possible to get the patch to test it? (saving memory is one of those
> things that makes people want to try patches ;) It'd be interesting to see
> how this affects to other (mine) environments.
I always wondered what happened to 'mergemem' - which did similar things
in 2.0.33.
On Aug 15, 2006 3:34 AM, Arjan van de Ven wrote:
>
> Hi,
>
> until OLS of this year, most people thought that Dave's shared page
> table patch was effectively a database-only hack, and thus not all that
> interesting...
>
> however it's also possible to use shared page tables for shared
> libraries, and thus the gain is MUCH wider in scope. Dave has been so
> kind as to send me his latest patch, and I've done some measurements on
> my new x86-64 test machine (which runs FC6-test2)...
>
> and the result is interesting:
> Just booting into runlevel 5 and logging into gnome (without starting
> any apps) gets a sharing of 1284 pte pages! This means that five
> megabytes (!!) of memory is saved, and countless pagefaults are avoided.
>
The shared page table patch also gives a nice boost to the fork speed as
tested on a woodcrest machine using lmbench-3.0-a7.
2.6.18-rc4 with shared page table enabled (usec):
fork 136
exec 303
sh 828
2.6.18-rc4 without shared page table (usec):
fork 179
exec 364
sh 949
The test was done with the lmbench option to pin both parent and child
onto the same cpu core, and lmbench includes the overhead of calling
sched_setaffinity.
Tim Chen