The idea of page coloring is based on the fact that common implementations
of caching can't put any page in memory in any line in the cache (such an
implementation is possible, but is more expensive to do so is not commonly
done)
With this implementation it means that if your program happens to use
memory that cannot be mapped to half of the cache lines then effectivly
the CPU cache is half it's rated size for your program. the next time your
program runs it may get a more favorable memory allocation and be able to
use all of the cache and therefor run faster.
Page coloring is an attampt to take this into account when allocating
memory to programs so that every program gets to use all of the cache.
David Lang
On Fri, 24 Jan 2003, Anoop J. wrote:
> Date: Fri, 24 Jan 2003 10:38:03 +0530 (IST)
> From: Anoop J. <[email protected]>
> To: [email protected], [email protected]
>
>
> How does page coloring work. Iwant its mechanism not the implementation.
> I went through some pages of W.L.Lynch's paper on cache and VM. Still not
> able to grasp it .
>
>
> Thanks in advance
>
>
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
>
The big challenge in Linux is that several serious attempts to add
page coloring have foundered on the shoals of "no benefit found". It
may be that the typical hardware Linux runs on just doesn't experience
the problem very much.
john
On Thu, 23 Jan 2003 21:11:10 -0800 (PST), David Lang
<[email protected]> wrote:
>The idea of page coloring is based on the fact that common implementations
>of caching can't put any page in memory in any line in the cache (such an
>implementation is possible, but is more expensive to do so is not commonly
>done)
>
>With this implementation it means that if your program happens to use
>memory that cannot be mapped to half of the cache lines then effectivly
>the CPU cache is half it's rated size for your program. the next time your
>program runs it may get a more favorable memory allocation and be able to
>use all of the cache and therefor run faster.
>
>Page coloring is an attampt to take this into account when allocating
>memory to programs so that every program gets to use all of the cache.
>
>David Lang
>
>
> On Fri, 24 Jan 2003, Anoop J. wrote:
>
>> Date: Fri, 24 Jan 2003 10:38:03 +0530 (IST)
>> From: Anoop J. <[email protected]>
>> To: [email protected], [email protected]
>>
>>
>> How does page coloring work. Iwant its mechanism not the implementation.
>> I went through some pages of W.L.Lynch's paper on cache and VM. Still not
>> able to grasp it .
>>
>>
>> Thanks in advance
>>
>>
>>
>> -
>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>> the body of a message to [email protected]
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>> Please read the FAQ at http://www.tux.org/lkml/
>>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to [email protected]
>More majordomo info at http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at http://www.tux.org/lkml/
At 10:06 PM 1/23/03 -0800, John Alvord wrote:
>The big challenge in Linux is that several serious attempts to add
>page coloring have foundered on the shoals of "no benefit found". It
>may be that the typical hardware Linux runs on just doesn't experience
>the problem very much.
Another strike against page coloring is that it gives tremendous benefits
when caches are large and not very associative, but if both of these are
not present the benefits are much smaller. In the case of latter-day PCs,
neither of these is the case: the caches are very small and at least 8-way
set associative.
For the record, I finally got to try my own page coloring patch on a 1GHz
Athlon Thunderbird system with 256kB L2 cache. With the present patch, my
own number crunching benchmarks and a kernel compile don't show any benefit
at all, and lmbench is completely unchanged except for the mmap latency,
which is slightly worse. Hardly a compelling case for PCs!
Oh well. At least now I'll be able to port to 2.5 :)
jasonp
> For the record, I finally got to try my own page coloring patch on a 1GHz
> Athlon Thunderbird system with 256kB L2 cache. With the present patch, my
> own number crunching benchmarks and a kernel compile don't show any benefit
> at all, and lmbench is completely unchanged except for the mmap latency,
> which is slightly worse. Hardly a compelling case for PCs!
If it works correctly then the variability in lat_ctx should go away.
Try this
for p in 2 4 8 12 16 24 32 64
do for size in 0 2 4 8 16
do for i in 1 2 3 4 5 6 7 8 9 0
do lat_ctx -s$size $p
done
done
done
on both the with and without kernel. The page coloring should make the
numbers rock steady, without it, they will bounce a lot.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
Larry McVoy <[email protected]> writes:
> > For the record, I finally got to try my own page coloring patch on a 1GHz
> > Athlon Thunderbird system with 256kB L2 cache. With the present patch, my
> > own number crunching benchmarks and a kernel compile don't show any benefit
> > at all, and lmbench is completely unchanged except for the mmap latency,
> > which is slightly worse. Hardly a compelling case for PCs!
>
> If it works correctly then the variability in lat_ctx should go away.
> Try this
>
> for p in 2 4 8 12 16 24 32 64
> do for size in 0 2 4 8 16
> do for i in 1 2 3 4 5 6 7 8 9 0
> do lat_ctx -s$size $p
> done
> done
> done
>
> on both the with and without kernel. The page coloring should make the
> numbers rock steady, without it, they will bounce a lot.
On the same kind of vein I have seen some tremendous variability in the
stream benchmark. Under linux I have gotten it to very as much
as a 100MB/sec by running updatedb, between runs. In one case
it ran faster with updatedb running in the background.
But at the same time streams tends to be very steady if you have a quiet
machine and run it several times in a row repeatedly because it gets
allocated essentially the same memory every run.
So I do no the variables of cache contention do have effect on some
real programs. I have not yet tracked it down to see if cache coloring
could be a benefit. I suspect the buddy allocator actually comes
quite close most of the time, and tricks like allocating multiple pages
at once could improve that even more with very little effort, while reducing
page fault miss times.
I am wondering if there is any point in biasing page addresses in between
processes so that processes are less likely to have a cache conflict.
i.e. process 1 address 0 %16K == 0, process 2 address 0 %16K == 4K
Eric
> I am wondering if there is any point in biasing page addresses in between
> processes so that processes are less likely to have a cache conflict.
> i.e. process 1 address 0 %16K == 0, process 2 address 0 %16K == 4K
All good page coloring implementation do exactly that. The starting
index into the page buckets is based on process id.
--
---
Larry McVoy lm at bitmover.com http://www.bitmover.com/lm
On Sat, 2003-01-25 at 15:10, Larry McVoy wrote:
> All good page coloring implementation do exactly that. The starting
> index into the page buckets is based on process id.
I think everyone interested in learning more about this
topic should go read the following papers, they were very
helpful when I was fiddling around in this area.
These papers, in turn, reference several others which are
good reads as well.
1) W. L. Lynch, B. K. Bray, and M. J. Flynn. "The effect of page
allocation on caches". In Micro-25 Conference Proceedings, pages
222-225, December 1992.
2) W. Lynch and M. Flynn. "Cache improvements through colored page
allocation". ACM Transactions on Computer Systems, 1993. Submitted
for review, 1992.
3) William L. Lynch. "The Interaction of Virtual Memory and Cache
Memory". PhD thesis, Stanford University, October
1993. CSL-TR-93-587.