There currently does not exist a way to answer the question: "What is in
the page cache?". There are various heuristics and counters but nothing
that can tell you anything like:
* 3M from /home/dxu/foo.txt
* 5K from ...
* etc.
The answer to the question is particularly useful in the stacked
container world. Stacked containers implies multiple containers are run
on the same physical host. Memory is precious resource on some (if not
most) of these systems. On these systems, it's useful to know how much
duplicated data is in the page cache. Once you know the answer, you can
do something about it. One possible technique would be bind mount common
items from the root host into each container.
NOTES:
* This patch compiles and (maybe) works -- totally not fully tested
or in a final state
* I'm sending this early RFC to get comments on the general approach.
I chatted w/ Johannes a little bit and it seems like the best way to
do this is through superblock -> inode -> address_space iteration
rather than going from numa node -> LRU iteration
* I'll most likely add a page_hash() helper (or something) that hashes
a page so that userspace can more easily tell which pages are
duplicate
Daniel Xu (1):
bpf: Introduce iter_pagecache
kernel/bpf/Makefile | 2 +-
kernel/bpf/pagecache_iter.c | 293 ++++++++++++++++++++++++++++++++++++
2 files changed, 294 insertions(+), 1 deletion(-)
create mode 100644 kernel/bpf/pagecache_iter.c
--
2.26.3
On Wed, Apr 07, 2021 at 02:46:10PM -0700, Daniel Xu wrote:
> There currently does not exist a way to answer the question: "What is in
> the page cache?". There are various heuristics and counters but nothing
> that can tell you anything like:
>
> * 3M from /home/dxu/foo.txt
> * 5K from ...
> * etc.
>
> The answer to the question is particularly useful in the stacked
> container world. Stacked containers implies multiple containers are run
> on the same physical host. Memory is precious resource on some (if not
Just to clarify: what are "stacked containers"? Do you mean nested
containers, i.e. containers running within containers?
Christian
Hi Christian, thanks for taking a look.
On Thu, Apr 08, 2021 at 09:51:17AM +0200, Christian Brauner wrote:
> On Wed, Apr 07, 2021 at 02:46:10PM -0700, Daniel Xu wrote:
> > There currently does not exist a way to answer the question: "What is in
> > the page cache?". There are various heuristics and counters but nothing
> > that can tell you anything like:
> >
> > * 3M from /home/dxu/foo.txt
> > * 5K from ...
> > * etc.
> >
> > The answer to the question is particularly useful in the stacked
> > container world. Stacked containers implies multiple containers are run
> > on the same physical host. Memory is precious resource on some (if not
>
> Just to clarify: what are "stacked containers"? Do you mean nested
> containers, i.e. containers running within containers?
I mean multiple containers running side by side on the same host.
Thanks,
Daniel
On Wed, Apr 7, 2021 at 2:47 PM Daniel Xu <[email protected]> wrote:
>
> There currently does not exist a way to answer the question: "What is in
> the page cache?". There are various heuristics and counters but nothing
> that can tell you anything like:
>
> * 3M from /home/dxu/foo.txt
> * 5K from ...
> * etc.
>
So, you need the list of inodes for a filesystem that are in memory.
With that list, you can open, map and do mincore on them?
I don't know of a way to get that inode list.
> The answer to the question is particularly useful in the stacked
> container world. Stacked containers implies multiple containers are run
> on the same physical host. Memory is precious resource on some (if not
> most) of these systems. On these systems, it's useful to know how much
> duplicated data is in the page cache. Once you know the answer, you can
> do something about it. One possible technique would be bind mount common
> items from the root host into each container.
>
Oh you want to share page cache between different containers/jobs?
That would complicate charging.
On Wed, Apr 07, 2021 at 02:46:10PM -0700, Daniel Xu wrote:
> There currently does not exist a way to answer the question: "What is in
> the page cache?". There are various heuristics and counters but nothing
> that can tell you anything like:
>
> * 3M from /home/dxu/foo.txt
> * 5K from ...
5K? That's an extraordinary Weird Machine(tm).
> * etc.
>
> The answer to the question is particularly useful in the stacked
> container world. Stacked containers implies multiple containers are run
> on the same physical host. Memory is precious resource on some (if not
> most) of these systems. On these systems, it's useful to know how much
> duplicated data is in the page cache. Once you know the answer, you can
> do something about it. One possible technique would be bind mount common
> items from the root host into each container.
Um, are you describing a system that uses BPF to deduplicating the page
cache by using bind mounts? Can the containers scribble on these files
and thereby mess up the other containers? What happens if the container
wants to update itself and clobbers the root host's copy instead? How
do you deal with a software update process failing because the root host
fights back against the container trying to change its files?
Also, I thought we weren't supposed to share resources across security
boundaries anymore?
--D
>
> NOTES:
>
> * This patch compiles and (maybe) works -- totally not fully tested
> or in a final state
>
> * I'm sending this early RFC to get comments on the general approach.
> I chatted w/ Johannes a little bit and it seems like the best way to
> do this is through superblock -> inode -> address_space iteration
> rather than going from numa node -> LRU iteration
>
> * I'll most likely add a page_hash() helper (or something) that hashes
> a page so that userspace can more easily tell which pages are
> duplicate
>
> Daniel Xu (1):
> bpf: Introduce iter_pagecache
>
> kernel/bpf/Makefile | 2 +-
> kernel/bpf/pagecache_iter.c | 293 ++++++++++++++++++++++++++++++++++++
> 2 files changed, 294 insertions(+), 1 deletion(-)
> create mode 100644 kernel/bpf/pagecache_iter.c
>
> --
> 2.26.3
>
On Thu, Apr 08, 2021 at 04:13:32PM -0700, Darrick J. Wong wrote:
> On Wed, Apr 07, 2021 at 02:46:10PM -0700, Daniel Xu wrote:
> > There currently does not exist a way to answer the question: "What is in
> > the page cache?". There are various heuristics and counters but nothing
> > that can tell you anything like:
> >
> > * 3M from /home/dxu/foo.txt
> > * 5K from ...
>
> 5K? That's an extraordinary Weird Machine(tm).
Just typing random numbers :)
> > * etc.
> >
> > The answer to the question is particularly useful in the stacked
> > container world. Stacked containers implies multiple containers are run
> > on the same physical host. Memory is precious resource on some (if not
> > most) of these systems. On these systems, it's useful to know how much
> > duplicated data is in the page cache. Once you know the answer, you can
> > do something about it. One possible technique would be bind mount common
> > items from the root host into each container.
>
> Um, are you describing a system that uses BPF to deduplicating the page
> cache by using bind mounts? Can the containers scribble on these files
> and thereby mess up the other containers? What happens if the container
> wants to update itself and clobbers the root host's copy instead? How
> do you deal with a software update process failing because the root host
> fights back against the container trying to change its files?
No, the BPF progs are not intended to modify the pages. This is just for
read only observability.
> Also, I thought we weren't supposed to share resources across security
> boundaries anymore?
I can't speak to this, but bpf progs can pretty much be attached to
anywhere so this iterator doesn't expose anything new.
<...>
Thanks,
Daniel