Hello,
Similar to
<https://unix.stackexchange.com/questions/202586/limit-the-inode-cache-used-by-a-command>
I would like to prevent certain programs from trashing the inode/dentry
cache, which is a shared resource for all processes:
- For example the nightly <man:updatedb(8)> used <man:find(1) > to
recursively walk the complete file system. As long as `d_name` and the
`d_type` information from <man:readdir(3)> is enough this only pollutes
the dentry cache.
- Similar our backup software, but this also needs to <man:stat(2)> each
path to get the `mtime`, which additionally pollutes the inode cache.
Both examples only walk the tree once (per day). In my case the caches
do not fit into memory completely, so the second process does not even
benefit from the first process filling the cache as that data is already
replaced again.
The trashed caches affect all other processes running in parallel or the
first processes started each morning.
Is it possible to prevent inode/dentry trashing for example by limiting
the cache per process(-group)?
Something like MADV_DONTNEED from <man:madvise(2)> for IO would be nice.
An external knob to limit the cache usage per process(-group) would be
nice, but even a hint for an API for such kind of programs to prevent
trashing would help me.
Thank you in advance.
Philipp
Hello Phillip.
On Mon, Jun 07, 2021 at 02:39:35PM +0200, Philipp Hahn <[email protected]> wrote:
> The trashed caches affect all other processes running in parallel or the
> first processes started each morning.
>
> Is it possible to prevent inode/dentry trashing for example by limiting the
> cache per process(-group)?
Yes. Unless you have disabled it with CONFIG_MEMCG_KMEM or
cgroup.memory=nokmem, dentries and inodes are charged to respective
cgroups. And you can limit overall memory of a cgroup, see
memory.{max,high} attributes. (You suggest this inode/dentry consumption
is dominant enough to affect other jobs, so the limit would keep it
constrained as you intend).
HTH,
Michal
On 25.06.21 17:55, Michal Koutn? wrote:
>> Is it possible to prevent inode/dentry trashing for example by limiting the
>> cache per process(-group)?
>
> Yes. Unless you have disabled it with CONFIG_MEMCG_KMEM or
> cgroup.memory=nokmem, dentries and inodes are charged to respective
> cgroups. And you can limit overall memory of a cgroup, see
> memory.{max,high} attributes. (You suggest this inode/dentry consumption
> is dominant enough to affect other jobs, so the limit would keep it
> constrained as you intend).
Could you please tell a bit more how this really works ?
(maybe some pointers to the code)
I'm curios what happens if those cache objects are used by different
cgroups - are they accounted to multiple times (once per cgroup) ?
What happens when one cgroup using some cache object reaching its limit,
wile another one does not ?
--mtx
--
---
Hinweis: unverschl?sselte E-Mails k?nnen leicht abgeh?rt und manipuliert
werden ! F?r eine vertrauliche Kommunikation senden Sie bitte ihren
GPG/PGP-Schl?ssel zu.
---
Enrico Weigelt, metux IT consult
Free software and Linux embedded engineering
[email protected] -- +49-151-27565287
On Mon, Jun 28, 2021 at 11:40:39AM +0200, "Enrico Weigelt, metux IT consult" <[email protected]> wrote:
> Could you please tell a bit more how this really works ?
> (maybe some pointers to the code)
When cgroup's consumption is about to cross the configured limit,
reclaim is started
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/memcontrol.c?id=62fb9874f5da54fdb243003b386128037319b219#n2579
that may evict old entries
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/vmscan.c?id=62fb9874f5da54fdb243003b386128037319b219#n2852
and if there's still no success freeing some space the dentry allocation
can fail
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/mm/slab.h?id=62fb9874f5da54fdb243003b386128037319b219#n277
(This describes just one code path, the subject isn't always a dentry.)
> I'm curios what happens if those cache objects are used by different
> cgroups - are they accounted to multiple times (once per cgroup) ?
> What happens when one cgroup using some cache object reaching its limit,
> wile another one does not ?
That's explained here
https://www.kernel.org/doc/html/v5.13/admin-guide/cgroup-v2.html#memory-ownership
Michal