Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 31 Mar 2022 19:45:49 +0000
From:   Al Viro <viro@zeniv.linux.org.uk>
To:     Stephen Brennan <stephen.s.brennan@oracle.com>
Cc:     Andrew Morton <akpm@linux-foundation.org>,
        Luis Chamberlain <mcgrof@kernel.org>,
        Arnd Bergmann <arnd@arndb.de>,
        Matthew Wilcox <willy@infradead.org>,
        James Bottomley <James.Bottomley@hansenpartnership.com>,
        Gao Xiang <hsiangkao@linux.alibaba.com>,
        Dave Chinner <david@fromorbit.com>,
        Roman Gushchin <roman.gushchin@linux.dev>,
        Colin Walters <walters@verbum.org>,
        linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [RFC PATCH 2/2] fs/dcache: Add negative-dentry-ratio config
Message-ID: <YkYE7RXgP8hL9aLd@zeniv-ca.linux.org.uk>
References: <20220331190827.48241-1-stephen.s.brennan@oracle.com>
 <20220331190827.48241-3-stephen.s.brennan@oracle.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220331190827.48241-3-stephen.s.brennan@oracle.com>
Sender: Al Viro <viro@ftp.linux.org.uk>
Precedence: bulk

On Thu, Mar 31, 2022 at 12:08:27PM -0700, Stephen Brennan wrote:
> Negative dentry bloat is a well-known problem. For systems without
> memory pressure, some workloads (like repeated stat calls) can create an
> unbounded amount of negative dentries quite quickly. In the best case,
> these dentries could speed up a subsequent name lookup, but in the worst
> case, they are never used and their memory never freed.
> 
> While systems without memory pressure may not need that memory for other
> purposes, negative dentry bloat can have other side-effects, such as
> soft lockups when traversing the d_subdirs list or general slowness with
> managing them. It is a good idea to have some sort of mechanism for
> controlling negative dentries, even outside memory pressure.
> 
> This patch attempts to do so in a fair way. Workloads which create many
> negative dentries must create many dentries, or convert dentries from
> positive to negative. Thus, negative dentry management is best done
> during these same operations, as it will amortize its cost, and
> distribute the cost to the perpetrators of the dentry bloat. We
> introduce a sysctl "negative-dentry-ratio" which sets a maximum number
> of negative dentries per positive dentry, N:1. When a dentry is created
> or unlinked, the next N+1 dentries of the parent are scanned. If no
> positive dentries are found, then a candidate negative dentry is killed.

Er...  So what's to stop d_move() from leaving you with your cursor
pointer poiting into the list of children of another parent?

What's more, your dentry_unlist() logics will be defeated by that -
if victim used to have a different parent, got moved, then evicted,
it looks like you could end up with old parent cursor pointing
to the victim and left unmodified by dentry_unlist() (since it looks
only at the current parent's cursor).  Wait for it to be freed and
voila - access to old parent's cursor will do unpleasant things.

What am I missing here?