Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <164568221518.25116.18139840533197037520@noble.neil.brown.name>
 <893053D7-E5DD-43DB-941A-05C10FF5F396@dilger.ca> <20220224233848.GC8269@magnolia>
 <164600974741.15631.8678502963654197325@noble.neil.brown.name>
In-Reply-To: <164600974741.15631.8678502963654197325@noble.neil.brown.name>
From:   Daire Byrne <daire@dneg.com>
Date:   Thu, 10 Mar 2022 11:10:08 +0000
Message-ID: <CAPt2mGNkQFu=czuM+XO7aVF3AmbPoiq_SU72-Bgfax59JyMkdg@mail.gmail.com>
Subject: Re: [PATCH/RFC] VFS: support parallel updates in the one directory.
To:     NeilBrown <neilb@suse.de>
Cc:     "Darrick J. Wong" <djwong@kernel.org>,
        Andreas Dilger <adilger@dilger.ca>,
        Dave Chinner <david@fromorbit.com>,
        Al Viro <viro@zeniv.linux.org.uk>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        linux-fsdevel@vger.kernel.org, LKML <linux-kernel@vger.kernel.org>,
        Andreas Dilger <adilger.kernel@dilger.ca>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

Just to add that I have also been testing this patch with heavy
production workloads over high latency NFS clients (200ms) and have
not seen any issues so far.

As the latency increases, parallel operations for multiple client
processes becomes ever more important for maintaining good aggregate
throughputs, be it reads, writes or metadata.

With 1000 client processes/threads we see the file creates per single
directory increase from 3 per second to 1200 per second with this
patch.

It solves a real world application for us (high latency NFS clients)
without having to redesign our workflows around deeper (hashed)
directory structures.

Much appreciated Neil.

Daire


On Mon, 28 Feb 2022 at 00:56, NeilBrown <neilb@suse.de> wrote:
>
> On Fri, 25 Feb 2022, Darrick J. Wong wrote:
> > On Thu, Feb 24, 2022 at 09:31:28AM -0700, Andreas Dilger wrote:
> > > On Feb 23, 2022, at 22:57, NeilBrown <neilb@suse.de> wrote:
> > > >
> > > > for i in {1..70}; do ( for j in {1000..8000}; do touch $j; rm -f $j ; done ) & done
> >
> > I think you want something faster here, like ln to hardlink an existing
> > file into the directory.
> >
>
> And probably written in C too..
>
>
> > (I am also not a fan of "PAR_UPDATE", since 'par' is already an English
> > word that doesn't mean 'parallel'.)
>
> :-)
> We already have DCACHE_PAR_LOOKUP for parallel lookups in a directory
> (though it is really a lock bit and I think should be named as soch).
> So it made sense to use DCACHE_PAR_UPDATE for parallel updates.
> And then S_PAR_UPDATE for the inode flag to enable this seemed logical.
>
> But I agree that these names are sub-par :-)
>
> NeilBrown