Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
MIME-Version: 1.0
References: <CAPt2mGOaRsKOiL_wuSK_D5oYYnn0R-pvVsZc5HYGdEbT2FngtQ@mail.gmail.com>
 <20220124193759.GA4975@fieldses.org> <adce2b72-ed5c-3056-313c-caea9bad4e15@math.utexas.edu>
 <20220125212055.GB17638@fieldses.org> <164315533676.5493.13243313269022942124@noble.neil.brown.name>
 <20220126025722.GD17638@fieldses.org> <CAPt2mGP2guMMf1C9VoQ0AvZ819jPuz0vDoEzJJhtL8q5DJ300A@mail.gmail.com>
 <CAPt2mGNXq==1KUskF3U6-CDeoX57=d7NW4Qn_esDqarf9bTBaw@mail.gmail.com>
 <20220211155949.GA4941@fieldses.org> <CAPt2mGOx0qNTWoY9vmyVBtZ3gxdbv5qQ-2qVbtqWW9FiZFRhEg@mail.gmail.com>
 <164517040900.10228.8956772146017892417@noble.neil.brown.name> <CAPt2mGMLQCEPqsYGeaMd3BPGRne4F4h-2-pzqm1a8nwfKqv1ug@mail.gmail.com>
In-Reply-To: <CAPt2mGMLQCEPqsYGeaMd3BPGRne4F4h-2-pzqm1a8nwfKqv1ug@mail.gmail.com>
From:   Daire Byrne <daire@dneg.com>
Date:   Mon, 25 Apr 2022 14:00:32 +0100
Message-ID: <CAPt2mGMt3Sq66qmPBeGYE0CASTTy7nY2K_LjQK6VZx-uz2P-wg@mail.gmail.com>
Subject: Re: parallel file create rates (+high latency)
To:     NeilBrown <neilb@suse.de>
Cc:     "J. Bruce Fields" <bfields@fieldses.org>,
        Patrick Goetz <pgoetz@math.utexas.edu>,
        linux-nfs <linux-nfs@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Precedence: bulk

On Mon, 21 Feb 2022 at 13:59, Daire Byrne <daire@dneg.com> wrote:
>
> On Fri, 18 Feb 2022 at 07:46, NeilBrown <neilb@suse.de> wrote:
> > I've ported it to mainline without much trouble.  I started some simple
> > testing (parallel create/delete of the same file) and hit a bug quite
> > easily.  I fixed that (eventually) and then tried with more than 1 CPU,
> > and hit another bug.  But then it was quitting time.  If I can get rid
> > of all the easy to find bugs, I'll post it with a CC to you, and you can
> > find some more for me!
>
> That would be awesome! I have a real world production case for this
> and it's a pretty heavy workload. If that doesn't shake out any bugs,
> nothing will.
>
> The only caveat being that it will likely be restricted to NFSv3
> testing due to the concurrency limitations with NFSv4.1+ (from the
> other thread).
>
> Daire

Just to follow up on this again - I have been using Neil's patch for
parallel file creates (thanks!) but I'm a bit confused as to why it
doesn't seem to help in my NFS re-export case.

With the patch, I can achieve much higher parallel (multi process)
creates directly on my re-export server to a high latency remote
server mount, but when I re-export that to multiple clients, the
aggregate create rate again degrades to that which we might expect
either without the patch or if there was only one process creating the
files in sequence.

My assumption was that the nfsd threads of the re-export server would
act as multiple independent processes and it's clients would be spread
across them such that they would also benefit from the parallel
creates patch on the re-export server. So I expected many clients
creating files in the same directory would achieve much higher
aggregate performance.

Am I missing some other interaction here that limits parallel
performance in my unusual re-export case?

Cheers,

Daire