Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
MIME-Version: 1.0
From:   Pradeep <pradeep.thomas@gmail.com>
Date:   Tue, 30 Jun 2020 17:55:15 -0700
Message-ID: <CAD8zhTBCEbOkA_jFzzDXcd0jiH9ZgUh1KUJAJzhnWFMC+JGjNA@mail.gmail.com>
Subject: File create performance between NFSv3 and NFSv4.
To:     Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-nfs-owner@vger.kernel.org
Precedence: bulk

Hello,

While testing a performance issue, I noticed that the number of
creates that an NFS client can do over NFSv4 is significantly lower
compared to NFSv3. I'm using the test below:

https://github.com/distributed-system-analysis/smallfile

Command run:
smallfile_cli.py  --operation create --threads 128 --file-size 1
--files 1024 --top <nfs-dir>
[ 128 threads, each doing 1024 file creates of 1K size]

This gives around 1169 creates/sec with NFSv4.1 and 8073 creates/sec
with NFSv3. This is with the exact same client and server. NFS server
is tuned to use 200 threads.

When I looked at tcpdump, I noticed that over NFSv3 multiple parallel
requests are being sent and NFSv4 is pretty much serial. Is there
anything that I can tune to improve NFSv4 performance w.r.t to file
creations?

This also shows up in benchmarks like SpecFS SWBUILD's INIT phase
where millions of files are getting created.

Thanks,
Pradeep