Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::3:3 as permitted sender) client-ip=2620:137:e000::3:3;
From:   NeilBrown <neilb@suse.de>
To:     Steve Dickson <steved@redhat.com>
Cc:     linux-nfs@vger.kernel.org,
        Trond Myklebust <trond.myklebust@hammerspace.com>
Subject: [PATCH 3/6] export: move cache_open() before workers are forked.
Date:   Mon, 23 Oct 2023 12:58:33 +1100
Message-ID: <20231023021052.5258-4-neilb@suse.de>
In-Reply-To: <20231023021052.5258-1-neilb@suse.de>
References: <20231023021052.5258-1-neilb@suse.de>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: bulk

If each worker has a separate open on a cache channel, then each worker
will potentially receive every upcall request resulting in duplicated
work.

A worker will only not see a request that another worker sees if that
other worker answers the request before this worker gets a chance to
read it.

To avoid duplicate effort between threads and so get maximum benefit
from multiple threads, open the cache channels before forking.

Note that the kernel provides locking so that only one thread can be
reading to writing to any channel at any given moment.

Fixes: 5fc3bac9e0c3 ("mountd: Ensure we don't share cache file descriptors among processes.")
Signed-off-by: NeilBrown <neilb@suse.de>
---
 utils/exportd/exportd.c | 8 ++++++--
 utils/mountd/mountd.c   | 8 ++++++--
 2 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/utils/exportd/exportd.c b/utils/exportd/exportd.c
index 2dd12cb6015b..6f866445efc2 100644
--- a/utils/exportd/exportd.c
+++ b/utils/exportd/exportd.c
@@ -289,12 +289,16 @@ main(int argc, char **argv)
 	else if (num_threads > MAX_THREADS)
 		num_threads = MAX_THREADS;
 
+	/* Open cache channel files BEFORE forking so each upcall is
+	 * only handled by one thread.  Kernel provides locking for both
+	 * read and write.
+	 */
+	cache_open();
+
 	if (num_threads > 1)
 		fork_workers();
 
 
-	/* Open files now to avoid sharing descriptors among forked processes */
-	cache_open();
 	v4clients_init();
 
 	/* Process incoming upcalls */
diff --git a/utils/mountd/mountd.c b/utils/mountd/mountd.c
index bcf749fabbb3..f9c62cded66c 100644
--- a/utils/mountd/mountd.c
+++ b/utils/mountd/mountd.c
@@ -916,12 +916,16 @@ main(int argc, char **argv)
 	else if (num_threads > MAX_THREADS)
 		num_threads = MAX_THREADS;
 
+	/* Open cache channel files BEFORE forking so each upcall is
+	 * only handled by one thread.  Kernel provides locking for both
+	 * read and write.
+	 */
+	cache_open();
+
 	if (num_threads > 1)
 		fork_workers();
 
 	nfsd_path_init();
-	/* Open files now to avoid sharing descriptors among forked processes */
-	cache_open();
 	v4clients_init();
 
 	xlog(L_NOTICE, "Version " VERSION " starting");
-- 
2.42.0