Received-SPF: pass (google.com: domain of linux-nfs+bounces-2007-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
From: Dai Ngo <dai.ngo@oracle.com>
To: chuck.lever@oracle.com, jlayton@kernel.org
Cc: linux-nfs@vger.kernel.org
Subject: [PATCH 1/1] NFSD: send OP_CB_RECALL_ANY to clients when number of delegations reach a threshold
Date: Sat, 17 Feb 2024 10:00:59 -0800
Message-Id: <1708192859-25002-1-git-send-email-dai.ngo@oracle.com>
Precedence: bulk

When the number of granted delegation becomes large, there are some
undesire effects happen on the NFS server. Besides the consumption
of system resources, the number of entries on the linked lists of the
file cache can grow significantly.

When this condition happens, the NFS performace grounds to a halt as
reported here [1].

This patch attempts to alleviate this problem by asking the clients to
voluntarily return any unused delegations when the number of delegation
reaches 3/4 of the max_delegations by sending OP_CB_RECALL_ANY to all
clients that hold delegations.

[1] https://lore.kernel.org/all/PH0PR14MB5493F59229B802B871407F9CAA442@PH0PR14MB5493.namprd14.prod.outlook.com

Signed-off-by: Dai Ngo <dai.ngo@oracle.com>
---
 fs/nfsd/nfs4state.c | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
index fdc95bfbfbb6..5fb83853533f 100644
--- a/fs/nfsd/nfs4state.c
+++ b/fs/nfsd/nfs4state.c
@@ -130,6 +130,7 @@ static const struct nfsd4_callback_ops nfsd4_cb_notify_lock_ops;
 static const struct nfsd4_callback_ops nfsd4_cb_getattr_ops;
 
 static struct workqueue_struct *laundry_wq;
+static void deleg_reaper(struct nfsd_net *nn);
 
 int nfsd4_create_laundry_wq(void)
 {
@@ -696,6 +697,9 @@ static struct nfsd_file *find_any_file_locked(struct nfs4_file *f)
 static atomic_long_t num_delegations;
 unsigned long max_delegations;
 
+/* threshold to trigger deleg_reaper */
+static unsigned long delegations_soft_limit;
+
 /*
  * Open owner state (share locks)
  */
@@ -6466,6 +6470,7 @@ nfs4_laundromat(struct nfsd_net *nn)
 	struct nfs4_cpntf_state *cps;
 	copy_stateid_t *cps_t;
 	int i;
+	long n;
 
 	if (clients_still_reclaiming(nn)) {
 		lt.new_timeo = 0;
@@ -6550,6 +6555,9 @@ nfs4_laundromat(struct nfsd_net *nn)
 	/* service the server-to-server copy delayed unmount list */
 	nfsd4_ssc_expire_umount(nn);
 #endif
+	n = atomic_long_inc_return(&num_delegations);
+	if (n > delegations_soft_limit)
+		deleg_reaper(nn);
 out:
 	return max_t(time64_t, lt.new_timeo, NFSD_LAUNDROMAT_MINTIMEOUT);
 }
@@ -8482,6 +8490,7 @@ set_max_delegations(void)
 	 * giving a worst case usage of about 6% of memory.
 	 */
 	max_delegations = nr_free_buffer_pages() >> (20 - 2 - PAGE_SHIFT);
+	delegations_soft_limit = (max_delegations / 4) * 3;
 }
 
 static int nfs4_state_create_net(struct net *net)
-- 
2.39.3