Received-SPF: pass (google.com: domain of linux-nfs+bounces-1435-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.80.249 as permitted sender) client-ip=147.75.80.249;
Date: Thu, 25 Jan 2024 16:01:27 -0500
From: Chuck Lever <chuck.lever@oracle.com>
To: Josef Bacik <josef@toxicpanda.com>
Cc: linux-nfs@vger.kernel.org, kernel-team@fb.com
Subject: Re: [PATCH v2 10/13] nfsd: move th_cnt into nfsd_net
Message-ID: <ZbLMJxLWIvomQIzO@tissot.1015granger.net>
References: <cover.1706212207.git.josef@toxicpanda.com>
 <0fa7bf5b5bbc863180e50363435b5a56c43dc5e3.1706212208.git.josef@toxicpanda.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <0fa7bf5b5bbc863180e50363435b5a56c43dc5e3.1706212208.git.josef@toxicpanda.com>
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: 
	=?us-ascii?Q?Hikhrsjalis9SL2N3ppAh474SbpozYjW4r+RxW+L/Qig6WTCY7NoAvIoPDOf?=
 =?us-ascii?Q?lXXPlQifmQeq+g6O7eNXL7nz7vQr6Ca1qygi4IhHHmvgYzPtYRuk4BpeCaiB?=
 =?us-ascii?Q?Qft+3uYsoszT9rbNmp/g0ktiM2tk7hRMdiFVLiiLeakyiuT2aMTtgrHCATvc?=
 =?us-ascii?Q?nmZWh53ePnknZLLKT47k3cy57FyW282d0ZlPsu+fd3uhXphQR6L8socIk1OL?=
 =?us-ascii?Q?tmKujmD2cYPtO+OG7NtvZ3mgFSulOscHBLQDBrJfLWqPEXeK/1+rHR9s33r3?=
 =?us-ascii?Q?6xFHo8kffPoN0D1Eegy2SAL5yr89vTkf8XEoaP3KmDnalXW/hx363O4+lKH0?=
 =?us-ascii?Q?cvrPoalzGhgWFYZxnXML8GIkbf90nWVBBJF/gVMtEgPtonWzijP+4O1j9QFa?=
 =?us-ascii?Q?xt3u7OGsntsjAeYy7zgFpEsa45NAdyxvk6v6xGmEy1k3ZUKbrwoUmpvtAJuC?=
 =?us-ascii?Q?GRUffuelr2OlUvXjTe7kFwc5Seght2mSkvoUMeDCwC9uApjbRoRoDwDUu+Qr?=
 =?us-ascii?Q?zpo79dIv89cRwJIwxHN630oTcCg3n2DRzobyrV5t6N6giHkqodbWqFhg9+7B?=
 =?us-ascii?Q?Opz7QB4wliS4+F0aNPpPXlDepnQq5K1E65VtnuJBBj6pPj3pz5553m3gyc7D?=
 =?us-ascii?Q?O5RH2ThCAyzLe6ko/Ma/DXR7hXcYaXvN/LKwxGhTzTonjj9Y2q5KHVpGXnkD?=
 =?us-ascii?Q?0pbxLUMYwqDlfJ5qGjq0RvN7gDFbLDeWsm0RghvEQSijirud4ZcBI/nre1lw?=
 =?us-ascii?Q?y1KoQXuRdbQRstl7G/R5nd+8kbXb0lpx6znZSGqhh+6rubbX54Yl6tvx8OLs?=
 =?us-ascii?Q?WomprBR6IUKhxm9FS7KkRyRcoy0dUyHbtMC7W6ad8fjh/rFVxBVzzIdd9Cy/?=
 =?us-ascii?Q?SLXCEtptv6a+H6lAWCiCuuXewQQvY6aG9B1AD4v+XBtEXfpS1EqzgvabZ0E9?=
 =?us-ascii?Q?sMORE38VP/8bvKnRro/HNDUTPcVcwWjwMBdBManAinaxl9ALrvac0HEmi8WQ?=
 =?us-ascii?Q?duvAO9BpAIHriL7jVL0slJSB822qD3mmZvdvArRP/Xy1iLDfEmAzj3T+shvN?=
 =?us-ascii?Q?VRvekokNj/1d9LAT3cNKQB4dZsxPeVEapyqW4qy/iA5G4Aug1+0smaF49Jgg?=
 =?us-ascii?Q?5JsSDuFwcK4+ONyaUTshVZd+usNXVDV1e0nFi+pn3AAoWyRwENVfkqCwZmZr?=
 =?us-ascii?Q?6D2zH2HDWs19CpxLkzEAd5R+L5r7l86cjqiIWl+9KII/vTmMU3jR/TTGLD/Y?=
 =?us-ascii?Q?mtLbP/8E8Hnd5uyK1/hXr0Y9gQnA2GcvnfwiCG7lepeOZj/ccXsY8HVCAT9I?=
 =?us-ascii?Q?ihZFslsTtxuNAB2aoqkFMaIQML+w9UxqN2rbwJK7RUjOwO2AlPS7+ul34grZ?=
 =?us-ascii?Q?5/mME78EzzTDrc6XzYTrFzVMn2DVYaPABdD+RIEdEcTdaL5yql+jl7H5NQH+?=
 =?us-ascii?Q?MWqA3c/U58cKuaKKwoWzY1MwUxjFh2NQM72p43zGTves9UtUDtNmhkS9sZhd?=
 =?us-ascii?Q?UvYcQgQHyZXfxlZf2CMQf9+fwgsrVHK/NSP1P0Bak50jOZ5lE/YOHwJnIcyW?=
 =?us-ascii?Q?+VO6+9RAgiSK0wgTTVuuTSGNNKaYYgwGDNzyzpu0z7at9u80m+05oFqRjzcN?=
 =?us-ascii?Q?xg=3D=3D?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: 
	pHh9MKaok6CiKmaHokIBy2EcK0YE4kf7TV3bbxu62vOVoGaTcT1VEBBmFYVxVdMZUb16TJ+T/ApGoj46CxWOgINwn5qeg4H2wYBRa92GS1Hb0BTNXu18qnB1MTLGCotd6UUNr5drGIJPIjTU8NJ4e0M6GhIfeAmeIRiE0L59iSkJ+COEd27nnA5IN/RFXOTgzFwiYwPX+NireYPkipMfsc+JI92xREPPdyKwAX96rXnO2k6f3mcg+d/kAEjuypdlRb/wYjaYiT4kvGl9xTROSG5oVibYYx39uM3ysvmHu/qbMm+K9OiAgNxmpd0PWe273NwpMdiNj8Z4zv3Tg83hWbhf8NYZ6zoqG3gcrEpuhjcN+KkAdU5mzYb32ZrcbxoaSlVXhvv2JEzKa1QmZyT1dm031MifkGGzw/qygSeSBYswLyoiFaWwhTRHPc9xvHQ5njrhrped3oerNgIIGB7nn2c88XIsQkIq5DofCnwj8QbdpRf5HhBcshRi9+dKNrIOO8eb9fu7y9MLJBHYYnAEGPrefDs9UW7T0+u0VikJwgBut08w90maUhZYjfMpo5HRN8eClpgx9JeQTNdW7bcGpfsYmQ0keX8eoGOD0+uK1zA=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 37913340-7c7b-4011-a633-08dc1de8cda6
X-MS-Exchange-CrossTenant-AuthSource: BN0PR10MB5128.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Jan 2024 21:01:30.1136
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: +NEKMp/PqrjZp0sEb/AUILhboLP28Oed70Tle/l1QCFsoFzgdP9rMATxAVzXav0e4EN4GMcl5IuHZ6q30CGOmQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR10MB6963
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.272,Aquarius:18.0.1011,Hydra:6.0.619,FMLib:17.11.176.26
 definitions=2024-01-25_13,2024-01-25_01,2023-05-22_02
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 mlxlogscore=976
 spamscore=0 mlxscore=0 adultscore=0 bulkscore=0 phishscore=0
 malwarescore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2311290000 definitions=main-2401250151
X-Proofpoint-ORIG-GUID: MvcAlh79f_KWpLYUGK05Ldzn4_jVbGrt
X-Proofpoint-GUID: MvcAlh79f_KWpLYUGK05Ldzn4_jVbGrt

On Thu, Jan 25, 2024 at 02:53:20PM -0500, Josef Bacik wrote:
> This is the last global stat, move it into nfsd_net and adjust all the
> users to use that variant instead of the global one.

Hm. I thought nfsd threads were a global resource -- they service
all network namespaces. So, shouldn't the same thread count be
surfaced to all containers? Won't they all see all of the nfsd
processes?


> Signed-off-by: Josef Bacik <josef@toxicpanda.com>
> ---
>  fs/nfsd/netns.h  | 3 +++
>  fs/nfsd/nfssvc.c | 4 ++--
>  fs/nfsd/stats.c  | 4 ++--
>  fs/nfsd/stats.h  | 6 ------
>  4 files changed, 7 insertions(+), 10 deletions(-)
> 
> diff --git a/fs/nfsd/netns.h b/fs/nfsd/netns.h
> index 0cef4bb407a9..8d3f4cb7cab4 100644
> --- a/fs/nfsd/netns.h
> +++ b/fs/nfsd/netns.h
> @@ -179,6 +179,9 @@ struct nfsd_net {
>  	/* Per-netns stats counters */
>  	struct percpu_counter    counter[NFSD_STATS_COUNTERS_NUM];
>  
> +	/* number of available threads */
> +	atomic_t                 th_cnt;
> +
>  	/* longest hash chain seen */
>  	unsigned int             longest_chain;
>  
> diff --git a/fs/nfsd/nfssvc.c b/fs/nfsd/nfssvc.c
> index d98a6abad990..0961b95dcef6 100644
> --- a/fs/nfsd/nfssvc.c
> +++ b/fs/nfsd/nfssvc.c
> @@ -924,7 +924,7 @@ nfsd(void *vrqstp)
>  
>  	current->fs->umask = 0;
>  
> -	atomic_inc(&nfsdstats.th_cnt);
> +	atomic_inc(&nn->th_cnt);
>  
>  	set_freezable();
>  
> @@ -940,7 +940,7 @@ nfsd(void *vrqstp)
>  		nfsd_file_net_dispose(nn);
>  	}
>  
> -	atomic_dec(&nfsdstats.th_cnt);
> +	atomic_dec(&nn->th_cnt);
>  
>  out:
>  	/* Release the thread */
> diff --git a/fs/nfsd/stats.c b/fs/nfsd/stats.c
> index 44e275324b06..360e6dbf4e5c 100644
> --- a/fs/nfsd/stats.c
> +++ b/fs/nfsd/stats.c
> @@ -27,7 +27,6 @@
>  
>  #include "nfsd.h"
>  
> -struct nfsd_stats	nfsdstats;
>  struct svc_stat		nfsd_svcstats = {
>  	.program	= &nfsd_program,
>  };
> @@ -47,7 +46,7 @@ static int nfsd_show(struct seq_file *seq, void *v)
>  		   percpu_counter_sum_positive(&nn->counter[NFSD_STATS_IO_WRITE]));
>  
>  	/* thread usage: */
> -	seq_printf(seq, "th %u 0", atomic_read(&nfsdstats.th_cnt));
> +	seq_printf(seq, "th %u 0", atomic_read(&nn->th_cnt));
>  
>  	/* deprecated thread usage histogram stats */
>  	for (i = 0; i < 10; i++)
> @@ -112,6 +111,7 @@ void nfsd_percpu_counters_destroy(struct percpu_counter counters[], int num)
>  
>  int nfsd_stat_counters_init(struct nfsd_net *nn)
>  {
> +	atomic_set(&nn->th_cnt, 0);
>  	return nfsd_percpu_counters_init(nn->counter, NFSD_STATS_COUNTERS_NUM);
>  }
>  
> diff --git a/fs/nfsd/stats.h b/fs/nfsd/stats.h
> index c24be4ddbe7d..5675d283a537 100644
> --- a/fs/nfsd/stats.h
> +++ b/fs/nfsd/stats.h
> @@ -10,12 +10,6 @@
>  #include <uapi/linux/nfsd/stats.h>
>  #include <linux/percpu_counter.h>
>  
> -struct nfsd_stats {
> -	atomic_t	th_cnt;		/* number of available threads */
> -};
> -
> -extern struct nfsd_stats	nfsdstats;
> -
>  extern struct svc_stat		nfsd_svcstats;
>  
>  int nfsd_percpu_counters_init(struct percpu_counter *counters, int num);
> -- 
> 2.43.0
> 
> 

-- 
Chuck Lever