Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <8640dbe0-cece-4515-fa4f-efa2e0a14303@oracle.com>
Date:   Mon, 25 Apr 2022 08:27:22 -0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.8.1
Subject: Re: [PATCH RFC v21 3/7] NFSD: move create/destroy of laundry_wq to
 init_nfsd and exit_nfsd
Content-Language: en-US
From:   dai.ngo@oracle.com
To:     chuck.lever@oracle.com, bfields@fieldses.org
Cc:     jlayton@redhat.com, viro@zeniv.linux.org.uk,
        linux-nfs@vger.kernel.org, linux-fsdevel@vger.kernel.org
References: <1650739455-26096-1-git-send-email-dai.ngo@oracle.com>
 <1650739455-26096-4-git-send-email-dai.ngo@oracle.com>
In-Reply-To: <1650739455-26096-4-git-send-email-dai.ngo@oracle.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?OWxWVDdyc1U1bzdKM3YwNG80UklhUGttQWhURHJNUlVDMXdVQVR1V1k4RXVz?=
 =?utf-8?B?TnFLWjZJSm5DbEVaV2RvUlJZejJqYlk5UVJSNXlzMHFndTN4dGpxVDJoTHJa?=
 =?utf-8?B?dEkyUHdOR1RiQ2R0UTZrUXRRbXA4cTZ6akpLY05lQmlKb28ra2lVb1MwOVBN?=
 =?utf-8?B?ZWphS3ZyRUNNRnZ5bTdwNW9EVHprUzFpSEZxcldrRHh5U1ZRenZxU2hLV2ln?=
 =?utf-8?B?RlROV3IzVHJyS1ZLdHNYUnoxQS9JcldTNFVzSGk5T2NsYW5xTk1ValJIRGEx?=
 =?utf-8?B?TVNBdnI5UVZGbytybnNNRnZ4eG9KRGF0TEdQWkxuUlJ6cm4xNkwwVDF6LzFE?=
 =?utf-8?B?bHlJMzZoYXFMWXFxWTBOV1VqUUhwSm5oUHJsQ294Y2xFL01Jbit4VW1SZjd4?=
 =?utf-8?B?QURHaXdXSzhyd1pqWWRkMHF4aWRhalVoZGFadW1VKy9oU1RkUUJpL3U2MkZ4?=
 =?utf-8?B?Z040bDg4R3B6YkxVZjlRNCs5TFBoUjBlZVBTbnhWNi9IdFk4aE5xZ0NmYlRh?=
 =?utf-8?B?Y0tROVdOTlF5MHVDdGtWMm84alJXMWRPWllRajZyYWlXbUNGdW1tVVl2dlVQ?=
 =?utf-8?B?N0NmKzBUYVJQR3lHNWtCekIvUldJUzVyUHliSHgycVozOHVZa0VzRUw4UE4y?=
 =?utf-8?B?RHprMUZvSzVkQzlWRHgxRSsvcHNDS1VFUGo0Q216ekdQV1ZaQlZKYXh3bUpx?=
 =?utf-8?B?OVM5cnRwVUNoOTVEamk1RUVuMmVxdDEvcDRIQmFoUUJueEpzTmJ2SVQzVWVS?=
 =?utf-8?B?ck1kSkhSQVdWRHZmYk5LbzAxVlVZZ3ZiRjlWN0g2a1R0d25BMVoxd3JtaXZv?=
 =?utf-8?B?WkV3empYVEhxandudFhuZUREM2NnOWl1VTBlUkRzT1YvMUJBbE4wNWE1N1I3?=
 =?utf-8?B?T2NlUWtvN3hXZHdBdU1sTEt2dFBaUzF0WUdpKzhucFZqVGJHdFBJSHFqNUh1?=
 =?utf-8?B?RXphYjRpbmhLbDR2RmlsSzkxSmtCaDVJT3lVVmhDQ1I4NXNUYmJiUDhrMEhv?=
 =?utf-8?B?THhpNm8xVC96RWN6UmM2djFyMmowWitEOEEwUkZxVGNMNDl6S1dVN2lXdnRU?=
 =?utf-8?B?VGVFOUxlc3c1Q2xHdVJFbFUzQlBDdytGNGo1TjlsQlF0TFJ4NVIzSUZuY2RT?=
 =?utf-8?B?a2xDR3FlaDRKbG5kLzNZRzNXQUl0Q3VQNEcrUHAyd3ZoOGczMGhvcnJscVgw?=
 =?utf-8?B?WjFvaFBTVm9GWUd0dk9rdFczLzFXQy91QWxXTU53WDFDM2U4andVUG9uemZ2?=
 =?utf-8?B?bWlBSjMzOTZydHZoNFR5dExFcXhnM3RUcStNV1ovY205dkplSXBxUjVVaGd0?=
 =?utf-8?B?MGEyUWt6aU9hTUszWExyOFhLVkZoSStUejF3YVRmT05lajRGTWpKSitEY3BU?=
 =?utf-8?B?NjRqTTM1WXRLajY1NjZHRjFXRW51L1pMNU1ydjBFSUFoL0ZpOWRYcnhOcW5F?=
 =?utf-8?B?UWZrdGlIYlVxNDdVWnVlbFpuZ2ZqL1B5dGE2MW9GOFN5Zk4zL3A1azRKQWRS?=
 =?utf-8?B?RkJRb1RDcDVzdjRPT05zOWZQSy81RDR1WUJRZ1E3VS9kT3ZmUzVJSnBFdjRv?=
 =?utf-8?B?bWtEUWgvMEhkQ3ExMlQ3bTh6cjcrME5wdDdScUEyVlk4S0VQS0I3RmczcUVw?=
 =?utf-8?B?MHh0RmlXRDBOaXk2Kzd1U3dHUVhBbjVWV1EyVzJiUVphREdlbUhZZ1JtWUdk?=
 =?utf-8?B?MnNQVmdOSHlYamZqMUMyMXBVNzE5VTIvNkxQRjdkTGJIZDgxemxKRWhvcFc0?=
 =?utf-8?B?TnUwYmdBekJnQkxOZGhyNlFsbDVFT1lvcG05SmswQTEwTFVKSlJ0by9uTkdX?=
 =?utf-8?B?L1FRN2U5SkZDYkpIMnJnRXkrcnVLMy9jbnhlQ3JtK2xyN3V2Ymw5aEptaS9r?=
 =?utf-8?B?VmZJUDBRS0l2dGVqTm1Ja0NObzVEYWxzbWVmSzJSZldGRzR1T3dUNUR3YjRK?=
 =?utf-8?B?VFRSaDhXS3FvTUFaUDNKVy9ETGNJZ1RlemxwaHp6aGhnTXAybXZhek0vVk5u?=
 =?utf-8?B?UlZSUzRkak9semJIOUVLNCsrbTRqOU9QOXRhd0I3Zk5kam4vZnJuZExnUGZ4?=
 =?utf-8?B?T1ZEdHdUbE9kdzgzTzdlMHhEYUhkNnVuZEdydkN2TVNHbnd0d05ZcVBtbWcv?=
 =?utf-8?B?NCtPRm9UUE85V3BVMWhyUklIV01EaEQ0T3dYSFREcUJXZjQ2cHVFVjFFVlpE?=
 =?utf-8?B?cC9GQUlzVU16aHVUSy9WY1hYalNkMUtOMmNaSEFsanc5UUZPL3cwNnFxZjZu?=
 =?utf-8?B?SzhBTlc2Nm9YcmoyMUxLYlg5NUV0dS96cGZYbHJSaGRDdlRwZUlmbG5KdU96?=
 =?utf-8?B?V0toTm1FTU9paEZIRFJnUEYwU0pCWlo0anJuRVFGbUdIejBCRVRvZz09?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 5c326985-da0c-48cd-fb62-08da26d01985
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4257.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 25 Apr 2022 15:27:25.0547
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: NkAqzkn9jm+pOWPrfR/n0ZtiNB8vHVOgh2zng+3BBOhPnAXpYOTZldUWUYpOC3YywSATOlBcbQ1sYXMAdOITcQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR10MB5404
X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.486,18.0.858
 definitions=2022-04-25_06:2022-04-25,2022-04-25 signatures=0
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxlogscore=999
 suspectscore=0 malwarescore=0 adultscore=0 spamscore=0 mlxscore=0
 phishscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2202240000 definitions=main-2204250067
X-Proofpoint-GUID: yfaDE2qeNlCHQZlNgk-w02F-X2mG5YMY
X-Proofpoint-ORIG-GUID: yfaDE2qeNlCHQZlNgk-w02F-X2mG5YMY
X-Spam-Status: No, score=-4.7 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org

This patch has problem to build with this error:

>> nfsctl.c:(.exit.text+0x0): undefined reference to `laundry_wq'
>> mipsel-linux-ld: nfsctl.c:(.exit.text+0x4): undefined reference to `laundry_wq'

This happens when CONFIG_NFSD is defined but CONFIG_NFSD_V4
is not. I think to fix this we need to also move the declaration
of laundry_wq from nfs4state.c to nfsctl.c. However this seems
odd since the laundry_wq is only used for v4, unless you have
any other suggestion.

-Dai

On 4/23/22 11:44 AM, Dai Ngo wrote:
> This patch moves create/destroy of laundry_wq from nfs4_state_start
> and nfs4_state_shutdown_net to init_nfsd and exit_nfsd to prevent
> the laundromat from being freed while a thread is processing a
> conflicting lock.
>
> Signed-off-by: Dai Ngo <dai.ngo@oracle.com>
> ---
>   fs/nfsd/nfs4state.c | 15 ++-------------
>   fs/nfsd/nfsctl.c    |  6 ++++++
>   fs/nfsd/nfsd.h      |  1 +
>   3 files changed, 9 insertions(+), 13 deletions(-)
>
> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
> index b08c132648b9..b70ba2eb5665 100644
> --- a/fs/nfsd/nfs4state.c
> +++ b/fs/nfsd/nfs4state.c
> @@ -125,7 +125,7 @@ static void free_session(struct nfsd4_session *);
>   static const struct nfsd4_callback_ops nfsd4_cb_recall_ops;
>   static const struct nfsd4_callback_ops nfsd4_cb_notify_lock_ops;
>   
> -static struct workqueue_struct *laundry_wq;
> +struct workqueue_struct *laundry_wq;
>   
>   static bool is_session_dead(struct nfsd4_session *ses)
>   {
> @@ -7798,22 +7798,12 @@ nfs4_state_start(void)
>   {
>   	int ret;
>   
> -	laundry_wq = alloc_workqueue("%s", WQ_UNBOUND, 0, "nfsd4");
> -	if (laundry_wq == NULL) {
> -		ret = -ENOMEM;
> -		goto out;
> -	}
>   	ret = nfsd4_create_callback_queue();
>   	if (ret)
> -		goto out_free_laundry;
> +		return ret;
>   
>   	set_max_delegations();
>   	return 0;
> -
> -out_free_laundry:
> -	destroy_workqueue(laundry_wq);
> -out:
> -	return ret;
>   }
>   
>   void
> @@ -7850,7 +7840,6 @@ nfs4_state_shutdown_net(struct net *net)
>   void
>   nfs4_state_shutdown(void)
>   {
> -	destroy_workqueue(laundry_wq);
>   	nfsd4_destroy_callback_queue();
>   }
>   
> diff --git a/fs/nfsd/nfsctl.c b/fs/nfsd/nfsctl.c
> index 16920e4512bd..884e873b46ad 100644
> --- a/fs/nfsd/nfsctl.c
> +++ b/fs/nfsd/nfsctl.c
> @@ -1544,6 +1544,11 @@ static int __init init_nfsd(void)
>   	retval = register_cld_notifier();
>   	if (retval)
>   		goto out_free_all;
> +	laundry_wq = alloc_workqueue("%s", WQ_UNBOUND, 0, "nfsd4");
> +	if (laundry_wq == NULL) {
> +		retval = -ENOMEM;
> +		goto out_free_all;
> +	}
>   	return 0;
>   out_free_all:
>   	unregister_pernet_subsys(&nfsd_net_ops);
> @@ -1566,6 +1571,7 @@ static int __init init_nfsd(void)
>   
>   static void __exit exit_nfsd(void)
>   {
> +	destroy_workqueue(laundry_wq);
>   	unregister_cld_notifier();
>   	unregister_pernet_subsys(&nfsd_net_ops);
>   	nfsd_drc_slab_free();
> diff --git a/fs/nfsd/nfsd.h b/fs/nfsd/nfsd.h
> index 23996c6ca75e..d41dcf1c4312 100644
> --- a/fs/nfsd/nfsd.h
> +++ b/fs/nfsd/nfsd.h
> @@ -72,6 +72,7 @@ extern unsigned long		nfsd_drc_max_mem;
>   extern unsigned long		nfsd_drc_mem_used;
>   
>   extern const struct seq_operations nfs_exports_op;
> +extern struct workqueue_struct *laundry_wq;
>   
>   /*
>    * Common void argument and result helpers