Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <612ef738-20f6-55f0-1677-cc035ba2fd0d@oracle.com>
Date:   Tue, 29 Mar 2022 09:20:02 -0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.7.0
Subject: Re: [PATCH RFC v18 02/11] NFSD: Add courtesy client state, macro and
 spinlock to support courteous server
Content-Language: en-US
To:     "J. Bruce Fields" <bfields@fieldses.org>
Cc:     chuck.lever@oracle.com, jlayton@redhat.com,
        viro@zeniv.linux.org.uk, linux-nfs@vger.kernel.org,
        linux-fsdevel@vger.kernel.org
References: <1648182891-32599-1-git-send-email-dai.ngo@oracle.com>
 <1648182891-32599-3-git-send-email-dai.ngo@oracle.com>
 <20220329154750.GE29634@fieldses.org>
From:   dai.ngo@oracle.com
In-Reply-To: <20220329154750.GE29634@fieldses.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?SmNYNWlZQkxEZTFwTjJvU2tYVG5QcWFrSkJ1YmJHTTV6eGxLWmRlb0xCcnl3?=
 =?utf-8?B?d3dJSzVKNUQ2UnAvdkh6VE9nTmV2U2x0aHAvUDV3TkUxZTc1Q0pmN3c1SE50?=
 =?utf-8?B?SllTeXBJU0tjVTNIcEJHcXorei90OVdxNHJIMmRpcjk3aHpjTW85aVVjdEJv?=
 =?utf-8?B?TUoxdXpDL216ejJzN1Z2bjZ6MmVaeS9ZWkpMbGQrbnQvTis1VXQ3TUhKcFh6?=
 =?utf-8?B?MERGajgwSUZhMVNxQjZjOTRNSXduWGR0N0h1dk95YjdNbmJGV0NVcGJrMEJx?=
 =?utf-8?B?alVkWGtMUmJZYTU4aXZ1VVUxNkxHY3lFZVB5bi85WFQ5YzBoQTNBZGc0OTJW?=
 =?utf-8?B?MktLNlJScVRyN0t6WEIycjNRUHdpN3cycDZHR1YxT09VZDkvclBNcDVWN0Ry?=
 =?utf-8?B?Y2drSndmUUxWK0tUSUVMUElnMlVPazdGSWRCUnRnN3gzb0pXQlUzSXJFbGo4?=
 =?utf-8?B?TTJrMHRETHVFeVZUN1VuanQwZzlGSkdlRHJLMW5tSTBlOXBpSlN6aFU1OGJa?=
 =?utf-8?B?dkpLRUdKVjRsMmxIZHkxekpxNDJsT25PR01naWpyS2xnUmhWVE50OVlKcWRx?=
 =?utf-8?B?L3gzY09PZG9GOWp5Y2hUalVESG8vdUlhcjF1alJpajVDYnRaUVdwR3R2eC80?=
 =?utf-8?B?Uk5GeHoyYWhKRDZPRXcxMFlRWmY1MmpHcXlZYmI2UXB1OHQwYlMvOHBhbUFO?=
 =?utf-8?B?bURhc0M2N0E5YTFXRVcrU01PRHRTbnRkcG83dzhaNFVlamwvR0Y3RTE4UlRN?=
 =?utf-8?B?REE5ZTlzODI5WXNjUHlmYkZZMWxzK1p1ZGhsbmJVMis5amFRVmhVMGw4ZzFK?=
 =?utf-8?B?RC84b2JteGJ4NUN0d0hxd0J1NG5meHFsSHR1Sktud2RiWnlJNmt4M21EdW5E?=
 =?utf-8?B?M1k0Z05oc2REOTJlRDljRE1nOFZPcDZ6VlN4bmRHdDVCcTlUV2JHbnY5ZlJ3?=
 =?utf-8?B?b3BNdHljNkFrQVBjdFVWdEkwbFkzcmRzOTlkRjE3SU9vRjNISVc0Mk9OVXlG?=
 =?utf-8?B?RXJpbnRVNWU2RE5HbGpwYkUyT3VCZ0ZyeEJjQmg1Sm5sa0dmSFpXVkg5MkVs?=
 =?utf-8?B?dWx2bVd0ODAwc2xPM2VjSmZvaGQrM3hyNUZTMUoxeERmMDhjUXFwbkNkbGJx?=
 =?utf-8?B?MC9sUDNONmdzWmw1ZXlJOStVcHRZYXgzVUNMZzVHeTA1NW5jKzF0MC9QMlRp?=
 =?utf-8?B?Z3pKZ1ZhVjUwNmlhTnNWUkQ1eHUrS2QySUlnNlJoWTNxZ0hUZUQwU3h2NXhR?=
 =?utf-8?B?NmF4cVIyVkxTeTg4aHpUdFNTbjV5K1M1bWF6ZDFtb3VrQitWSmFheGxTV25F?=
 =?utf-8?B?ZHZnL0hoTDFBWFhBSUVUTU54OFdkUWpDRzBNTHQ4aUtSMWQ1T1NvMXNpYjV3?=
 =?utf-8?B?Y0h2M1VlZC9zdVZ3alhRcmd2ZG96STI4bTJJRTBOdEw5blpRNGFRa2hOOHNH?=
 =?utf-8?B?ekxQRDlQTE52QXZ6UXZUb0ZKSzdFT0NmaDZyQ1drdnNmOVVjYWVRZFRrUGtY?=
 =?utf-8?B?a0U0VVI5TlRoSHphN3pXcG8vKzM0VFpqYVVpUjB1OXZUdnlQelpwMzREVVho?=
 =?utf-8?B?WGtJbGF6RnkwM0QzNUhlZXovc0hKeTcwN0dwNWF4UlFROVIwM1NkajM0ekxk?=
 =?utf-8?B?MkRYY1VjMzI0aGhzRGljY3c1S2xPaXZsOGZVZFVoUTVZTTVZUDNzOWVySStC?=
 =?utf-8?B?elA3ZzgzMFRkcWlJT2syRGtTR2h5SDVGUFJuc0RYY2F5V2s2OWgxVzdIc21Q?=
 =?utf-8?B?eFZZVEdTcTVKRzUwOGVvWDRQTXhFQjJXUktTZmlwWkJ1V29HSU9HM0NhZVNr?=
 =?utf-8?B?OXRNUmlqa1o0QkxzZDJ0NE1xajV1c0xFTGNDME80bG9yT3RPTEY3aWMrTmlt?=
 =?utf-8?B?Q0IwTTFIbHExSkdFUXVBaDVSZDNDdjF3UWNlaVVoQm1DK3FlZHZqRlhKM004?=
 =?utf-8?B?RlU0eVgvVi9OOXloQ0Q1bjBnbnhaZlhpeko5Y1pVYTFIdDNvem1aWlpsUjZp?=
 =?utf-8?B?NVh1SWJlOFN1R29LOG5yRld3bUdPanIwbXhGUWt3SUJNN2k4OGVPWkNoV0ZB?=
 =?utf-8?B?UUI1ZUhwRzZFVzk0Q2U0YXpqVGs0V3hxamJDREpwakMrYTFJWm1MSi92OGNR?=
 =?utf-8?B?bEVHbGE2WFJEbFFtVkwvemgrdzJXWkZOZC9ENEY1N1lOaHNlWFVTaUxDNW9E?=
 =?utf-8?B?M0Y5bVAxM0RObnpxUnhEcDFWeU9td0FMYVY0c0VSM2RLUy9jYWtYd3U5bTI5?=
 =?utf-8?B?bG1qU2d3bldoc0FKZnVrNDhrMVlGNGxaNEhRRDNnT1MwbDBMeHlIUUVJK3dX?=
 =?utf-8?B?MGJVWnVidGdGZnNLL3pnWWV2dDIwZFduNFFDdndOZVZGUU1CN2xWZz09?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 53db6588-33dc-4e10-a415-08da119ffc3d
X-MS-Exchange-CrossTenant-AuthSource: MN2PR10MB4270.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 29 Mar 2022 16:20:05.6966
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: n0L6tFZA56YXnYs4LVrMv4xjGadpzjd9FlWlCZTcaQXIHTNbzl0bFSLiP6Sc3/eudM3zLWmZ31Z4r2aURb2INA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW5PR10MB5739
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10301 signatures=695566
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxscore=0 malwarescore=0 spamscore=0
 phishscore=0 adultscore=0 mlxlogscore=999 suspectscore=0 bulkscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000
 definitions=main-2203290094
X-Proofpoint-ORIG-GUID: SqyZpb2GGvUQjhJ2u7K-Lyl8-6Kdm3ET
X-Proofpoint-GUID: SqyZpb2GGvUQjhJ2u7K-Lyl8-6Kdm3ET
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


On 3/29/22 8:47 AM, J. Bruce Fields wrote:
> On Thu, Mar 24, 2022 at 09:34:42PM -0700, Dai Ngo wrote:
>> Update nfs4_client to add:
>>   . cl_cs_client_state: courtesy client state
>>   . cl_cs_lock: spinlock to synchronize access to cl_cs_client_state
>>   . cl_cs_list: list used by laundromat to process courtesy clients
>>
>> Modify alloc_client to initialize these fields.
>>
>> Signed-off-by: Dai Ngo <dai.ngo@oracle.com>
>> ---
>>   fs/nfsd/nfs4state.c |  2 ++
>>   fs/nfsd/nfsd.h      |  1 +
>>   fs/nfsd/state.h     | 33 +++++++++++++++++++++++++++++++++
>>   3 files changed, 36 insertions(+)
>>
>> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
>> index 234e852fcdfa..a65d59510681 100644
>> --- a/fs/nfsd/nfs4state.c
>> +++ b/fs/nfsd/nfs4state.c
>> @@ -2009,12 +2009,14 @@ static struct nfs4_client *alloc_client(struct xdr_netobj name)
>>   	INIT_LIST_HEAD(&clp->cl_delegations);
>>   	INIT_LIST_HEAD(&clp->cl_lru);
>>   	INIT_LIST_HEAD(&clp->cl_revoked);
>> +	INIT_LIST_HEAD(&clp->cl_cs_list);
>>   #ifdef CONFIG_NFSD_PNFS
>>   	INIT_LIST_HEAD(&clp->cl_lo_states);
>>   #endif
>>   	INIT_LIST_HEAD(&clp->async_copies);
>>   	spin_lock_init(&clp->async_lock);
>>   	spin_lock_init(&clp->cl_lock);
>> +	spin_lock_init(&clp->cl_cs_lock);
>>   	rpc_init_wait_queue(&clp->cl_cb_waitq, "Backchannel slot table");
>>   	return clp;
>>   err_no_hashtbl:
>> diff --git a/fs/nfsd/nfsd.h b/fs/nfsd/nfsd.h
>> index 4fc1fd639527..23996c6ca75e 100644
>> --- a/fs/nfsd/nfsd.h
>> +++ b/fs/nfsd/nfsd.h
>> @@ -336,6 +336,7 @@ void		nfsd_lockd_shutdown(void);
>>   #define COMPOUND_ERR_SLACK_SPACE	16     /* OP_SETATTR */
>>   
>>   #define NFSD_LAUNDROMAT_MINTIMEOUT      1   /* seconds */
>> +#define	NFSD_COURTESY_CLIENT_TIMEOUT	(24 * 60 * 60)	/* seconds */
>>   
>>   /*
>>    * The following attributes are currently not supported by the NFSv4 server:
>> diff --git a/fs/nfsd/state.h b/fs/nfsd/state.h
>> index 95457cfd37fc..40e390abc842 100644
>> --- a/fs/nfsd/state.h
>> +++ b/fs/nfsd/state.h
>> @@ -283,6 +283,35 @@ struct nfsd4_sessionid {
>>   #define HEXDIR_LEN     33 /* hex version of 16 byte md5 of cl_name plus '\0' */
>>   
>>   /*
>> + * CLIENT_  CLIENT_ CLIENT_
>> + * COURTESY EXPIRED RECONNECTED      Meaning                  Where set
>> + * -----------------------------------------------------------------------------
>> + * | false | false | false | Confirmed, active    | Default                    |
>> + * |---------------------------------------------------------------------------|
>> + * | true  | false | false | Courtesy state.      | nfs4_get_client_reaplist   |
>> + * |       |       |       | Lease/lock/share     |                            |
>> + * |       |       |       | reservation conflict |                            |
>> + * |       |       |       | can cause Courtesy   |                            |
>> + * |       |       |       | client to be expired |                            |
>> + * |---------------------------------------------------------------------------|
>> + * | false | true  | false | Courtesy client to be| nfs4_laundromat            |
>> + * |       |       |       | expired by Laundromat| nfsd4_lm_lock_expired      |
>> + * |       |       |       | due to conflict     | nfsd4_discard_courtesy_clnt |
>> + * |       |       |       |                      | nfsd4_expire_courtesy_clnt |
>> + * |---------------------------------------------------------------------------|
>> + * | false | false | true  | Courtesy client      | nfsd4_courtesy_clnt_expired|
>> + * |       |       |       | reconnected,         |                            |
>> + * |       |       |       | becoming active      |                            |
>> + * -----------------------------------------------------------------------------
> These are mutually exclusive values, not bits that may set to 0 or 1, so
> the three boolean columns are confusing.  I'd just structure the table
> like:
>
> 	client state	meaning			where set
> 	0		Confirmed, active	Default
> 	CLIENT_COURTESY	Courtesy state....	nfs4_get_client_reaplist
> 	CLIENT_EXPIRED	Courtesy client to be..	nfs4_laundromat
>
> etc.

will fix in v19.

Thanks,
-Dai

>
> --b.
>
>> + */
>> +
>> +enum courtesy_client_state {
>> +	NFSD4_CLIENT_COURTESY = 1,
>> +	NFSD4_CLIENT_EXPIRED,
>> +	NFSD4_CLIENT_RECONNECTED,
>> +};
>> +
>> +/*
>>    * struct nfs4_client - one per client.  Clientids live here.
>>    *
>>    * The initial object created by an NFS client using SETCLIENTID (for NFSv4.0)
>> @@ -385,6 +414,10 @@ struct nfs4_client {
>>   	struct list_head	async_copies;	/* list of async copies */
>>   	spinlock_t		async_lock;	/* lock for async copies */
>>   	atomic_t		cl_cb_inflight;	/* Outstanding callbacks */
>> +
>> +	enum courtesy_client_state	cl_cs_client_state;
>> +	spinlock_t		cl_cs_lock;
>> +	struct list_head	cl_cs_list;
>>   };
>>   
>>   /* struct nfs4_client_reset
>> -- 
>> 2.9.5