Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <17e69efc-f290-e729-5292-f470780126a5@oracle.com>
Date:   Tue, 22 Mar 2022 17:42:58 -0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.7.0
Subject: Re: [PATCH RFC v17 05/11] NFSD: Update nfs4_get_vfs_file() to handle
 courtesy client
Content-Language: en-US
To:     "J. Bruce Fields" <bfields@fieldses.org>
Cc:     chuck.lever@oracle.com, jlayton@redhat.com,
        viro@zeniv.linux.org.uk, linux-nfs@vger.kernel.org,
        linux-fsdevel@vger.kernel.org
References: <1647503028-11966-1-git-send-email-dai.ngo@oracle.com>
 <1647503028-11966-6-git-send-email-dai.ngo@oracle.com>
 <20220321214400.GA19715@fieldses.org>
From:   dai.ngo@oracle.com
In-Reply-To: <20220321214400.GA19715@fieldses.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?U0VMNjdqQmh2SmRhcDF2eVB6SHRBK0xXSmtvOFkweHZXMUJDZXJRSWN3Sm5C?=
 =?utf-8?B?N3ZlaEhnMHhjYmowaUQ2T2pwaWk1NnVYaGVjU3o3a0NMbE5JWEEvK0VCUk55?=
 =?utf-8?B?MFpsMHFYSFc1TGVaY2NKSFRmV2RKbGVKa3pGNkh6RlljZVNHdDJoZ2RBSlNO?=
 =?utf-8?B?T2ppZzVzUmtjL0RVT0ltWHNzKzQ1TXVaZFVLWmJ6OHM5R1NvNm1kSlNrZkRK?=
 =?utf-8?B?QU8yRCtEWExNRmRLbVdTZzNhU09PaWFSWVpZRkpkQkkxeVhWNEZRQVJ4YU0z?=
 =?utf-8?B?L1J3MTZKQUF4dHMrY1NjRUlkemtlbElRanZVcmxWdzNYSk5rTDdWMDBaTEFM?=
 =?utf-8?B?S1VKLzc2ZHZUM1VLMHZ2c3c0TkdBR0JBTFhWTFNVaXFNUXN6TlJUM0ZodHBO?=
 =?utf-8?B?QzlCRlBKalgzNm5pUU9zUThyWU1IMlZxVk44b3Q1cXNRd3FlNWJhbW9oTGV4?=
 =?utf-8?B?enBac0NhaEFqRFVJRE5DNENiMmZ3UElRbFpZZXRJSTFvL3NwdEhQcTBqMDVI?=
 =?utf-8?B?b1JPcXduaFdvbml1RWNvaW15MkF4RnJRR25yWEt1U1JlbklYK1FzaUt0amtm?=
 =?utf-8?B?WmhNYlFzZUQwWjY3ZEhnZFR3NnRaWXNTeTNwc05MWXR5L3I5SWpFWnE5eDRJ?=
 =?utf-8?B?UWExZ3pTY29QSUtxWHRXZ1lPaWdtMk5JVXhYNVRnNFVUY3M3QlhiOUZiamE4?=
 =?utf-8?B?L0FNbzh1MXY4N05GS3pYbFNpeGhublB3NkVLZlhFM2xJQnIwYmNJMTIyUEtx?=
 =?utf-8?B?UkV6YmhVK3hJWGlrQnRldzlDUGpibTdZbTJrNTlZVHczaW9WaC9tZnBLZkE3?=
 =?utf-8?B?R1loYURHYnM4eExpKzMvSHRub3lLQVZGRVluS0pjc1dNOUdteGVoR2Q0RFpy?=
 =?utf-8?B?bC90RHVoSC9FRnRwYy9TUnNvNmpnQ2Niek9adSttQ2Qxa0l1WHZoZEoxZjBo?=
 =?utf-8?B?YXRqMC9QcHFSQ0pUZXh5WnRwWk9VbVNFc29HK1RsZWRiaDl3OWxVV3RjV3RI?=
 =?utf-8?B?SzZZV0UwMjdNVEhjV3RQcHhDbmx6L0hOSERwZEV6WWpyZTllRS9TQWlOdFpa?=
 =?utf-8?B?M01QN2dUR0M3UTdYRUhqWVRDRlBDbmtYWHRuWU51M1cyM0RTNnBFK3M2UUhk?=
 =?utf-8?B?TjdkeVUrcm1RODY4YmhDWi9TNlVWRkh2VkVWMHBXUDNTaEFIM0g2eVdiMWlR?=
 =?utf-8?B?QUVCOFdMZ0I0dTFtQTRITDdNR2hqelAxTEFFa1FVVDJveXVsRGwyYk5veU1X?=
 =?utf-8?B?OG5XaDdjc1doQmZjSE1tYzkxbzRUdUhiL1pqd21xQ2piazdLL1Y0akQ0SVcr?=
 =?utf-8?B?ZWFCQ1RWM2pFNXh1K0lNZC9aWk9FUFowYnBHYjZZanRPaEhrRDlIUlV4QUtq?=
 =?utf-8?B?aVhTV3BQR3BucWtIenZObDEwK2dycWZhS21nTjIrZUdRVTh2S1RUc1BCcUlV?=
 =?utf-8?B?UGplcExCRWtNemVHb2NrZVBjNWRQWnV0amxNb29MNzd1eFd4MnI1SFVOUkxM?=
 =?utf-8?B?bmhBYzcwdlpvc00yN24xYUhiblZKai8wTWFHc29vbzZJMVliZCs2NDV1YVZy?=
 =?utf-8?B?TEV0b2RwYkFMRXdnaXNNMjdCRlpOMnI2TzF2OVhQek1sYk02anZ3YnFJam45?=
 =?utf-8?B?dGhLN3RmeFVNeU9OUGVDVkNHVW1KaVNhRkp3d1NqVmh5UEFiYUxDRnIrdWJV?=
 =?utf-8?B?Z29yK2dSN2FRanZUQlkrUW4vYlhNQ2tIbGd0OERRYUQyc29EQXIzdGtCYk1t?=
 =?utf-8?B?TEJmQVlDcDBUaEhQOHMxUVp0YU5KSGpBV1hDdlRBdDVaMnh3cXQ2YjVOdVh5?=
 =?utf-8?B?UC82cGZwMkpQTHFsNnpneVBVSVAxYTU0aEZZY1BoT2IxOWlqK0kyYXRDeURM?=
 =?utf-8?B?UXFWczVHOTNxWUZLVjcyYi84WE11UmhBVzVMUGhJaERqc3NDMG1hY0hpN3lo?=
 =?utf-8?Q?uCM63Zj4+isqCXyaZBPDOyCFsQ0hUG2u?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b4c4b758-48ba-43f9-b67f-08da0c66162a
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4257.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 23 Mar 2022 00:43:04.5547
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: V7+USw/Q/xPItkrSBmCEL94i9l6VbmZ4/TO+HHxGM+HuFDgXzeCTNCICOoyAx37unXVfMJmbpD6B8MhTAw9/qw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL0PR10MB2803
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10294 signatures=694350
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 malwarescore=0 mlxscore=0
 bulkscore=0 phishscore=0 spamscore=0 mlxlogscore=999 suspectscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2202240000
 definitions=main-2203230002
X-Proofpoint-GUID: d_lM3THiw1-1jVzOYXSWs7gCAbeIgrqP
X-Proofpoint-ORIG-GUID: d_lM3THiw1-1jVzOYXSWs7gCAbeIgrqP
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H5,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


On 3/21/22 2:44 PM, J. Bruce Fields wrote:
> On Thu, Mar 17, 2022 at 12:43:42AM -0700, Dai Ngo wrote:
>> Update nfs4_get_vfs_file and nfs4_upgrade_open to handle share
>> reservation conflict with courtesy client.
>>
>> Update nfs4_get_vfs_file and nfs4_upgrade_open to handle share
>> reservation conflict with courtesy client.
>>
>> When we have deny/access conflict we walk the fi_stateids of the
>> file in question, looking for open stateid and check the deny/access
>> of that stateid against the one from the open request. If there is
>> a conflict then we check if the client that owns that stateid is
>> a courtesy client. If it is then we set the client state to
>> CLIENT_EXPIRED and allow the open request to continue. We have
>> to scan all the stateid's of the file since the conflict can be
>> caused by multiple open stateid's.
>>
>> Client with CLIENT_EXPIRED is expired by the laundromat.
>>
>> Signed-off-by: Dai Ngo <dai.ngo@oracle.com>
>> ---
>>   fs/nfsd/nfs4state.c | 99 ++++++++++++++++++++++++++++++++++++++++++++++++-----
>>   1 file changed, 91 insertions(+), 8 deletions(-)
>>
>> diff --git a/fs/nfsd/nfs4state.c b/fs/nfsd/nfs4state.c
>> index f20c75890594..c6b5e05c9c34 100644
>> --- a/fs/nfsd/nfs4state.c
>> +++ b/fs/nfsd/nfs4state.c
>> @@ -4950,9 +4950,75 @@ nfsd4_truncate(struct svc_rqst *rqstp, struct svc_fh *fh,
>>   	return nfsd_setattr(rqstp, fh, &iattr, 0, (time64_t)0);
>>   }
>>   
>> +static bool
>> +nfs4_check_access_deny_bmap(struct nfs4_ol_stateid *stp, u32 access,
>> +			bool share_access)
>> +{
>> +	if (share_access) {
>> +		if (!stp->st_deny_bmap)
>> +			return false;
>> +
>> +		if ((stp->st_deny_bmap & (1 << NFS4_SHARE_DENY_BOTH)) ||
>> +			(access & NFS4_SHARE_ACCESS_READ &&
>> +				stp->st_deny_bmap & (1 << NFS4_SHARE_DENY_READ)) ||
>> +			(access & NFS4_SHARE_ACCESS_WRITE &&
>> +				stp->st_deny_bmap & (1 << NFS4_SHARE_DENY_WRITE))) {
>> +			return true;
>> +		}
>> +		return false;
> Looks like you could do this with just:
>
> 		return share_access & bmap_to_share_mode(stp->st_deny_bmap);

Thanks, this makes it much simpler. Fix in v18.

>
>
>> +	}
>> +	if ((access & NFS4_SHARE_DENY_BOTH) ||
>> +		(access & NFS4_SHARE_DENY_READ &&
>> +			stp->st_access_bmap & (1 << NFS4_SHARE_ACCESS_READ)) ||
>> +		(access & NFS4_SHARE_DENY_WRITE &&
>> +			stp->st_access_bmap & (1 << NFS4_SHARE_ACCESS_WRITE))) {
>> +		return true;
>> +	}
> Likewise.

Fix in v18.

>
> Also, I think it'd be simpler to check for both access and deny
> conflicts here, instead of just one or the other.

I'm not clear here.

I remove nfs4_check_access_deny_bmap, fold this functionality
into nfs4_resolve_deny_conflicts_locked by making use of
bmap_to_share_mode.
  

>
>> +	return false;
>> +}
>> +
>> +/*
>> + * Check whether courtesy clients have conflicting access
>> + *
>> + * access:  is op_share_access if share_access is true.
>> + *	    Check if access mode, op_share_access, would conflict with
>> + *	    the current deny mode of the file 'fp'.
>> + * access:  is op_share_deny if share_access is false.
>> + *	    Check if the deny mode, op_share_deny, would conflict with
>> + *	    current access of the file 'fp'.
>> + * stp:     skip checking this entry.
>> + * new_stp: normal open, not open upgrade.
>> + *
>> + * Function returns:
>> + *	true   - access/deny mode conflict with normal client.
>> + *	false  - no conflict or conflict with courtesy client(s) is resolved.
>> + */
>> +static bool
>> +nfs4_resolve_deny_conflicts_locked(struct nfs4_file *fp, bool new_stp,
>> +		struct nfs4_ol_stateid *stp, u32 access, bool share_access)
>> +{
>> +	struct nfs4_ol_stateid *st;
>> +	struct nfs4_client *clp;
>> +	bool conflict = false;
>> +
>> +	lockdep_assert_held(&fp->fi_lock);
>> +	list_for_each_entry(st, &fp->fi_stateids, st_perfile) {
>> +		if (st->st_openstp || (st == stp && new_stp) ||
> I'd split this into separate if statements and add a comment at least
> for the st_openstp case, which isn't too obvious:
>
> 		if (st->st_openstp) /* ignore lock stateids */
> 			continue;

Fix in v18.

>
>> +			(!nfs4_check_access_deny_bmap(st,
>> +					access, share_access)))
>> +			continue;
>> +		clp = st->st_stid.sc_client;
>> +		if (nfsd4_expire_courtesy_clnt(clp))
>> +			continue;
>> +		conflict = true;
>> +		break;
>> +	}
>> +	return conflict;
>> +}
>> +
>>   static __be32 nfs4_get_vfs_file(struct svc_rqst *rqstp, struct nfs4_file *fp,
>>   		struct svc_fh *cur_fh, struct nfs4_ol_stateid *stp,
>> -		struct nfsd4_open *open)
>> +		struct nfsd4_open *open, bool new_stp)
>>   {
>>   	struct nfsd_file *nf = NULL;
>>   	__be32 status;
>> @@ -4968,15 +5034,29 @@ static __be32 nfs4_get_vfs_file(struct svc_rqst *rqstp, struct nfs4_file *fp,
>>   	 */
>>   	status = nfs4_file_check_deny(fp, open->op_share_deny);
>>   	if (status != nfs_ok) {
>> -		spin_unlock(&fp->fi_lock);
>> -		goto out;
>> +		if (status != nfserr_share_denied) {
>> +			spin_unlock(&fp->fi_lock);
>> +			goto out;
>> +		}
>> +		if (nfs4_resolve_deny_conflicts_locked(fp, new_stp,
>> +				stp, open->op_share_deny, false)) {
>> +			spin_unlock(&fp->fi_lock);
>> +			goto out;
>> +		}
>>   	}
>>   
>>   	/* set access to the file */
>>   	status = nfs4_file_get_access(fp, open->op_share_access);
>>   	if (status != nfs_ok) {
>> -		spin_unlock(&fp->fi_lock);
>> -		goto out;
>> +		if (status != nfserr_share_denied) {
>> +			spin_unlock(&fp->fi_lock);
>> +			goto out;
>> +		}
>> +		if (nfs4_resolve_deny_conflicts_locked(fp, new_stp,
>> +				stp, open->op_share_access, true)) {
>> +			spin_unlock(&fp->fi_lock);
>> +			goto out;
>> +		}
>>   	}
>>   
>>   	/* Set access bits in stateid */
>> @@ -5027,7 +5107,7 @@ nfs4_upgrade_open(struct svc_rqst *rqstp, struct nfs4_file *fp, struct svc_fh *c
>>   	unsigned char old_deny_bmap = stp->st_deny_bmap;
>>   
>>   	if (!test_access(open->op_share_access, stp))
>> -		return nfs4_get_vfs_file(rqstp, fp, cur_fh, stp, open);
>> +		return nfs4_get_vfs_file(rqstp, fp, cur_fh, stp, open, false);
>>   
>>   	/* test and set deny mode */
>>   	spin_lock(&fp->fi_lock);
>> @@ -5036,7 +5116,10 @@ nfs4_upgrade_open(struct svc_rqst *rqstp, struct nfs4_file *fp, struct svc_fh *c
>>   		set_deny(open->op_share_deny, stp);
>>   		fp->fi_share_deny |=
>>   				(open->op_share_deny & NFS4_SHARE_DENY_BOTH);
>> -	}
>> +	} else if (status == nfserr_share_denied &&
>> +		!nfs4_resolve_deny_conflicts_locked(fp, false, stp,
>> +			open->op_share_deny, false))
> Looks to me like these nfs4_resolve_deny_conflicts_locked() calls could
> go into nfs4_file_check_deny and nfs4_file_get_access instead.

Fix in v18.

-Dai

>
> --b.
>
>> +		status = nfs_ok;
>>   	spin_unlock(&fp->fi_lock);
>>   
>>   	if (status != nfs_ok)
>> @@ -5376,7 +5459,7 @@ nfsd4_process_open2(struct svc_rqst *rqstp, struct svc_fh *current_fh, struct nf
>>   			goto out;
>>   		}
>>   	} else {
>> -		status = nfs4_get_vfs_file(rqstp, fp, current_fh, stp, open);
>> +		status = nfs4_get_vfs_file(rqstp, fp, current_fh, stp, open, true);
>>   		if (status) {
>>   			stp->st_stid.sc_type = NFS4_CLOSED_STID;
>>   			release_open_stateid(stp);
>> -- 
>> 2.9.5