Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Subject: Re: [PATCH RFC v5 0/2] nfsd: Initial implementation of NFSv4
 Courteous Server
To:     "J. Bruce Fields" <bfields@fieldses.org>
Cc:     chuck.lever@oracle.com, linux-nfs@vger.kernel.org,
        linux-fsdevel@vger.kernel.org
References: <20210929005641.60861-1-dai.ngo@oracle.com>
 <20211001205327.GN959@fieldses.org>
From:   dai.ngo@oracle.com
Message-ID: <a6c9ba13-43d7-4ea9-e05d-f454c2c9f4c2@oracle.com>
Date:   Fri, 1 Oct 2021 14:41:55 -0700
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:78.0)
 Gecko/20100101 Thunderbird/78.14.0
In-Reply-To: <20211001205327.GN959@fieldses.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Content-Language: en-US
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?WFVnVHlRMTNQVGJHQ29NQTRVcTczSWFXNThsdXBPdTU0UGl3dytaMkdyRE5T?=
 =?utf-8?B?UUFNZ2hEK2hCVWJnMVVrdkEzclBmMUdNeHExbklueEtQZkpCYll1aDZWTWxQ?=
 =?utf-8?B?YlBlbEVabmVPdUR5elhhbzFuZGxnN2M0TGhmZDRQSnI1VDFLdG95N095SENo?=
 =?utf-8?B?TzBwczZ0dmlzQUs3cUpIbGthYktDUGc2eGl5dHBXL3BMTzhkd0lYOFZYT0xy?=
 =?utf-8?B?K29HWHFHN0xXRFpKL3NheHFpbzlaWXZzRGJxUmgyUzNiSzNRREN2ZmUzM0hm?=
 =?utf-8?B?eWYzTmY0U0djVEhoU3FLS20zckZDL2JUYThUanRnQ1JQTEtkdjNZTXZUZDV1?=
 =?utf-8?B?SjhweDJDQXRCOFpCcjlIUHlxUWR6WEFOTFZ3RUpwS2d4aDFQTW1TMC9NN2ZN?=
 =?utf-8?B?OXc0eEtxSTQzL3RWVmhrSE4vaUlsbndIdXNYWU1va0htYkV2RWNqeUxsUG9p?=
 =?utf-8?B?QmZ4M0FqS0xtUWRqWGVERkRycjU5U3hoTk9yZHRqL2FjZTh2OXJnWGdmZS9z?=
 =?utf-8?B?S3JVa0ltaXBwVnF2SkUvMUZLWXFFcVhQNDdCT1ViOHJuK2R1ZklUbDFtLzVy?=
 =?utf-8?B?Wlo2RTBDVXJtRjBBRnhidEFWZytITnVza1ZBTHBRc2NnVXg4Q3prSDBwLzJR?=
 =?utf-8?B?TjRTSkpYQTYwczc4RFRlVllHTW9pTVIzQU1lWEhwR3lnbURucVhSMUs2U1Fw?=
 =?utf-8?B?YXk3K3NYYmRrWENOMTlpRk1mYTRIaDllOFZydjN0MDRheVphWFYrVWcvWDU3?=
 =?utf-8?B?emlXUk9Dd3YzYURpRjByWnR3TkJOVHdPRXg3THh5Uyt1bGVMb3UvMWRBUzRU?=
 =?utf-8?B?dTJSejE5dWtld2Fyb0dRenBQQ0tUb01aQ1BPQ2FuYi9GdWgvRkIrTktETXBY?=
 =?utf-8?B?ZnFtWll3YUVvQytRT0poUDJNYk1GRUF4aTFGdHNVYlY3VVROajRBN3h4U0pm?=
 =?utf-8?B?akF2MW1BVTZIbWdzNk1VbzRCZWc0UzQ4UXE1UEd2MUdoaVlkUm1oaHBNK3Ey?=
 =?utf-8?B?TEZyYlQ4dTRRNzZhNkoyT3dsRVhrNktHYlJJencyUERDa09IZ2tNS0hFYXZu?=
 =?utf-8?B?TmlGNEhrU2JZTnJUN0ZibFZTSU9ZbHVsWnpYOXA2WGFleUVJd3NEVDlxczE5?=
 =?utf-8?B?WUwrdWVhMi9XcVZnZlBqdHExUWNLd3FleGRNRGhYb21EVlh0Z3hvOVh3dm9v?=
 =?utf-8?B?NERJZkdBbXpDWWEvZFNLaHBlOHBTV1ZmTkZsT1ZCM0p0MmJlSFphVnk5Zjdx?=
 =?utf-8?B?dy9wc08yZEZCM3h2dEZmMDRQMXBSMUpXamNTUzQzUXhBTlR1V0RVUWdWY1Vm?=
 =?utf-8?B?K0l0MFV5elFjTU9oRUI3cExpUklLT05uMGVzc1I0Vys2MWRIUFJyOFJ4OTNJ?=
 =?utf-8?B?UVhEYmNhOFZucHpTeVB0ajhYTVVtcHVQbWVnOERkMFVWa2ljWEVsYmQyM2Zi?=
 =?utf-8?B?TVVxanZ5RlBneklQbGdBOE55dmxuSGcrNUEwTzZJRzBaZWErT1lNSzBST0Nx?=
 =?utf-8?B?WTBkQTR0cGtOdlJaNWZJZ1BRUHcyRXQxOHZtcEVkejVTNkVGbDFBalZ4c0di?=
 =?utf-8?B?ei9JcnN0UDVuQlZ1bHUvbmtuRGdlcHpKdEZvbS84K1psVzh0bWFuYWVRMDUz?=
 =?utf-8?B?TXhXTHJjSDl2Ui9DOWU4K0ZoZjI0TTRKNWRmVkgrVXJGTWdwWDcvUzRSSHYw?=
 =?utf-8?B?ZDFMLzYxNDdtVHVnVWJqaGg1SUJ6LzYxT2U3dE1aSm93cVk2TkZ4cHNzTjJH?=
 =?utf-8?Q?20fLxoNP9R+actqFf9Yw//ErWAo0Mqy+h7ld/Qe?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: be610056-c586-45ef-f5a8-08d985244a35
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4257.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Oct 2021 21:41:56.1824
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: P4vwrjR9jNy+xn0NWWTPBwWlc7pCd83wradJkel8GJDUW2gW3dYT/Oy4izHc18WuFXgdDp2Ybgw+OBfvi3UnrA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2440
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10124 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 mlxscore=0 mlxlogscore=999
 phishscore=0 bulkscore=0 suspectscore=0 malwarescore=0 spamscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2109230001
 definitions=main-2110010150
X-Proofpoint-GUID: F5YJ6a6HbqWUCTewuSn-0SgCrdM0Q-iM
X-Proofpoint-ORIG-GUID: F5YJ6a6HbqWUCTewuSn-0SgCrdM0Q-iM
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


On 10/1/21 1:53 PM, J. Bruce Fields wrote:
> On Tue, Sep 28, 2021 at 08:56:39PM -0400, Dai Ngo wrote:
>> Hi Bruce,
>>
>> This series of patches implement the NFSv4 Courteous Server.
> Apologies, I keep meaning to get back to this and haven't yet.
>
> I do notice I'm seeing a timeout on pynfs 4.0 test OPEN18.

It's weird, this test passes on my system:


[root@nfsvmf25 nfs4.0]# ./testserver.py $server --rundeps -v OPEN18
INIT     st_setclientid.testValid                                 : RUNNING
INIT     st_setclientid.testValid                                 : PASS
MKFILE   st_open.testOpen                                         : RUNNING
MKFILE   st_open.testOpen                                         : PASS
OPEN18   st_open.testShareConflict1                               : RUNNING
OPEN18   st_open.testShareConflict1                               : PASS
**************************************************
INIT     st_setclientid.testValid                                 : PASS
OPEN18   st_open.testShareConflict1                               : PASS
MKFILE   st_open.testOpen                                         : PASS
**************************************************
Command line asked for 3 of 673 tests
Of those: 0 Skipped, 0 Failed, 0 Warned, 3 Passed
[root@nfsvmf25 nfs4.0]#

Do you have a network trace?

-Dai

>
> --b.
>
>> A server which does not immediately expunge the state on lease expiration
>> is known as a Courteous Server.  A Courteous Server continues to recognize
>> previously generated state tokens as valid until conflict arises between
>> the expired state and the requests from another client, or the server
>> reboots.
>>
>> The v2 patch includes the following:
>>
>> . add new callback, lm_expire_lock, to lock_manager_operations to
>>    allow the lock manager to take appropriate action with conflict lock.
>>
>> . handle conflicts of NFSv4 locks with NFSv3/NLM and local locks.
>>
>> . expire courtesy client after 24hr if client has not reconnected.
>>
>> . do not allow expired client to become courtesy client if there are
>>    waiters for client's locks.
>>
>> . modify client_info_show to show courtesy client and seconds from
>>    last renew.
>>
>> . fix a problem with NFSv4.1 server where the it keeps returning
>>    SEQ4_STATUS_CB_PATH_DOWN in the successful SEQUENCE reply, after
>>    the courtesy client re-connects, causing the client to keep sending
>>    BCTS requests to server.
>>
>> The v3 patch includes the following:
>>
>> . modified posix_test_lock to check and resolve conflict locks
>>    to handle NLM TEST and NFSv4 LOCKT requests.
>>
>> . separate out fix for back channel stuck in SEQ4_STATUS_CB_PATH_DOWN.
>>
>> The v4 patch includes:
>>
>> . rework nfsd_check_courtesy to avoid dead lock of fl_lock and client_lock
>>    by asking the laudromat thread to destroy the courtesy client.
>>
>> . handle NFSv4 share reservation conflicts with courtesy client. This
>>    includes conflicts between access mode and deny mode and vice versa.
>>
>> . drop the patch for back channel stuck in SEQ4_STATUS_CB_PATH_DOWN.
>>
>> The v5 patch includes:
>>
>> . fix recursive locking of file_rwsem from posix_lock_file.
>>
>> . retest with LOCKDEP enabled.
>>
>> NOTE: I will submit pynfs tests for courteous server including tests
>> for share reservation conflicts in a separate patch.
>>