Message-ID: <bef516d0-19cf-3f30-00cd-8359daeff6ab@oracle.com>
Date:   Sun, 21 Nov 2021 19:04:32 -0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:91.0)
 Gecko/20100101 Thunderbird/91.3.2
Subject: Re: [PATCH RFC v5 0/2] nfsd: Initial implementation of NFSv4
 Courteous Server
Content-Language: en-US
To:     "J. Bruce Fields" <bfields@fieldses.org>
Cc:     chuck.lever@oracle.com, linux-nfs@vger.kernel.org,
        linux-fsdevel@vger.kernel.org
References: <20210929005641.60861-1-dai.ngo@oracle.com>
 <20211001205327.GN959@fieldses.org>
 <a6c9ba13-43d7-4ea9-e05d-f454c2c9f4c2@oracle.com>
 <33c8ea5a-4187-a9fa-d507-a2dcec06416c@oracle.com>
 <20211117141433.GB24762@fieldses.org>
 <400143c8-c12a-6224-1b36-3e19f20a7ee4@oracle.com>
 <908ded64-6412-66d3-6ad5-429700610660@oracle.com>
 <20211118003454.GA29787@fieldses.org>
From:   dai.ngo@oracle.com
In-Reply-To: <20211118003454.GA29787@fieldses.org>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?Vjhyb3JyTC8zZUFWbytWZTg4cWs0cWRYVHl4M0FPdUVXV3k0VE1JamN3RGJK?=
 =?utf-8?B?RVVSaWxyaWJROHg5NDlmRHRnQ1E0WnF3SjNIUDZKNUkzNDdoZExTdXZTZGY5?=
 =?utf-8?B?Rlo1MlpORHJWelkrall6V0hIOUxIdVgvU21IWlg4MDl0QzU0TFU1WTd0MUFw?=
 =?utf-8?B?akIxY29CbXdBdCtVZjNhL2NKS1g0a2NpWDRlcHg2Y0tYdHI1R1A4ZnZVTUJV?=
 =?utf-8?B?akU3Ym5OazBVUzNKUXN6K0VpRjZBYUdLMTZ1d0dXcXhLVmE4S1dlK0syNlNN?=
 =?utf-8?B?bnAvaHRZeXNzVjJkY1JXdXIxOE5iSWVFMDAzZFl3OEJwWTVsaW5KZTNSMlN1?=
 =?utf-8?B?aGJXNC80eGJPVGlJKy9YaWl5MjJrZ0NtQ1FkcjFuRURtOUlhYk9UN2ZaSTFU?=
 =?utf-8?B?aUxMTlpMZStLbkpqR0hTc05WalZrUS9ra21ENDRTb3VLQW5zdzFQZktNSWlU?=
 =?utf-8?B?bTFvU2hERnhKQ2gxeUtGWWgyZGxVaVJ1YVlqUWlTOHExTllnNVFpOWNvTEZs?=
 =?utf-8?B?S1Fra0dHNjJrbytJTHFpaVlkRDNHN0VDZ0NaRHErSHFhN1MxVGpxMGZjOUlv?=
 =?utf-8?B?TzVBbG8yUyt3eE9aM2pHTSsrZHJSNnA5WmZmQTNCS2hxKzBLUWwyY2RIWHFG?=
 =?utf-8?B?blArS05nRjNwQ2UrWGVRdTBmTmkyMlN3Z2toQVA3STdjNHhnd3BWU3NRSmRz?=
 =?utf-8?B?OGVjcU5tUTlLMCs4Q3Z5MUVFcldCcnRTbE45TVRiaTVWekRDMkJTNGpVdWRT?=
 =?utf-8?B?VmFiWktNU0V2c1E3MjdJdlU2bndmUHhCblVESWFOTVZmVVFXYm53d0IzaWpC?=
 =?utf-8?B?azgxRVVCcWs0U1k1Z2hFU0M2c3lYRjlhbUdQWnlJcFhrQWRweU1iSm5MZFZa?=
 =?utf-8?B?VHpkOElNWTN6Y29QUFNFSlFnQXpaZG0yRW10NWk2bENCcFJxUXpIcEFYK243?=
 =?utf-8?B?MUJ1QjMzNlRDR0diWFlCcDl2Tlo3MDVWcTNvM3VoZ1FjK0ZyV1BvejRoNEc5?=
 =?utf-8?B?U1ZPSzhkSHdyTy8wUnZzSGFMWUZlSWlOZjZFb0ZmOWtGcEZQK0E5MHRlQ3Zi?=
 =?utf-8?B?dlZ0b3pqdmwwTFhhaERNcUZZMEVCd045bVdpREN3Q1NhZzJTYy92Z2x4WXpv?=
 =?utf-8?B?K253eWsrUVJjYVJ5YW9BVEFUSjBDT2FSVXp0dFFWNGpLZFVhNGRFVmJWdUhy?=
 =?utf-8?B?Mm9jMWR1MGZtWFliTGpMbWdJNFpCOFN3TjRhYXN2WEZXWGQ0NTFxMEpCQlV3?=
 =?utf-8?B?aUdGRDR3UlgvVFVWS2lXWjF2WkU1M3dQNmViT0VkbStycjZlM0lqVTFiUjdL?=
 =?utf-8?B?ZmEwTkNWNzg5Sktoci9qS2ZMZ2ZTaVk4YnZvYVJ2THVXUzlERTRZRE92Ni9i?=
 =?utf-8?B?eUxCb05TN3RobVBGa2EvRVlWMkJLREVCZXJRODY1OS9DY01NTGo2M3BZMmlR?=
 =?utf-8?B?dlI2L28vS1RzMmhIY2dqbnFNUTdBeHIyUWloU0FiVjdmNlY5Q0F6TWxScStX?=
 =?utf-8?B?Q2pJRnRTNHd1ZXowMFNsd0lQL05GV1FHQmdqenJFcVdnS2QzUmc4ZUJjbUpV?=
 =?utf-8?B?Y09aMWxTQ1g4cnhXRkpzNGFNYU1odW5PWElQT1lKSUZMa1hCTnhJU002b2Mr?=
 =?utf-8?B?ZWE5VitNS0ZuVWhSdnpJTVM4QWVtZGFBMk1IVGdmdE1UOGpyc3hQNXJKS2Yy?=
 =?utf-8?B?eDkwTzY0b25iRDVpS0RiWGFFWkZ6c1NxRFM3RXp6Q1R6c0FwVDdsU2N3V0o0?=
 =?utf-8?B?dmhrRXEyOU5Vd2V2ajA5VlNvYmR5VnVmSEl6YndHTDRadFowNzU2ZFlQQzND?=
 =?utf-8?B?UVFYNXhzWkZITEx0bno0RTB0TVV0TjdhVlpPZ2JrR3k5Vlh6NjVwd3Fibmpy?=
 =?utf-8?B?NXpVQWtUU0JmWWZlaWN3Skk5ZVU3b1lzZkFaV2l1TXFYaUZFOVRyZ3JadlFK?=
 =?utf-8?B?c1VkNklDTHFBS0RKaWt4aXgvczNvQ3E4T3hoK2djOWxUV0d3SHl1aVlHdXlj?=
 =?utf-8?B?UVdYSTlmQ3VZQWhITWZ2TmdxWjdlaHdHY2xhcVdIeXVIM2Zpa0tJZXBMNkk0?=
 =?utf-8?B?ZTdWUldZemQyS3NzeFd1dWFBNkZvc09ldEloYkhqV1UwSEI3bGlqNTVWMWEz?=
 =?utf-8?B?ZzNXNlRadktLQVRKS0ZIZ2RmL1ZqSVdUQUQyZFRnY3lZM3hDcWtTT2RIcEpk?=
 =?utf-8?Q?CXdy9Ec2vk4KpEQQ5tP3+40=3D?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 8fdf32f0-f8ba-458c-ea39-08d9ad64d05c
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4257.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Nov 2021 03:04:35.5015
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: AFEmfaHYtecZxAp86OfJtNPvFbmkpRgVRo//rQvZ5lilBF0kFN86DF82HZNfUHrIumbpAc8Q/2ey2Wsk7SkYDA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BYAPR10MB2566
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10175 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 adultscore=0 phishscore=0
 bulkscore=0 suspectscore=0 malwarescore=0 mlxscore=0 mlxlogscore=999
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000
 definitions=main-2111220015
X-Proofpoint-GUID: rHDQawj6xfYqb4gZn8EIpFpy5ov4lBBD
X-Proofpoint-ORIG-GUID: rHDQawj6xfYqb4gZn8EIpFpy5ov4lBBD
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


On 11/17/21 4:34 PM, J. Bruce Fields wrote:
> On Wed, Nov 17, 2021 at 01:46:02PM -0800, dai.ngo@oracle.com wrote:
>> On 11/17/21 9:59 AM, dai.ngo@oracle.com wrote:
>>> On 11/17/21 6:14 AM, J. Bruce Fields wrote:
>>>> On Tue, Nov 16, 2021 at 03:06:32PM -0800, dai.ngo@oracle.com wrote:
>>>>> Just a reminder that this patch is still waiting for your review.
>>>> Yeah, I was procrastinating and hoping yo'ud figure out the pynfs
>>>> failure for me....
>>> Last time I ran 4.0 OPEN18 test by itself and it passed. I will run
>>> all OPEN tests together with 5.15-rc7 to see if the problem you've
>>> seen still there.
>> I ran all tests in nfsv4.1 and nfsv4.0 with courteous and non-courteous
>> 5.15-rc7 server.
>>
>> Nfs4.1 results are the same for both courteous and non-courteous server:
>>> Of those: 0 Skipped, 0 Failed, 0 Warned, 169 Passed
>> Results of nfs4.0 with non-courteous server:
>>> Of those: 8 Skipped, 1 Failed, 0 Warned, 577 Passed
>> test failed: LOCK24
>>
>> Results of nfs4.0 with courteous server:
>>> Of those: 8 Skipped, 3 Failed, 0 Warned, 575 Passed
>> tests failed: LOCK24, OPEN18, OPEN30
>>
>> OPEN18 and OPEN30 test pass if each is run by itself.
> Could well be a bug in the tests, I don't know.

The reason OPEN18 failed was because the test timed out waiting for
the reply of an OPEN call. The RPC connection used for the test was
configured with 15 secs timeout. Note that OPEN18 only fails when
the tests were run with 'all' option, this test passes if it's run
by itself.

With courteous server, by the time OPEN18 runs, there are about 1026
courtesy 4.0 clients on the server and all of these clients have opened
the same file X with WRITE access. These clients were created by the
previous tests. After each test completed, since 4.0 does not have
session, the client states are not cleaned up immediately on the
server and are allowed to become courtesy clients.

When OPEN18 runs (about 20 minutes after the 1st test started), it
sends OPEN of file X with OPEN4_SHARE_DENY_WRITE which causes the
server to check for conflicts with courtesy clients. The loop that
checks 1026 courtesy clients for share/access conflict took less
than 1 sec. But it took about 55 secs, on my VM, for the server
to expire all 1026 courtesy clients.

I modified pynfs to configure the 4.0 RPC connection with 60 seconds
timeout and OPEN18 now consistently passed. The 4.0 test results are
now the same for courteous and non-courteous server:

8 Skipped, 1 Failed, 0 Warned, 577 Passed

Note that 4.1 tests do not suffer this timeout problem because the
4.1 clients and sessions are destroyed after each test completes.


-Dai

>> I will look into this problem.
> Thanks!
>
> --b.