Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <7834d4f4-47a1-f503-902f-fe33f35a10c6@oracle.com>
Date:   Thu, 17 Nov 2022 10:43:35 -0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
 Gecko/20100101 Thunderbird/102.4.2
Subject: Re: [PATCH v5 0/4] add support for CB_RECALL_ANY and the delegation
 reaper
Content-Language: en-US
To:     Chuck Lever III <chuck.lever@oracle.com>
Cc:     "jlayton@kernel.org" <jlayton@kernel.org>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
References: <1668656688-22507-1-git-send-email-dai.ngo@oracle.com>
 <8517F18C-9207-4DE4-A217-2A0EA4C4484C@oracle.com>
 <7ee9c36b-b3d8-bb84-36bb-393eaa2369f2@oracle.com>
 <A5796183-BE7E-4B05-A620-4BC76A2608D1@oracle.com>
From:   dai.ngo@oracle.com
In-Reply-To: <A5796183-BE7E-4B05-A620-4BC76A2608D1@oracle.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UEk1YWZ1K2ZQbmtGbVRrWUdWM3lyaDBCNHdlYjgyMVppMk9VZUgvM2xSUzZp?=
 =?utf-8?B?Q1BHMFYvVTVpbzRaL3lEbUx6RWJrdjNLWEdtZ0VwRXBXM2djMktSTXV5SzZ6?=
 =?utf-8?B?V0N4Y0NaSDdKS29Zb2YrY0FCZ3B5dTVyV1h1dG5sN1NYVmczUjNJQVBSWlRx?=
 =?utf-8?B?enlXNGJTUnFMTjEzRUtlaFhkS2loVTMra0ExeXBJUmtZMTA1Yy9ldDFsSE0w?=
 =?utf-8?B?ekpBVkVjc3FyUFoyV3dyQkIyMTRyRVdHMGxkQkk0eE53UTZpN3RCMFcrNGNy?=
 =?utf-8?B?YzVDeHZ6aWZJYkRYY2FjK1oxVUhScDNiLzd5QjZ0OWJQTjJubENsM00rTmIv?=
 =?utf-8?B?Nm16NEcvSDNaRnUxUnJwWlA2T0hlWWgzWVlBM3Y5dTlyQnhzTDlmd2FFTDFU?=
 =?utf-8?B?Mkh5SUYvVmpWSnJsSWozQVRvZ3liSW81cG4zYURuL2hyTjFvV092SzhLTnVR?=
 =?utf-8?B?Z2QxZHBRZFVBMjV6NytWakFacDdvTEJ0OEcvNEhqajhQNjNGT2RIV01YZ1hN?=
 =?utf-8?B?K25rS2dMZUFlcDlOZ3pEM2ZhaWhkMkhQSnN3ak1jNTZnaGlpejhka085WnBO?=
 =?utf-8?B?dFVZNytkUVJ1U3RjaXpVWVdubjNEQkxiMWlTdm5sZlU4S2dRRmNkZ2Rtbk96?=
 =?utf-8?B?OUhKQlhaWXJxc2laeVo1YlV1SEFxcnlWb1RuekJXWlVRaGszV1Q4U1pQdEFj?=
 =?utf-8?B?eS9nYWs2TnpkVWIySVR2dkZUa2hzRFZNUzVIU2o2SldrSDNHOXR2eWRBdVBa?=
 =?utf-8?B?dXNIdGpsdHNURUJBd0tOOU9Sb0ZkWHF2SnRWcUpOK29uaW54aUlzdWQ4Rlpl?=
 =?utf-8?B?djhrbWR2eUdVdU90RFVZUWI1Sk1pZWJ5UllETDcrSjJvbzFkVGVJYXdBdHJK?=
 =?utf-8?B?NlRLc0tYY2REMXRkbDVYY1FGMzNmaXpicUNOdUR2Q0FaN1l6TzlacE5kR0F5?=
 =?utf-8?B?NjZOV010a0xhempQb09XdUlrRkxYRy9wbjgrS3NRSEN4OWltUTJEdEVmQW1I?=
 =?utf-8?B?U0g4RkpVamoydFQzUDRGN0k4dzR2T0dsVWh0dHB5WnFIcFJMdmI3Tkl5NUtl?=
 =?utf-8?B?aHNRTWU2eDJPSGJ6c2JxT01reGM2ekZZRXA1RnFZOWRYbGZLOThsUWhUMmFW?=
 =?utf-8?B?V3U0QStpVjNxc1Q4VVFmR0QvMWJPRzQrSEwzUVQ5N1pvRXBaZzgrWFFrbFJi?=
 =?utf-8?B?aVFMMjZZSm1wS3RKUXZrck1MR3d0ZjdzREg4NjJTbS9QUHI3RVRUbzZKblA1?=
 =?utf-8?B?ckJvWGFzeDBaWFNyckRCK0c4YVFzRUhiam1wTHFnek1JbnN3ckhVK25VeVE1?=
 =?utf-8?B?ZUZmRk1yV29PYkEweWZWMlRCTUhkK2V4SGlCSUJQOE5DcnlxeGRmMDQ1ZEdC?=
 =?utf-8?B?aEJ5cFgwbGFaSnlLeE03bEd3OEJTc2lSdXFCek03cDFpUUowTUZhRFYwWWdO?=
 =?utf-8?B?NGdkUHVYT3k1Z1dTVkJuN216OG84WjhXZ0RNVUJ6WjBaR0FBVGkvRW9nWGdT?=
 =?utf-8?B?bGs4Q0w2R1pGQjgwWFJGdDFDdlBVSTFsUWVsbHB3WHJ4UkJHVWVNUXQzcGpL?=
 =?utf-8?B?Uml3RXdydGhFbFFTOWZiSktwSVZFWFAzUXNyeDF2NlozbFJuUWNRYkJMQm5t?=
 =?utf-8?B?elhqRzYwamJqZll3dWNURllzYWJxVzI3UDBBLzFPQWd4eTZITm5tTi9OSHY2?=
 =?utf-8?B?MUpRMDR1WDRuZnFVMDljQzY5NUgxNHRxZ28wczUvd09IcVJFT3MrTzVoME11?=
 =?utf-8?B?aUNVQTNxVWZ0M0JIdFdBWWFHMW4zWk5ZNG1ZLytncnE5Qk9lV1lzN3Y4ZitL?=
 =?utf-8?B?RHNrYTQzRlhZVmErditZbXo2dCsvN1drODFWcEpmYjBpcjIvakhvUGt6K2ov?=
 =?utf-8?B?NEx1NmZINE9uY1JyN3Y5SU9hQVNKUzQ4S3I2eGkyZ29UdFh6eE00cTlJU1lQ?=
 =?utf-8?B?OUVkWm5aUUhtUzNsQlpKT3craTdoY3ZlQ1VLSW5tM0RjSDBvUzRlZEpwb0hi?=
 =?utf-8?B?aTZTaGwzNTh5ZENuTjQvbXZVTG0xazRUUWpoeTBEcjNlcGJqenN0ZGFPWSty?=
 =?utf-8?B?R0UvL1ZxVFhzZkl1akI0aUQwN1J3dlBBVjFvZHIxMjVyMU9DV1JQMFNBZXBn?=
 =?utf-8?Q?FWMEZn0t6JJrWLw8/o99CTyEz?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: juWeJ/Pc4julei1bc3RPRRQrIaWdqOPgIONk8p+IODrn1jq7bI0AHxXUGG52uZEDddRwSP5/SFw2al2cbGrg3MDoKEs7umFdGLQdr9+kGBRrj+HV+hiZmSxL29oFqBbSPVxcqIzuoYsmTIk/9siC8a1xXmammpVNXw8Oz47op6WIfetROB/pqsh0BlAKq2Eudig3JTVtX7WzhrhkswbTo+t6TbLbaUGYWt5KRbm5szAz2y8/L8DWK6VU+9FBo+DE1fRSt/7Lk/pBGuE3B91IOfE32YR2GhKzpUEvj3X0Jsq6NrVRI8aFBKrXe+LeytZZLGI8dze0rEd88gdnNvfNHHTUn+RHvLoFtwzghEGZec4Ku0XxrGl916In55zVmiGk6Ut1HkgUvOWLhlcG8kj0xaONEhuhu8PaaN+JtcpERkttkfxXQzd/PZsn/L+AHF0H5fg04OJBJUnUicEdNXReKgQYpvQy/dC5QVmYdZUtf+EJ98IN8fvBQfSO3NOb5859W1/7uKXbfZP1QVRfksLpGF/UpcwWR8aLnPmIfy3qQKdyupp/K5MFyv23B3dPhwYTyL6ovBp45/tGDaa+EHw++ehukuEZaVNJCAhUfUHobgXphf1AMu/rDieX0+hu+TPyOgN9hVffRcJyDRSHGzWRIKDkmUzhZO5cKIsHFNqOYNyDha8yfth/nt6Fl/0IFr1azsi3D45hMYmN4KTkQeMbtXUZjeWf70dK3SYO2tOPgDVIbsIAeuHrs24v/Ngbv8wdQ4lPt89pNp4JLNwnvVT4eVMqGdWwlF9Ejwbatf/wH+g=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e408dafe-d5b5-4c88-fd35-08dac8cba410
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4257.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Nov 2022 18:43:38.4714
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: WuYhoHk9YvQ5ZxjOo0ISVlLpOUvu+FYaPTdYuzx5A46Zdx7ZbnjDFswcVgBcB7LxYUivpx/lrCo26bab84j7XA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BLAPR10MB5233
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1
 definitions=2022-11-17_06,2022-11-17_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 adultscore=0 phishscore=0
 malwarescore=0 bulkscore=0 spamscore=0 mlxscore=0 mlxlogscore=999
 suspectscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2210170000 definitions=main-2211170135
X-Proofpoint-GUID: HJ-CEL3xg7fvkQ-39vRJKrxjYINah0mo
X-Proofpoint-ORIG-GUID: HJ-CEL3xg7fvkQ-39vRJKrxjYINah0mo
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


On 11/17/22 9:04 AM, Chuck Lever III wrote:
>
>> On Nov 17, 2022, at 11:53 AM, Dai Ngo <dai.ngo@oracle.com> wrote:
>>
>>
>> On 11/17/22 6:44 AM, Chuck Lever III wrote:
>>>> On Nov 16, 2022, at 10:44 PM, Dai Ngo <dai.ngo@oracle.com> wrote:
>>>>
>>>> This patch series adds:
>>>>
>>>>     . refactor courtesy_client_reaper to a generic low memory
>>>>       shrinker.
>>>>
>>>>     . XDR encode and decode function for CB_RECALL_ANY op.
>>>>
>>>>     . the delegation reaper that sends the advisory CB_RECALL_ANY
>>>>       to the clients to release unused delegations.
>>>>       There is only one nfsd4_callback added for each nfs4_cleint.
>>>>       Access to it must be serialized via the client flag
>>>>       NFSD4_CLIENT_CB_RECALL_ANY.
>>>>
>>>>     . Add CB_RECALL_ANY tracepoints.
>>>>
>>>> v2:
>>>>     . modify deleg_reaper to check and send CB_RECALL_ANY to client
>>>>       only once per 5 secs.
>>>> v3:
>>>>     . modify nfsd4_cb_recall_any_release to use nn->client_lock to
>>>>       protect cl_recall_any_busy and call put_client_renew_locked
>>>>       to decrement cl_rpc_users.
>>>>
>>>> v4:
>>>>     . move changes in nfs4state.c from patch (1/2) to patch(2/2).
>>>>     . use xdr_stream_encode_u32 and xdr_stream_encode_uint32_array
>>>>       to encode CB_RECALL_ANY arguments.
>>>>     . add struct nfsd4_cb_recall_any with embedded nfs4_callback
>>>>       and params for CB_RECALL_ANY op.
>>>>     . replace cl_recall_any_busy boolean with client flag
>>>>       NFSD4_CLIENT_CB_RECALL_ANY
>>>>     . add tracepoints for CB_RECALL_ANY
>>>>
>>>> v5:
>>>>     . refactor courtesy_client_reaper to a generic low memory
>>>>       shrinker
>>>>     . merge courtesy client shrinker and delegtion shrinker into
>>>>       one.
>>>>     . reposition nfsd_cb_recall_any and nfsd_cb_recall_any_done
>>>>       in nfsd/trace.h
>>>>     . use __get_sockaddr to display server IP address in
>>>>       tracepoints.
>>>>     . modify encode_cb_recallany4args to replace sizeof with
>>>>       ARRAY_SIZE.
>>> Hi-
>>>
>>> I'm going to apply this version of the series with some minor
>>> changes. I'll reply to the individual patches where we can
>>> discuss those.
>> Thank you Chuck!
> Changes folded in and pushed to nfsd's for-next. For the trace
> point patch, I've rebased your series on top of the patch that
> relocates include/trace/events/nfs.h so that the new
> show_rca_mask() macro can go in the common nfs.h instead of
> the NFSD-specific fs/nfsd/trace.h.
>
> Feel free to pull and test to make sure I didn't do anything
> bone-headed.

I removed the get_sockaddr temporarily to test show_rca_mask.
It works fine:

[root@nfsvmf24 ~]# trace-cmd report
trace-cmd: No such file or directory
   Error: expected type 4 but read 5
cpus=1
     kworker/u2:6-2297  [000]  1349.863391: nfsd_cb_recall_any:   client 63767ac9:adb1a3fb keep=0 bmval0=RDATA_DLG
     kworker/u2:0-8698  [000]  1349.869652: nfsd_cb_recall_any_done: client 63767ac9:adb1a3fb status=0

Thanks,
-Dai

>
>
> --
> Chuck Lever
>
>
>