Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Chuck Lever III <chuck.lever@oracle.com>
To:     Dai Ngo <dai.ngo@oracle.com>
CC:     "jlayton@kernel.org" <jlayton@kernel.org>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: [PATCH v5 0/4] add support for CB_RECALL_ANY and the delegation
 reaper
Thread-Topic: [PATCH v5 0/4] add support for CB_RECALL_ANY and the delegation
 reaper
Thread-Index: AQHY+jb3b6PpTwE2G0+HfmxJhBZEiK5DMdoAgAAj2wCAAAMyAIAAG6qAgAAAUAA=
Date:   Thu, 17 Nov 2022 18:44:44 +0000
Message-ID: <E38D4674-543E-4117-B445-6094CCEA90EB@oracle.com>
References: <1668656688-22507-1-git-send-email-dai.ngo@oracle.com>
 <8517F18C-9207-4DE4-A217-2A0EA4C4484C@oracle.com>
 <7ee9c36b-b3d8-bb84-36bb-393eaa2369f2@oracle.com>
 <A5796183-BE7E-4B05-A620-4BC76A2608D1@oracle.com>
 <7834d4f4-47a1-f503-902f-fe33f35a10c6@oracle.com>
In-Reply-To: <7834d4f4-47a1-f503-902f-fe33f35a10c6@oracle.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?YTPtveDqIcKG24oaJvKBomWYGBXxFD2oUwxfN/3CY8C7zfARLBAMf38Rnb52?=
 =?us-ascii?Q?g6HKTkaIVwCVXUg91k+CqgNjfCvtJzyl7prvw82Y3VLl11isNaD15dfiIjbW?=
 =?us-ascii?Q?vqG5sU59YwSLdDV43gE0L9vC9mSrXqj8Og4nhIsZf4gC2yTfSNLC8tymBhK1?=
 =?us-ascii?Q?qwFJzgDW+Juhmq+anjV3B6ThMpwrOP6T6DXZFMm4+WUSxymld5oFNsqfIJtf?=
 =?us-ascii?Q?3ILCi/nY84SXS/xdrH3Yk8kXW6a1+KQFHXOQGStIhCYrneRU9xd4x/2hKIFu?=
 =?us-ascii?Q?sWt/yuajxZ81dWt7SjqOz4xHJsr9gdzTPb/pVXKjsUbgjb+e+E4SaADIgUoR?=
 =?us-ascii?Q?RDv7Gg7FkHg7sJakO9DPRhmW1EtPawe4oV1ylUW83vg8BfFkRpEIgMy2uJWV?=
 =?us-ascii?Q?WYQyV/mld1VgecBhGRxeS7KCL96lrJ+VQvlBzMPuITE/jQ7i/3prhe9m5NoE?=
 =?us-ascii?Q?EiGqh17dFYe/3nCKbv431u8ggu1ZGblkqDkwE6Erz5HifHEL/XiNwAMkitNn?=
 =?us-ascii?Q?wdGqKAFtvK8xmPnrIwnFg8zcftBl1aTiXpN4pVyK/GgrAzG0vA6uDbOGDmiu?=
 =?us-ascii?Q?UsgsRm2aTdQz2KrTO1mHS/lrEopIah1Z0OcPFzTAcK5ed7zMv1z5E+4v56/6?=
 =?us-ascii?Q?t4F3qcdmip8qrjNwuD2/iXd4X5OQx6hK+C7NCsBJ2rzRQxuv3K8vXkfS9/UW?=
 =?us-ascii?Q?MVY9Taez9R5Yf4+7SOAbxR/zDKg9KoNrsAOVvplveJUtxXOG/a3wQarVFZxr?=
 =?us-ascii?Q?qcC/4HUrQjj37DEXxTv2DnG6LQZGOEc7WkNEsUIM1NH7SQZGAa+HCqyrE+AQ?=
 =?us-ascii?Q?i3oD+MI4DzRMyoDegOGuc88QpMBqW+BRXZMLtkHfor1Mbteh5KxzTsoUpdSO?=
 =?us-ascii?Q?07hRt7GvSb6JVTB1cJL87CFofaWUVYn3J+1lkGjBqro1/JII/d4lu9W74xES?=
 =?us-ascii?Q?SV6VKfGaTgHyB5sUnlDXWHYsa0DANtSUeMMZczq267OkGkhjoNYRMT28x/fD?=
 =?us-ascii?Q?Rq0M9tFxc+8obYZOaNl1sYSWduEDNY7OeoBl6V5T5denrcy0OD/IFHrtNI1Z?=
 =?us-ascii?Q?DqUeP4zPQpMADXaCZ39rg2eSnKzsG5AiqhQLLu2dwkT2chHrL0r+ilXwRQsJ?=
 =?us-ascii?Q?NfqSv9iRyLcyBeLBCY1rvgigbxhpJMQ8fx0aveZL6LJ9x6vPGbM0Gc0KJ43E?=
 =?us-ascii?Q?+qzHQHEnL7S+In/4AiH56t5n3k++GGz2aJ0sa5zSL8eFNsZKvB3Cj8Yx+kym?=
 =?us-ascii?Q?vJglKwiZrK345X4dhY4rviLX4o2r6vdWqj/yURWvV/rmls+7XAaF0xdzvAeC?=
 =?us-ascii?Q?53vIPRGQqOB5cL5SWNJFATJx9j9Pi1y/A9TXhvh3RPHEJglCuTmoZyDEeeBC?=
 =?us-ascii?Q?IYy9I461830NkDAta3KO/JP0iL/PiTo37to5FZuTc7AfkpN5mp5+o4JSexKr?=
 =?us-ascii?Q?3nOfSfziYUQmgmxlO2f/sJp+vdp8Kr4p3vi07AfHCv5c54k+JMIIuAixyH/7?=
 =?us-ascii?Q?B13ZNTSkPX6cRNAaXFEfv4DgSHS4wZhGH7nZ3/jivEwMttMCd0QKdNgqIWYE?=
 =?us-ascii?Q?xwh7jbPbOxJ51t24LiEGrPA7HuX2X0y5tQPc+bXPWVtJnu7eb88yPHuUINts?=
 =?us-ascii?Q?LQ=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <ADB1723026605C4A8372B02BD5E338FE@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: hLNSAYjvZWyuyxc7819GexNAliTm3t5PXEAlQJIZhsrUZCeR66P/YabrpJY8Bg3lIlHnc5pUOAlsJfwjBGCfiMV2gCMd+X1prTiYNOwY+nGHn/0UVsifgEihzbYG3/NjKpAdKgVbwJtxTpZwBfFuZumJB7Q355Nvp4sZq0Sk71iBq3vUEx1NPglfj8+bXMhoSIL/ieXC4C3xVEHJ2eCIPYX+vuDmCkYqvZIfZCeCmt9Ek4VH5zHY/cI4rTDEVOH61mvwUusXw46LSvQKZzcRfFCW3/EREEl/19xJzQjqeagcxmfNZ6JhM3vcVpNP6xBenT87OEIrP/OtySyilTjnv+fyVPJaXJtVdG1kjcWyNpBMiC+9MmatRXjPnU8WTSyIer6/9yaLiHm920B1QVAPyTWRlktJOOv7LSWPjeV56zVjq8JevIglQU25QNirzBesy4kE0Nl13PHzeb+Ymfz1QJ5m1/0NSQRRJ4Psyfp89JJyFEsAPr1su+Je5xwMMOVbnZNG4VHs36Nby36AxocnnXfeGYLPv1QJt07HM1ku/T0kUZT6FaMVPZQ+E3VzfyjAGpT0iU1cnrxI+zJiNY6pAtcy8a6523EWEKk2BkiQmQNs4IAUF/VQ03eKSLEx3xDMvYkmqFv4rQAr28hOcBznuZgSvXCfD3+0kO1JXfYXD1WcPMYqsLXm2cJeuEIVraGCN/+1B1YzDfhRBhCrFYSJ6+hialEPQnthDC+BK2UBZXPvONzOvU3eec4DUCZPY7BBZD/z3SRrM+VU6nf84vdROx6/7b8maCN00xjrq4cIW+Y=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN0PR10MB5128.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: bb982ee6-a069-4cc1-5c58-08dac8cbcb57
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 Nov 2022 18:44:44.1450
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: OZVGGJsinqnRT+tqPlFFT+dfZhEqEQEFeyHO/QA5dT/vJfACblqMv6uk4NDQk40Uzrv5TW4A2IuwepL0cYotKA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR10MB6281
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1
 definitions=2022-11-17_06,2022-11-17_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 phishscore=0 bulkscore=0
 mlxlogscore=999 spamscore=0 adultscore=0 malwarescore=0 mlxscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000
 definitions=main-2211170135
X-Proofpoint-ORIG-GUID: xL8rJ39g-VeIfkjtyPbn4PFIqwa7L6yV
X-Proofpoint-GUID: xL8rJ39g-VeIfkjtyPbn4PFIqwa7L6yV
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


> On Nov 17, 2022, at 1:43 PM, Dai Ngo <dai.ngo@oracle.com> wrote:
>=20
>=20
> On 11/17/22 9:04 AM, Chuck Lever III wrote:
>>=20
>>> On Nov 17, 2022, at 11:53 AM, Dai Ngo <dai.ngo@oracle.com> wrote:
>>>=20
>>>=20
>>> On 11/17/22 6:44 AM, Chuck Lever III wrote:
>>>>> On Nov 16, 2022, at 10:44 PM, Dai Ngo <dai.ngo@oracle.com> wrote:
>>>>>=20
>>>>> This patch series adds:
>>>>>=20
>>>>>    . refactor courtesy_client_reaper to a generic low memory
>>>>>      shrinker.
>>>>>=20
>>>>>    . XDR encode and decode function for CB_RECALL_ANY op.
>>>>>=20
>>>>>    . the delegation reaper that sends the advisory CB_RECALL_ANY
>>>>>      to the clients to release unused delegations.
>>>>>      There is only one nfsd4_callback added for each nfs4_cleint.
>>>>>      Access to it must be serialized via the client flag
>>>>>      NFSD4_CLIENT_CB_RECALL_ANY.
>>>>>=20
>>>>>    . Add CB_RECALL_ANY tracepoints.
>>>>>=20
>>>>> v2:
>>>>>    . modify deleg_reaper to check and send CB_RECALL_ANY to client
>>>>>      only once per 5 secs.
>>>>> v3:
>>>>>    . modify nfsd4_cb_recall_any_release to use nn->client_lock to
>>>>>      protect cl_recall_any_busy and call put_client_renew_locked
>>>>>      to decrement cl_rpc_users.
>>>>>=20
>>>>> v4:
>>>>>    . move changes in nfs4state.c from patch (1/2) to patch(2/2).
>>>>>    . use xdr_stream_encode_u32 and xdr_stream_encode_uint32_array
>>>>>      to encode CB_RECALL_ANY arguments.
>>>>>    . add struct nfsd4_cb_recall_any with embedded nfs4_callback
>>>>>      and params for CB_RECALL_ANY op.
>>>>>    . replace cl_recall_any_busy boolean with client flag
>>>>>      NFSD4_CLIENT_CB_RECALL_ANY
>>>>>    . add tracepoints for CB_RECALL_ANY
>>>>>=20
>>>>> v5:
>>>>>    . refactor courtesy_client_reaper to a generic low memory
>>>>>      shrinker
>>>>>    . merge courtesy client shrinker and delegtion shrinker into
>>>>>      one.
>>>>>    . reposition nfsd_cb_recall_any and nfsd_cb_recall_any_done
>>>>>      in nfsd/trace.h
>>>>>    . use __get_sockaddr to display server IP address in
>>>>>      tracepoints.
>>>>>    . modify encode_cb_recallany4args to replace sizeof with
>>>>>      ARRAY_SIZE.
>>>> Hi-
>>>>=20
>>>> I'm going to apply this version of the series with some minor
>>>> changes. I'll reply to the individual patches where we can
>>>> discuss those.
>>> Thank you Chuck!
>> Changes folded in and pushed to nfsd's for-next. For the trace
>> point patch, I've rebased your series on top of the patch that
>> relocates include/trace/events/nfs.h so that the new
>> show_rca_mask() macro can go in the common nfs.h instead of
>> the NFSD-specific fs/nfsd/trace.h.
>>=20
>> Feel free to pull and test to make sure I didn't do anything
>> bone-headed.
>=20
> I removed the get_sockaddr temporarily to test show_rca_mask.
> It works fine:
>=20
> [root@nfsvmf24 ~]# trace-cmd report
> trace-cmd: No such file or directory
>  Error: expected type 4 but read 5
> cpus=3D1
>    kworker/u2:6-2297  [000]  1349.863391: nfsd_cb_recall_any:   client 63=
767ac9:adb1a3fb keep=3D0 bmval0=3DRDATA_DLG
>    kworker/u2:0-8698  [000]  1349.869652: nfsd_cb_recall_any_done: client=
 63767ac9:adb1a3fb status=3D0

Perfect, thank you!


--
Chuck Lever