Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <7ee9c36b-b3d8-bb84-36bb-393eaa2369f2@oracle.com>
Date:   Thu, 17 Nov 2022 08:53:08 -0800
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0)
 Gecko/20100101 Thunderbird/102.4.2
Subject: Re: [PATCH v5 0/4] add support for CB_RECALL_ANY and the delegation
 reaper
Content-Language: en-US
To:     Chuck Lever III <chuck.lever@oracle.com>
Cc:     "jlayton@kernel.org" <jlayton@kernel.org>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
References: <1668656688-22507-1-git-send-email-dai.ngo@oracle.com>
 <8517F18C-9207-4DE4-A217-2A0EA4C4484C@oracle.com>
From:   dai.ngo@oracle.com
In-Reply-To: <8517F18C-9207-4DE4-A217-2A0EA4C4484C@oracle.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?cFFjRGZXeERJRXk0NmNyWDB0eTJsK0hObmcvUEhPN0w4bXJad1lOcmxYcmxi?=
 =?utf-8?B?ZnBQQnBkNDk1T0FuSTBXWDlncmp4OHVaRnpWMTFwNzFjUXE4ODVDYU5KYWFO?=
 =?utf-8?B?ZTZPclJPRUFCL1ZQd1h1YkN1UHNZMUE3Q1JLdHorTHZlcDBXRHV6R3h0RDBR?=
 =?utf-8?B?WEZOcng2c1lhYmVGdHI5K3ZoUWlsY3BmMmZzQktLaUdaNkxidmxpbk9zMWRI?=
 =?utf-8?B?WHZQdTlBOHgzNS9aY0JFcjZMOU5JdVVaWlJKcGM2TFJkVmx1MEx3MVhGNDFy?=
 =?utf-8?B?Mks1dnNXR01oSjY1WUdSRzU0eUdZU0JvWU53dDB0Slhkcnpwb1BDT0VkTjhB?=
 =?utf-8?B?UC9RZ01wRmFkQ1FiZHU1aFpmalVyRTJON3ErODIrU1d4akpMUWJPeEJmNGJn?=
 =?utf-8?B?TWZNbmo3NUV2SW55SElrZHlBV3dPV3lqTFZtQmFwaUVmQlhObmY5bWJadVYv?=
 =?utf-8?B?SjNGOWgzY1VMMzNBd3o2SXlpQnpYR2FBc2lCVTk2Vk8vUjdKanF5KzEwMTF2?=
 =?utf-8?B?Yi9lbm5HYkxhd1F5UnJEUVpQUnlaTk1tSExJMXBzR2pBUSszT1NmUzJhM2Uv?=
 =?utf-8?B?RFNIckYwV0tySXNuTE9JL2JZdmFncVJKV2V6V1B5Q0l0VlJBZXdVU0ZLVytu?=
 =?utf-8?B?RGJKQ3JqN1hNSUVzWVBBL0FKVENGWGV3L1hMMTl5ZGI1UzIraWNVWlpENHd3?=
 =?utf-8?B?eTRnV1lGQWhKSjBNcWtLZ3h4RzVkN3hrcGthTHlXUVBjb3MyQ2tSR3VrWDBj?=
 =?utf-8?B?aWVKdlBWVWVaY0F5aURFemt4a01aZ3Ayd1A0d3c3VGNsOERiTkxzaFpEaGVR?=
 =?utf-8?B?elpCV3lzT1JYNFM2T3Uya0xMRC92L0VUN3lnQmhXNmtpTUpQby82blAxQ04w?=
 =?utf-8?B?c25kcnZrWS95cXdCQUtDeklic1FvYmkxVDhzMFpjWVdIdmV5b3grMktIM3A5?=
 =?utf-8?B?TjZIY09MMnY4NXpoYTBFck9uWVpJc0V5TjNId2pnRTV1VGp5Q1ZXM0R2b3Z4?=
 =?utf-8?B?WHBYRitrUlg4SGdwbEtYdEUwVXhDSWpsVS9JeW5PcTlOR054MTZ4dklTZksr?=
 =?utf-8?B?dWoxMXpsd2ZrVmVicm00NnYvNDBCWVpRTlFDeFAvcVdPeHVLc2VZdDlnNjI4?=
 =?utf-8?B?ZjE0L3Q0V3Y3SkJNdC8xakhwb1ZySDNkaEtUblZrK1JLTjVLVExFMVdTaFBX?=
 =?utf-8?B?NmpuL2d5Mmh2NFpKMTdWenZnZW5JRC9SZFZyc2I4clJIeWZLUzhteUZjelU3?=
 =?utf-8?B?b0JHTWlkcWxJU3JycHZmdDdtREZhTVZjd09RRU40VUxOcExNeWtJTFBpeVB1?=
 =?utf-8?B?Uk1xWDVKeHh6RDVrS0NqVjZtWURrcWhDRk9BdGJOc0R0bWllUVpOaUdKdldB?=
 =?utf-8?B?aTkwdGZlVmFjTlp4cUNocXBDTGFGRnRiYnF3cFd6aGV3WHZ3N3V5ZWNWNzda?=
 =?utf-8?B?Tzh1Yml0ZUlUYWk1VEk4bFh3RjlKZmt0dlV5T3F1Y1NYMUU2dUlMSDlrRUd5?=
 =?utf-8?B?dnBGaVhjdDZyTjNuV0NSZ2t4MUVrdUtvR0xhaUVPUzVsYndtRVIwNytMcmZB?=
 =?utf-8?B?ZEVDSWkzZnZIcGJSaDJtOVB4Y2t1eXRDRStWanc2Q3QvUWZmeE1yZERzR081?=
 =?utf-8?B?ZkNRNGNxL1dFaXVQYVhlMzVNTUcyTjlWdEJ1WmFGUHJQMUVFUkg1TTJ5MmRo?=
 =?utf-8?B?bUZQc0JxYW5zSTNRU2NHdkpJUEUySDJJTlU5R1J5ODl4YXM0KzdGOU1CSEJO?=
 =?utf-8?B?bm9IRVJzVnhzd1dxb1NNRkRCYjhWZFpBbGhiVkZtQ0o0MW15TDFIZ1VRMlZZ?=
 =?utf-8?B?SGlmRjZnTXBKUzBWRWwwOHBRTWQ0QU9nK3k3U1ZWSE1IUk1oOGc3cjNEM2ZM?=
 =?utf-8?B?ODBWdi9CVEZORUZZM1lLQWt1YnovTitrMDMvUlE4aTFDYktHdE9LT2M5bWh5?=
 =?utf-8?B?dFozbkpGcDRrZGZyTXJvSElwdVJ0a3RJV052cUNqbmZlaXRBV0V1d2xaTWwy?=
 =?utf-8?B?VmdIMjRIWWVnL2NPbkNPakgvVklPMDl2YitrTmJnTWxpdW02N3hEMkNIVzBx?=
 =?utf-8?B?Vy9QNWhOV2Z4ZC9MMExpaTRZY05DL1NYbGpONmVXOXpseWlGamV4aDJGYVJC?=
 =?utf-8?Q?giUTBMP1uvQkLh8Pg/3eCykgK?=
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: fSn4IxPy/K5Iz3hf9qG4kf8l9GhC6qtRTJVmX1ovKx+EBqbZeLQmGxQXiKzQ9ghPraB1dtIT8LnmTxDDqQTc5khWNWA++PdUrPgN8y2wZdxOFPp79lHs5ku+0dzyvF9VpF4JD7y4qML4EbaeQlofKPqW8b+dHEVNFDjF1PwTjXMm/j93LbPvCgpj5AyCMDX50nXCmVRNo1kvC/lH8DUNoMnJJCRohc+MntednfdCSnouK7RcUG4OyfPn25l3xQky9wsOx0JLb4idpifLrr0ILMQcvQXTGf+Xeu1Ny9esAtgkxEzaR+HE7spBHD6ShEfwYVi3mePI/wE1lA7xkhrl0L3xePTPiUadwRmwY2b52iLspzG0sIMq+On9HSk1OO9xNGG3Xn1w0v3G5BUSDCgfdC3FK1MuN6FgiAtiV1MMInzLf7ra3lfuCo3OttYAXBL2WIE9JzBbs8vyOJ5//r4bafVOME7HyIcQhZFfj5Qt2yNRT7B0p2f5YOFK7CwGQsp/oz0BMC0E6IeVyETkPQ3h7uxzZk1vF/RAmcjwRG7Aj87/EKTungNOvdBIMjtB2CbKFAdAqNDW3k8bZzbjjKoSpbwuh15AozDp/50M87HXNTOoM69aFu6HYVfnvzIh6F54/cxc/wv31m3dm6RQLNLjgLP3OB9Cr+humk/qpKz3mEF2/LG3J3MGBQjrD7Z3TX+wgugMOO/+1g9jCaYy9B5KdKHNUCMdL00V8c+7UU6w1zZyqV00mnlrLgKwi+R/Bd4MxEAfkmE4rYaSDWCEP/o78BALJSgfcVGIgiVm17dWTv8=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 17f89376-02aa-48c0-167d-08dac8bc36bb
X-MS-Exchange-CrossTenant-AuthSource: BY5PR10MB4257.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 17 Nov 2022 16:53:12.5390
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: yrvtAh68wXsCJIVBXcr6XxsxCy4nFuU2O3+zX7EnJtZrYDfHiRn6je63qZ0S81ux0eNFGExuFzkMP5g4Kyzhpw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS0PR10MB6150
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.219,Aquarius:18.0.895,Hydra:6.0.545,FMLib:17.11.122.1
 definitions=2022-11-17_06,2022-11-17_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxscore=0 adultscore=0
 phishscore=0 malwarescore=0 suspectscore=0 mlxlogscore=999 spamscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2210170000
 definitions=main-2211170125
X-Proofpoint-ORIG-GUID: VIf6ERSn8Sp-6QLluzIu-5RrLOqRCNyh
X-Proofpoint-GUID: VIf6ERSn8Sp-6QLluzIu-5RrLOqRCNyh
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


On 11/17/22 6:44 AM, Chuck Lever III wrote:
>
>> On Nov 16, 2022, at 10:44 PM, Dai Ngo <dai.ngo@oracle.com> wrote:
>>
>> This patch series adds:
>>
>>     . refactor courtesy_client_reaper to a generic low memory
>>       shrinker.
>>
>>     . XDR encode and decode function for CB_RECALL_ANY op.
>>
>>     . the delegation reaper that sends the advisory CB_RECALL_ANY
>>       to the clients to release unused delegations.
>>       There is only one nfsd4_callback added for each nfs4_cleint.
>>       Access to it must be serialized via the client flag
>>       NFSD4_CLIENT_CB_RECALL_ANY.
>>
>>     . Add CB_RECALL_ANY tracepoints.
>>
>> v2:
>>     . modify deleg_reaper to check and send CB_RECALL_ANY to client
>>       only once per 5 secs.
>> v3:
>>     . modify nfsd4_cb_recall_any_release to use nn->client_lock to
>>       protect cl_recall_any_busy and call put_client_renew_locked
>>       to decrement cl_rpc_users.
>>
>> v4:
>>     . move changes in nfs4state.c from patch (1/2) to patch(2/2).
>>     . use xdr_stream_encode_u32 and xdr_stream_encode_uint32_array
>>       to encode CB_RECALL_ANY arguments.
>>     . add struct nfsd4_cb_recall_any with embedded nfs4_callback
>>       and params for CB_RECALL_ANY op.
>>     . replace cl_recall_any_busy boolean with client flag
>>       NFSD4_CLIENT_CB_RECALL_ANY
>>     . add tracepoints for CB_RECALL_ANY
>>
>> v5:
>>     . refactor courtesy_client_reaper to a generic low memory
>>       shrinker
>>     . merge courtesy client shrinker and delegtion shrinker into
>>       one.
>>     . reposition nfsd_cb_recall_any and nfsd_cb_recall_any_done
>>       in nfsd/trace.h
>>     . use __get_sockaddr to display server IP address in
>>       tracepoints.
>>     . modify encode_cb_recallany4args to replace sizeof with
>>       ARRAY_SIZE.
> Hi-
>
> I'm going to apply this version of the series with some minor
> changes. I'll reply to the individual patches where we can
> discuss those.

Thank you Chuck!

-Dai

>
>
>> ---
>>
>> Dai Ngo (4):
>>      NFSD: refactoring courtesy_client_reaper to a generic low memory shrinker
>>      NFSD: add support for sending CB_RECALL_ANY
>>      NFSD: add delegation shrinker to react to low memory condition
>>      NFSD: add CB_RECALL_ANY tracepoints
>>
>> fs/nfsd/nfs4callback.c |  62 +++++++++++++++++++++++
>> fs/nfsd/nfs4state.c    | 116 +++++++++++++++++++++++++++++++++++++++-----
>> fs/nfsd/state.h        |   9 ++++
>> fs/nfsd/trace.h        |  49 +++++++++++++++++++
>> fs/nfsd/xdr4.h         |   5 ++
>> fs/nfsd/xdr4cb.h       |   6 +++
>> 6 files changed, 234 insertions(+), 13 deletions(-)
> --
> Chuck Lever
>
>
>