From:   Chuck Lever III <chuck.lever@oracle.com>
To:     Jeff Layton <jlayton@kernel.org>
CC:     Trond Myklebust <trond.myklebust@hammerspace.com>,
        Anna Schumaker <anna@kernel.org>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        "yoyang@redhat.com" <yoyang@redhat.com>
Subject: Re: [PATCH 0/7] lockd: fix races that can result in stuck filelocks
Thread-Topic: [PATCH 0/7] lockd: fix races that can result in stuck filelocks
Thread-Index: AQHZTcnxbdBJ4h9EiE6itgxSR0amOa7pIQIA
Date:   Fri, 3 Mar 2023 14:41:53 +0000
Message-ID: <0FC66364-4F59-4590-9211-EB54E918C97D@oracle.com>
References: <20230303121603.132103-1-jlayton@kernel.org>
In-Reply-To: <20230303121603.132103-1-jlayton@kernel.org>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?E84ivP77qDEJ1RLI1FtWLGJHfHz+PDQ8SWGPpOemLTqjjrxkdkUSdwAkAxWe?=
 =?us-ascii?Q?aKjk5825+L7jO9KNTSzudCzJGGUP2UC0lbzfe6TGdHnIDllLTOsojNDos2Hh?=
 =?us-ascii?Q?IO8i50B3i9dvDDBIintXKK0twJr8AUeDY5QNO9k5FWE1m0C+97QAo0K+ihUq?=
 =?us-ascii?Q?qtSHi4Ze79m9X8db4tCan0aBvHxtMqWEOv98k60hrQi+VpsDbYC+ekwizkHX?=
 =?us-ascii?Q?qqaYPF2d8x6AsSl8mS7ESQ01PawfSr80t58MNhQX6PJcD8MJGQKAsNkOjErH?=
 =?us-ascii?Q?JA6YGFuk+7+XbZcicHQ9vy/sKdCO4dr7Qim/Gpf3OYkqTuii+9LFVkHiNGfs?=
 =?us-ascii?Q?uhBn46jG1gwRgHKLSnManaiEhElaf+PG6BKIBBk0dc1Lxli86b+PH+uaCgpX?=
 =?us-ascii?Q?K6K/acoYgcsrAKQhdqRuTg24l9tK+Jrv9YY73K4Al7VSzwt/Z0Ddup2aSf4h?=
 =?us-ascii?Q?jwq9opt7INmzx4RX+l5BqI+EEBiP64F9PzyLDrZbAhKeNcNTkYORTS78n3/r?=
 =?us-ascii?Q?cBZ1BTqXv7hKJKSOqwXJK+SDHufayS6Bvttt3sMhfCboyD0Mauh37tyrVPbb?=
 =?us-ascii?Q?F0Qzwc/0HfjOTuFmxJ7hZkPs4mUs/GlQ1sK3amPv4038oggoKyOp5c0vkDf9?=
 =?us-ascii?Q?YvfvJom9BP/iwCyWROxvCRzOo5eQF2Mfu8VvRk3IoGTCLH1jjta83SzCL6Vb?=
 =?us-ascii?Q?srZN9Gg0covgIO01g1aUMZjIuHvcr1vrmEX8SoKgo4rPopOB7nLhC7x07Z8q?=
 =?us-ascii?Q?wUXB+nIZ+J5iPXY83WwVs1Fo6i0whL7DT/QOtvFrY3at47wEuMaKYT1BDNPK?=
 =?us-ascii?Q?HsdXPbn19iKl1oDqxPm6ej6bcBBfBao+RFNWtMePaFWsFbiK12Ufh9LsEL7m?=
 =?us-ascii?Q?TC94nCZ616aQ9Ol52Xb1s9mnk2w71zuiFSzswf4A/tL16/KfU4EgF5SLNNlI?=
 =?us-ascii?Q?AfRmBTUUOd16jpFqttAIV9j9uRjyJn/olZdAfZ6nsehJWbv+oGQWbTxGgUDp?=
 =?us-ascii?Q?7NP2JLiCtAHQsnXqR2X3U06PZhh6tEA0Iu/Z9RCk+f1uo85y41floce4uut6?=
 =?us-ascii?Q?cJxC5Hy/FE07ksDYzjF5hMfWDHBMH/FQ8Tip3+USPBaklaJmTRiEWLsm1ShK?=
 =?us-ascii?Q?zaiQ/f9jbOE/3Ef/HC0w/J782v5bNoyOlvUIor/egMO0IdCektVuc26BQGAe?=
 =?us-ascii?Q?8GPtJ6IuVFmsstclKIgkbdrqgR6JBZYmDXtzt7sTnCVDv7GCnyO08i0Ou4jQ?=
 =?us-ascii?Q?BX28iY5TJ/UvehMYHbdXxYJJKjNS36gBtUc8uWUg1cvvEFewmVKpIQRpExdg?=
 =?us-ascii?Q?XwLFy6vHXf2Ur00ztvqhJhrl1QmM7BhwibnxHllo5Yd2eDiyfE0f4e083UPO?=
 =?us-ascii?Q?fEuEJmOCBTn1VP0CT6YC37Xegbxd/nBUzXhQUpx4oYMM7sT5TWYUijBq7xac?=
 =?us-ascii?Q?hb1fASEkkWfSXxmU/9lX57yPdnakfE1yw7cZ+6jNA/9uEySUpioUPal+TYUp?=
 =?us-ascii?Q?DFc/QyaGd5ePzvylvOHtCaXVIBGveg0R+u2myRprCavEfphX/y+sA+A+z0/9?=
 =?us-ascii?Q?sO6QYtWuxjWdqYtYholg2q+gKuuGsC+a+G4zdl5/KLfx9upesIH4nfufdVs0?=
 =?us-ascii?Q?mg=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <BC950AC37A7DAA4CA0391AAB6B121C9A@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-ExternalHop-MessageData-0: zIk7KDBtj9FYwV1IEVNP0Ry6aHcP0WdO9wbf/Nn1LsJMXmLO/gubCBafevIPbM0A6B0yZCMeO6QD6sjiek148F2bY02zDiTwO/SV3t2CbncECu745UpiaqvC+uihfjqGY2XFMoC7jTcmMXJst3DhcBXp+0HAVaG0Tu+O1hu242/tXznYdf/XEFMiT+FhVqQdJ3StX9GoBaTPWQNrstIouETVaMma//dUb98WMae/7y2MFkop66vtsvDTjX63VUfcB0B9JWo95T8K/2JlOVc1C/0SNUp3x5kC2KdOF08+LLtSofyciU0JDcTOUIkLr5TrEq+3RUnsRfgd2+8vNBFYMlVwbAsBUULXtrjrqcgqnRxMQnrxK0JP5aRMB7EotXPv1aVRCaJMQ/a6XCYphkurrIFmohAyk1qqBuwJBXrU9bDNgS/cuAKV+Mz2Kmt087SRSDEPsYtaJvkR/vRAGInpUU4TykdwdlR9u+mQGeXQ019j4e4V1+5VTgrzQYRJfUOaWGn0FVSZaUyWHngsnokdiR1DdxPnJX1YNqe0fSuLuA3HwGgHePOeL/NXaD2rbHJUQiWRwbo+oyet3arLKwlMmBUAJrp9kBwKCFF5tanP7hIXO0Z31JWbnXAKC4JAnnt9MKVwUfgkOakk2rreEmasNQVhoPuvmc4lObWkXHlU8hY/XqBu8ErdY3wcBjHilpc5IJxCKyLM3cFT5GNhgUrNY/AYhKdxt2S8g5mtShgBJem3UXoHRJIHlr0uu2Y/AU0bxtsn8j2vIE2Vle8g7F1OCd4foEA3rMCjgyPH2xrc6f9zzAJiw/A7mfU6bdE9sbXVx+DIKzmIZG6IG6DKK4Vv8oRsi1f39u7j+ipkzMn8PCU=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN0PR10MB5128.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1d624fd0-1c57-4ed4-e104-08db1bf56e44
X-MS-Exchange-CrossTenant-originalarrivaltime: 03 Mar 2023 14:41:53.3377
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: jXJwhTTv5FPJekjG+23PRX6xOh07qR+qY5qd/WWHafDV14JO8O8AZLj8P4YUsWVuCo/xYMNb8kfmIbaqhWvMRg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR10MB6801
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.219,Aquarius:18.0.942,Hydra:6.0.573,FMLib:17.11.170.22
 definitions=2023-03-03_03,2023-03-03_01,2023-02-09_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 suspectscore=0 bulkscore=0
 malwarescore=0 mlxlogscore=999 phishscore=0 spamscore=0 adultscore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2212070000 definitions=main-2303030127
X-Proofpoint-ORIG-GUID: ATheqDkrYxxhMwhO6SylgRrn-5js2YX6
X-Proofpoint-GUID: ATheqDkrYxxhMwhO6SylgRrn-5js2YX6
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


> On Mar 3, 2023, at 7:15 AM, Jeff Layton <jlayton@kernel.org> wrote:
>=20
> I sent the first patch in this series the other day, but didn't get any
> responses.

We'll have to work out who will take which patches in this set.
Once fully reviewed, I can take the set if the client maintainers
send Acks for 2-4 and 6-7.

nfsd-next for v6.4 is not yet open. I can work on setting that up
today.


> Since then I've had time to follow up on the client-side part
> of this problem, which eventually also pointed out yet another bug on
> the server side. There are also a couple of cleanup patches in here too,
> and a patch to add some tracepoints that I found useful while diagnosing
> this.
>=20
> With this set on both client and server, I'm now able to run Yongcheng's
> test for an hour straight with no stuck locks.
>=20
> Jeff Layton (7):
>  lockd: purge resources held on behalf of nlm clients when shutting
>    down
>  lockd: remove 2 unused helper functions
>  lockd: move struct nlm_wait to lockd.h
>  lockd: fix races in client GRANTED_MSG wait logic
>  lockd: server should unlock lock if client rejects the grant
>  nfs: move nfs_fhandle_hash to common include file
>  lockd: add some client-side tracepoints
>=20
> fs/lockd/Makefile           |  6 ++-
> fs/lockd/clntlock.c         | 58 +++++++++++---------------
> fs/lockd/clntproc.c         | 42 ++++++++++++++-----
> fs/lockd/host.c             |  1 +
> fs/lockd/svclock.c          | 21 ++++++++--
> fs/lockd/trace.c            |  3 ++
> fs/lockd/trace.h            | 83 +++++++++++++++++++++++++++++++++++++
> fs/nfs/internal.h           | 15 -------
> include/linux/lockd/lockd.h | 29 ++++++-------
> include/linux/nfs.h         | 20 +++++++++
> 10 files changed, 200 insertions(+), 78 deletions(-)
> create mode 100644 fs/lockd/trace.c
> create mode 100644 fs/lockd/trace.h
>=20
> --=20
> 2.39.2
>=20

--
Chuck Lever