Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Chuck Lever III <chuck.lever@oracle.com>
To:     Mike Galbraith <efault@gmx.de>
CC:     LKML <linux-kernel@vger.kernel.org>,
        Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: Re: regression: nfs mount (even idle) eventually hangs server
Thread-Topic: regression: nfs mount (even idle) eventually hangs server
Thread-Index: AQHZFSMVb1dlHuUkzEyQB2fRdyO/kK55RIoAgAAJEgCAAAO9AA==
Date:   Thu, 22 Dec 2022 04:28:17 +0000
Message-ID: <1A404CDB-95A8-4D04-B76B-91D4F063B489@oracle.com>
References: <65ed34338c69cb034295f2c9cbe0af684d65d30f.camel@gmx.de>
 <360f3dcfb6cfbefdbcc42fc84c660995142a8645.camel@gmx.de>
 <241c118c2fb60df744bbe351387fc29a34ff6ab9.camel@gmx.de>
 <f533c2e38c0619ea0c3b4346d1c7c99c5ae2122b.camel@gmx.de>
In-Reply-To: <f533c2e38c0619ea0c3b4346d1c7c99c5ae2122b.camel@gmx.de>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?ptAlDpD3F6rmw/zyvtpARUnyyFYXk5wsU/yyquA6cHHnhSeFw/u+dXFT1B+T?=
 =?us-ascii?Q?cI50AjaAZBVzpn0nHBercx4EPdO+q7vp9Ks8Tpf6ICvgTsTKVSTzcYL9jnOU?=
 =?us-ascii?Q?hn9wOrd3ni29BkZnhfZ59Tjr1XYrR3OcYk6Q+9RdhIUdx1pYjUzXyYBCr7o2?=
 =?us-ascii?Q?zKaFk62s6PX4L8Vw2aKYksN8becpr+LLV/AVNe0JYo3KgbyGl7rSrEkOMt19?=
 =?us-ascii?Q?zOnBMMDcxBG0lauUC9PNrmZK51HE0VKPVVOwnq7NWdh+ASElFl2PCvOlJ8o4?=
 =?us-ascii?Q?cqdtAFHZDjr7gAGSJ0wtgq+YfOgt4Fv2599y+RcwyBGgWGl2OeOYu2xfNQWM?=
 =?us-ascii?Q?RYA9WKHSbQ+jb8RGdBhr3NC0oAMolgisiAB4HoyYXD4roAIAk/EyhhGxzbt7?=
 =?us-ascii?Q?X5V3FXx9ACoaRsqvKIBtNBSrztSK7SOFxY2j6MbRzwMqlKyrvMpPPwDXiGhr?=
 =?us-ascii?Q?QhrpIQL2D3AnmJGOEIIzSMW+8DOSkP01f2Ei+SGiBdk0J+b2p0scV3dTPxvi?=
 =?us-ascii?Q?4yeNT2c9MJ5HfvHoLvAU94O8lXHlH3AMVRUD0Momc6aeJCXwWofnPoy1JFEy?=
 =?us-ascii?Q?bYo3iMtkyWbhv1esQgMJcA1LcKmWZg0zKWtLcl/pszU0Gev8v+Mu5bSPf65u?=
 =?us-ascii?Q?bMM6piRohQOtd20v1BOZ2m7H6yCgIjQRF/Usiu2OEEkjnOcX0tX4BWXF7hfw?=
 =?us-ascii?Q?wdk6tGnHRylN/ZJwoLjrBOI2ZvSkVrTq4s/I3SoCGsp0IyD+qfMcyd/6mmCU?=
 =?us-ascii?Q?u8iDQQPpKA/asXp+97gYEA7f2/Yk9MQyShjoalQJ2nru4Jtd+A52oGvGZw2V?=
 =?us-ascii?Q?o9Fd4toR+URs4qmmgEUaVI8o50pWC05/J3dmYZoE9nDBAuAV2gWck2grbqnN?=
 =?us-ascii?Q?VmvEf84p7zAiWkJEsqkUQsG8X6eFOijtF1yKx6q2H6/2ilvhN3FWRs/J2aIP?=
 =?us-ascii?Q?BiprlPaLLiqB2u+QBxYqZwaTefw/cywayBj6Zj6UxwwAnBv5Hlh3+sxnN2Dg?=
 =?us-ascii?Q?29iWwORpA22u0sNQdapz24HqkqgjxBmO8ZkBrKnchIK1WsNI+LQI8v83mgQU?=
 =?us-ascii?Q?phD//Ehcahba6YFrdnA64AM1LfaSUupmpGKClLLfBnI/LlKtynAngx5fSjUA?=
 =?us-ascii?Q?oFDeWdBB2StDmAdFICi/rY81v0mDJzoS5z4HSFMXN6f+sIrIuYJ6a+TI/emY?=
 =?us-ascii?Q?mbzqzzdWjjAgdHfOsJ0oIGEjbofMZjb0sYz3XhASY1qIEseRbeeb4/JGisLW?=
 =?us-ascii?Q?Jse/eVH0KON6SDZ6/s1pey9jx4ZpvrqQq/nZVBWZvbUAX50EhG47sQnwaJco?=
 =?us-ascii?Q?yOnnp7Wj191891HOnMh+ybywibCqm7LgjIck/sRCTjV5xXYa4FyYkpnLODCt?=
 =?us-ascii?Q?59rHCfDye62OXaV28u2VxF6r+lGYm8ogAUaPJIFnbw6pWkkHAPHvrdhaJrZ9?=
 =?us-ascii?Q?HmH6IPswqbczMfSD7yOyf5tTh5xzsbRW5dBAf+bPX2xsZv2eS3I0a2itAMGh?=
 =?us-ascii?Q?ayCRwTuFdSVNr+7unAlNPub5seP5bLpRvMniTCsuiuOJtR7rU1nmkLWi8AXk?=
 =?us-ascii?Q?l4yTbu9Li4vPOqWCL9MXLvH4CIDiGATIvd7GoW8OND1JUiDZRXRjIQhCqeSs?=
 =?us-ascii?Q?yw=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <9802974D1E66B541A6942CB92A30F287@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN0PR10MB5128.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: ea12f93d-71db-4d53-25fd-08dae3d4f2d1
X-MS-Exchange-CrossTenant-originalarrivaltime: 22 Dec 2022 04:28:17.2242
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: aKFhFe9wzpWCC6Kc/hnts87RTAmqlY5hTxd9EnFK4OGTrFRGVjhNEURTIn2IE8ORpsyszSBDNm6hyDFKKw8IFQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY5PR10MB6072
X-Proofpoint-Virus-Version: vendor=baseguard
 engine=ICAP:2.0.205,Aquarius:18.0.923,Hydra:6.0.545,FMLib:17.11.122.1
 definitions=2022-12-22_01,2022-12-21_01,2022-06-22_01
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 adultscore=0 phishscore=0
 mlxlogscore=999 bulkscore=0 malwarescore=0 suspectscore=0 mlxscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2212070000
 definitions=main-2212220038
X-Proofpoint-GUID: Xj7_ZmjWzhcKKtRHM3zOCp5nH0TLq_KU
X-Proofpoint-ORIG-GUID: Xj7_ZmjWzhcKKtRHM3zOCp5nH0TLq_KU
X-Spam-Status: No, score=-2.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_LOW,
        RCVD_IN_MSPIKE_H2,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


> On Dec 21, 2022, at 11:14 PM, Mike Galbraith <efault@gmx.de> wrote:
>=20
> On Thu, 2022-12-22 at 04:42 +0100, Mike Galbraith wrote:
>> On Wed, 2022-12-21 at 10:56 +0100, Mike Galbraith wrote:
>>>> 6.1 didn't reproduce either, so it would appear to be a merge window b=
ug.
>>=20
>> Ah, not true, turning evolution loose in nfs mounted home and letting
>> it refresh mailboxes while desktop box was kept busy jammed up 6.1.0 in
>> fairly short order. =20
>=20
> Well crap.  That was _not_ virgin 6.1.0 after all, it was 6.1.0 with...
>=20
> 44df6f439a17 NFSD: add delegation reaper to react to low memory condition
> 3959066b697b NFSD: add support for sending CB_RECALL_ANY
> a1049eb47f20 NFSD: refactoring courtesy_client_reaper to a generic low me=
mory shrinker
>=20
> ...applied from poking about yesterday.  I had given up on those as
> culprit, and intended to pop them off and rebuild, but they were in
> fact in the booted kernel.  Oh well, booboo could have a bright side.

Hi Mike-

I'm guessing your client mounts with NFSv4.x.

We've been chasing a similar problem for a few months, but for
lack of a solid reproducer, it has remained elusive.

We believe the problem was introduced between 5.19 and 6.0. If
you have a fairly reliable way to trigger the server hang, then
maybe a bisect would be possible?


--
Chuck Lever