Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   Trond Myklebust <trondmy@hammerspace.com>
To:     "david@fromorbit.com" <david@fromorbit.com>,
        "neilb@suse.de" <neilb@suse.de>
CC:     "bfields@fieldses.org" <bfields@fieldses.org>,
        "linux-nfs@vger.kernel.org" <linux-nfs@vger.kernel.org>,
        "linux-fsdevel@vger.kernel.org" <linux-fsdevel@vger.kernel.org>,
        "chuck.lever@oracle.com" <chuck.lever@oracle.com>
Subject: Re: sporadic hangs on generic/186
Thread-Topic: sporadic hangs on generic/186
Thread-Index: AQHYSfqw0+tgllXvcEWzJalDi9pcWazjlQyAgAADk4CAAA5/AIAACbaA
Date:   Thu, 7 Apr 2022 01:54:20 +0000
Message-ID: <b282c5b98c4518952f62662ea3ba1d4e6ef85f26.camel@hammerspace.com>
References: <20220406195424.GA1242@fieldses.org>        ,
 <20220407001453.GE1609613@dread.disaster.area> ,
 <164929126156.10985.11316778982526844125@noble.neil.brown.name>
         <164929437439.10985.5253499040284089154@noble.neil.brown.name>
In-Reply-To: <164929437439.10985.5253499040284089154@noble.neil.brown.name>
Accept-Language: en-US, en-GB
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?utf-8?B?ZENuQ2ZKaGVvMG45MjRIZlNpbnZtTnBJRGpSL1EzejNRblFrNGRBb1dWcDlw?=
 =?utf-8?B?aGtrUjVOR0NwRVVmWmRPVEFFeWZSNHovY2RZY0VaOVRoMG1KYnNzNGJ3UW0y?=
 =?utf-8?B?WG5JRHFtc0ljQWlmR3d1QmJzNGpRSzFHUmxmRCtxc29vVjNFZ0QxR3UzaGZj?=
 =?utf-8?B?VkJIRFpOZWIybXluY1RXREhvNURZUTZUNWsvOE5OQkdWd1F0M2U4azJlbUs3?=
 =?utf-8?B?b0JSVU8ySWFnRFRld0Y2QnA1MkZoUGd5MDVJNENGdHhZUG5VRXNQcWpVWUl1?=
 =?utf-8?B?UEIxd3F3dUZ2RkptTXhGc21tKzhUVzhDUDBJVVRvdUlHSEQwbVFLdnJzd0dj?=
 =?utf-8?B?dDJ4N3lrTTlmMmcwOTRPMEFZbDFBclZTUnRtV2FKVTlUMjlPUXQxcHVhNHo2?=
 =?utf-8?B?WE1qSnlJY3NNQXdvYjF4SUJhTHRjS21yZjJkL2pmRkYrRE9TWnVDT0hnek11?=
 =?utf-8?B?alJycEtQVWlzdEhTNTByNkt4dmRTckZmN1BSVTF2SEhKOUFxdFlLOUZ6Z0Iw?=
 =?utf-8?B?MFZxNUd2UDlyYnZZN1dBNy9ZSVBVUzVJbENoNTAwY1h6a2x6cnM5b0pVSkg3?=
 =?utf-8?B?dXJ4QkhORk03TUpwcnU4MWNjTGFBNTlwQ0xLZjZOUFdwbDNGeGNRVjk2VkJ4?=
 =?utf-8?B?WGZEVEVUWjIwVFBnZTlqOHNTT2lKNnl2YmlBUGtBcllzcWtaTmNxeDlPZUtT?=
 =?utf-8?B?c2RoNnJKcEgyRlJtdWdpUFNnbHQ2R0pOQmJ6dTFWdjNwdE4xcnpoN3JraEor?=
 =?utf-8?B?a1V6QjVaVGlESTFFNk5iNTRjdS95aURrcnlDNEVMSXRoa0lQVCtGM2NaRGlD?=
 =?utf-8?B?TkRlNlVaWmZ0Q24zMVlGVXJuM0REOTM1THZYd01US3ozemIycnRkQUhscVZm?=
 =?utf-8?B?UEQ1ZnVHbGxJVnpNTWR6WmdmRDgyRUF3eUgwcFk2NmVSNGZ3SG9helJ1ZUNt?=
 =?utf-8?B?QVp1azlMR29xejdZK3VzWEVXUEhsbUdURXljNDNnTkpkb0FWdHppaUZBU1Mv?=
 =?utf-8?B?SmtZU1hHeThuVndPWmsvT3UySmEvVlRWM0Zua2ZkUFJ6RnJEQk1HYURCM294?=
 =?utf-8?B?ckRyQmlSVlFVNnRxWW13L04rZm9yOTNoWDFpMmNQc3NmeEJvbWNzYlF6WlJs?=
 =?utf-8?B?KzNJNkJlYkdjTGJYUGh2Sk9YQ1R4aVJPZmlZSnIzMjhNbDFNbUsrOEUwMnJ0?=
 =?utf-8?B?cVZCa0prK2VxMjBuNXhlU3N0OURvdldFZk1vbUZ5MjU5Y3hlNFVkT3JXTCt4?=
 =?utf-8?B?bFF6UnlKbEtPajdFaW1kZDA2eEx0OTlEMit5QTdnOSt4ZmxITzVrZ1NJb3BL?=
 =?utf-8?B?QVE3OWtXajUvcXZwV0FmTUpydUFHcDBocGNUTVNEK255TzdoeThwNExlZU5v?=
 =?utf-8?B?a1gzdExJRWRyREk3N2hjQ2NDRWtvMkxWUVhWOGNlRWppc0NBVi9DM1dPaUlZ?=
 =?utf-8?B?alNGQkl3dmFXNU1pU1Q5dzdONDVSd1dwR0tBaXdac0JkK2hCalNQU1NoaUlX?=
 =?utf-8?B?ajZldm5OOExyN3pvTGxtWHVPZUNWcXYvNEppTU9hNU9GeWgrTUxvVmhqTXRK?=
 =?utf-8?B?bkhuaTErUkdFVmZabS9wTXBEZnVjME1KT0VxQ3l5UE1iVHRTUGZtQWZhKzRT?=
 =?utf-8?B?L2wzNTRIeWNVa0xRWGZrR2t4dDhqNXhPYVhoOVBMQmR6MmFvVEtoU1U3WFRj?=
 =?utf-8?B?MFIwK0MvY2NlOGIyUk8zaDQ1YkJvMTNWcWZkblMzdjUxeWF1a2VDb090ZFBj?=
 =?utf-8?B?WGNkYnNmdzdlb1NDbStuNTZUYXBsWkF5SVZ4ZmJKSENxeS8rQ09PZzc3UE5l?=
 =?utf-8?B?bFY1NVYxbWVLd1E1N3JVNW1mTWxQRjF6dWNTeEFkWVZXVzVid0w2S25aMXVa?=
 =?utf-8?B?UDRFVmxYcmxrRnNGdmFmMWVsaUxIQkRBNXEzd1B4bUNNRHNDRURtUFRtYTBK?=
 =?utf-8?B?UW1vWU5Wem53WW9FTWV6czZ5SjR1Yy9ITTlIbFBub1Nxd3ZoU29yNUsrK2x1?=
 =?utf-8?B?SWR3SGdFK09HOFdIeTJLRTNCbGFwWE52L2txZE9sWk0zMElvYXVvUTZiM1BB?=
 =?utf-8?B?TmM5cE1Bdm5kblorSXU4eDJ2UURNWFdQZnlLOGRYMUttWjRaL01Cd3ljL1Vi?=
 =?utf-8?B?YTR0Z0ZuaXJZdlRjKzYvdlh3YytQUTMzV1RSckpheFlyTmcvcXJQRTlCNkMw?=
 =?utf-8?B?elJMeDR5clZqaytoWGxGNDV1ejJvc1VLYlhQL2F1TERYSnhmYzNXWUNXNW84?=
 =?utf-8?B?T2dQdG1ZYjl3Z1piU2ZMNjNrVmovOHd4di80eHc1a3pYZmZlRFNINkhFNlZo?=
 =?utf-8?B?YUhZcjNuN3ZDK0w3RGhpV2JDNE0yb1Uzbi9KUHk2bVRDMFZ6djNaS05tL0pJ?=
 =?utf-8?Q?R23OTM3ItNjpQEl4=3D?=
Content-Type: text/plain; charset="utf-8"
Content-ID: <B307DDE04B1D0E41B1E65725DEE8D692@namprd13.prod.outlook.com>
Content-Transfer-Encoding: base64
MIME-Version: 1.0
X-OriginatorOrg: hammerspace.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: CH0PR13MB5084.namprd13.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 7dcc1c91-6d74-48c9-f660-08da1839885a
X-MS-Exchange-CrossTenant-originalarrivaltime: 07 Apr 2022 01:54:20.4961
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 0d4fed5c-3a70-46fe-9430-ece41741f59e
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: WYp0mM0nU4dssg8WYnTXpMYaAdZK+W4ZLEG7m9ePfnQnSilgp4LM0hI9o8MO50foAqfTWRaqHAWrb3NRhZz5+Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR13MB5500
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,
        SPF_PASS,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org

T24gVGh1LCAyMDIyLTA0LTA3IGF0IDExOjE5ICsxMDAwLCBOZWlsQnJvd24gd3JvdGU6DQo+IE9u
IFRodSwgMDcgQXByIDIwMjIsIE5laWxCcm93biB3cm90ZToNCj4gPiBPbiBUaHUsIDA3IEFwciAy
MDIyLCBEYXZlIENoaW5uZXIgd3JvdGU6DQo+ID4gPiBPbiBXZWQsIEFwciAwNiwgMjAyMiBhdCAw
Mzo1NDoyNFBNIC0wNDAwLCBKLiBCcnVjZSBGaWVsZHMgd3JvdGU6DQo+ID4gPiA+IEluIHRoZSBs
YXN0IGNvdXBsZSBkYXlzIEkndmUgc3RhcnRlZCBnZXR0aW5nIGhhbmdzIG9uIHhmc3Rlc3RzDQo+
ID4gPiA+IGdlbmVyaWMvMTg2IG9uIHVwc3RyZWFtLsKgIEkgYWxzbyBub3RpY2UgdGhlIHRlc3Qg
Y29tcGxldGVzDQo+ID4gPiA+IGFmdGVyIDEwKw0KPiA+ID4gPiBob3VycyAodXN1YWxseSBpdCB0
YWtlcyBhYm91dCA1IG1pbnV0ZXMpLsKgIFNvbWV0aW1lcyB0aGlzIGlzDQo+ID4gPiA+IGFjY29t
cGFuaWVkDQo+ID4gPiA+IGJ5ICJuZnM6IFJQQyBjYWxsIHJldHVybmVkIGVycm9yIDEyIiBvbiB0
aGUgY2xpZW50Lg0KPiA+ID4gDQo+ID4gPiAjZGVmaW5lIEVOT01FTcKgwqDCoMKgwqDCoMKgwqDC
oCAxMsKgwqDCoMKgwqAgLyogT3V0IG9mIG1lbW9yeSAqLw0KPiA+ID4gDQo+ID4gPiBTbyBlaXRo
ZXIgdGhlIGNsaWVudCBvciB0aGUgc2VydmVyIGlzIHJ1bm5pbmcgb3V0IG9mIG1lbW9yeQ0KPiA+
ID4gc29tZXdoZXJlPw0KPiA+IA0KPiA+IFByb2JhYmx5IHRoZSBjbGllbnQuwqAgVGhlcmUgYXJl
IGEgYnVuY2ggb2YgY2hhbmdlcyByZWNlbnRseSB3aGljaA0KPiA+IGFkZA0KPiA+IF9fR0ZQX05P
UkVUUlkgdG8gbWVtb3J5IGFsbG9jYXRpb25zIGZyb20gUEZfV1FfV09SS0VScyBiZWNhdXNlIHRo
YXQNCj4gPiBjYW4NCj4gPiByZXN1bHQgaW4gZGVhZGxvY2tzIHdoZW4gc3dhcHBpbmcgb3ZlciBO
RlMuDQo+ID4gVGhpcyBtZWFucyB0aGF0IGttYWxsb2MgcmVxdWVzdCB0aGF0IHByZXZpb3VzbHkg
bmV2ZXIgZmFpbGVkDQo+ID4gKGJlY2F1c2UNCj4gPiBHRlBfS0VSTkVMIG5ldmVyIGZhaWxzIGZv
ciBrZXJuZWwgdGhyZWFkcyBJIHRoaW5rKSBjYW4gbm93IGZhaWwuwqANCj4gPiBUaGlzDQo+ID4g
aGFzIHRpY2tsZWQgb25lIGJ1ZyB0aGF0IEkga25vdyBvZi7CoCBUaGVyZSBhcmUgbGlrZWx5IHRv
IGJlIG1vcmUuDQo+ID4gDQo+ID4gVGhlIFJQQyBjb2RlIHNob3VsZCBzaW1wbHkgcmV0cnkgdGhl
c2UgYWxsb2NhdGlvbnMgYWZ0ZXIgYSBzaG9ydA0KPiA+IGRlbGF5LiANCj4gPiBIWi80IGlzIHRo
ZSBudW1iZXIgdGhhdCBpcyB1c2VkIGluIGEgY291cGxlIG9mIHBsYWNlcy7CoCBQb3NzaWJseQ0K
PiA+IHRoZXJlDQo+ID4gYXJlIG1vcmUgcGxhY2VzIHRoYXQgbmVlZCB0byBoYW5kbGUgLUVOT01F
TSB3aXRoIHJwY19kZWxheSgpLg0KPiANCj4gSSBoYWQgYSBsb29rIHRocm91Z2ggdGhlIHZhcmlv
dXMgcGxhY2VzIHdoZXJlIGFsbG9jIGNhbiBub3cgZmFpbC4NCj4gDQo+IEkgdGhpbmsgeGRyX2Fs
bG9jX2J2ZWMoKSBpbiB4cHJ0X3NlbnRfcGFnZWRhdGEoKSBpcyB0aGUgbW9zdCBsaWtlbHkNCj4g
Y2F1c2Ugb2YgYSBwcm9ibGVtIGhlcmUuwqAgSSBkb24ndCB0aGluayBhbiAtRU5PTUVNIGZyb20g
dGhlcmUgaXMNCj4gY2F1Z2h0LA0KPiBzbyBpdCBjb3VsZCBsaWtlbHkgZmlsdGVyIHVwIHRvIE5G
UyBhbmQgcmVzdWx0IGluIHRoZSBtZXNzYWdlIHlvdQ0KPiBnb3QuDQo+IA0KPiBJIGRvbid0IHRo
aW5rIHdlIGNhbiBlYXNpbHkgaGFuZGxlIGZhaWx1cmUgdGhlcmUuwqAgV2UgbmVlZCB0byBzdGF5
DQo+IHdpdGgNCj4gR0ZQX0tFUk5FTCByZWx5IG9uIFBGX01FTUFMTE9DIHRvIG1ha2UgZm9yd2Fy
ZCBwcm9ncmVzcyBmb3INCj4gc3dhcC1vdmVyLU5GUy4NCj4gDQo+IEJydWNlOiBjYW4geW91IGNo
YW5nZSB0aGF0IG9uZSBsaW5lIGJhY2sgdG8gR0ZQX0tFUk5FTCBhbmQgc2VlIGlmIHRoZQ0KPiBw
cm9ibGVtIGdvZXMgYXdheT8NCj4gDQoNCkNhbiB3ZSBwbGVhc2UganVzdCBtb3ZlIHRoZSBjYWxs
IHRvIHhkcl9hbGxvY19idmVjKCkgb3V0IG9mDQp4cHJ0X3NlbmRfcGFnZWRhdGEoKT8gTW92ZSB0
aGUgY2xpZW50IHNpZGUgYWxsb2NhdGlvbiBpbnRvDQp4c19zdHJlYW1fcHJlcGFyZV9yZXF1ZXN0
KCkgYW5kIHhzX3VkcF9zZW5kX3JlcXVlc3QoKSwgdGhlbiBtb3ZlIHRoZQ0Kc2VydmVyIHNpZGUg
YWxsb2NhdGlvbiBpbnRvIHN2Y191ZHBfc2VuZHRvKCkuDQoNClRoYXQgbWFrZXMgaXQgcG9zc2li
bGUgdG8gaGFuZGxlIGVycm9ycy4NCg0KPiBUaGUgb3RoZXIgcHJvYmxlbSBJIGZvdW5kIGlzIHRo
YXQgcnBjX2FsbG9jX3Rhc2soKSBjYW4gbm93IGZhaWwsIGJ1dA0KPiBycGNfbmV3X3Rhc2sgYXNz
dW1lcyB0aGF0IGl0IG5ldmVyIHdpbGwuwqAgSWYgaXQgZG9lcywgdGhlbiB3ZSBnZXQgYQ0KPiBO
VUxMDQo+IGRlcmVmLg0KPiANCj4gSSBkb24ndCB0aGluayBycGNfbmV3X3Rhc2soKSBjYW4gZXZl
ciBiZSBjYWxsZWQgZnJvbSB0aGUgcnBjaW9kIHdvcmsNCj4gcXVldWUsIHNvIGl0IGlzIHNhZmUg
dG8ganVzdCB1c2UgYSBtZW1wb29sIHdpdGggR0ZQX0tFUk5FTCBsaWtlIHdlDQo+IGRpZA0KPiBi
ZWZvcmUuIA0KPiANCk5vLiBXZSBzaG91bGRuJ3QgZXZlciB1c2UgbWVtcG9vbHMgd2l0aCBHRlBf
S0VSTkVMLg0KDQpNb3N0LCBpZiBub3QgYWxsIG9mIHRoZSBjYWxsZXJzIG9mIHJwY19ydW5fdGFz
aygpIGFyZSBzdGlsbCBjYXBhYmxlIG9mDQpkZWFsaW5nIHdpdGggZXJyb3JzLCBhbmQgZGl0dG8g
Zm9yIHRoZSBjYWxsZXJzIG9mIHJwY19ydW5fYmNfdGFzaygpLg0KDQotLSANClRyb25kIE15a2xl
YnVzdA0KTGludXggTkZTIGNsaWVudCBtYWludGFpbmVyLCBIYW1tZXJzcGFjZQ0KdHJvbmQubXlr
bGVidXN0QGhhbW1lcnNwYWNlLmNvbQ0KDQoNCg==