Received-SPF: pass (google.com: domain of linux-nfs-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Chuck Lever III <chuck.lever@oracle.com>
To:     Neil Brown <neilb@suse.de>
CC:     Linux NFS Mailing List <linux-nfs@vger.kernel.org>
Subject: dynamic management of nfsd thread count
Thread-Topic: dynamic management of nfsd thread count
Thread-Index: AQHYETWIjqH23KS61Ea8gdX7jSVUMQ==
Date:   Mon, 24 Jan 2022 15:17:43 +0000
Message-ID: <EF3A3E35-FD8F-4ADF-AAFD-A120D83F80F5@oracle.com>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?nt+JRGhAHkh08sKHqQFB4qozkNfs1zrBDaJl861/L0lXGaE6/Sgby8+NrCJ0?=
 =?us-ascii?Q?LhGVWRr1vHs8FVniVx27oqN9g+OBMkCDhGLRQBIVMMIqSGNdwHSmCZvn5sOw?=
 =?us-ascii?Q?yIJAIpZ/9EwQREY28UgjFDvGC3UQO0CTSgwWmumKogxme3yblhUC16CX6sGL?=
 =?us-ascii?Q?qrXMbjdXl5zGHDY4HAXYmO8QgEYWFoFQPZbL2gP18/B09G+KYIc+pLew99Wy?=
 =?us-ascii?Q?WRkR5HhukxroISVee8A1hGJz+kjUxdEeMgQjS3MoVeqQtoAs4wL/lp885XRZ?=
 =?us-ascii?Q?7jiU4fH61b0gaw+hA7y0R4kjlkb/1btyVOH1ZCei9viI1lJBIDOKS8OZHoXx?=
 =?us-ascii?Q?zJO541VF1gH9ea2/4F7GfcVVQFcWmX8Y/XhCxbIUjzkYZPvGyhodY4VaoIY3?=
 =?us-ascii?Q?+x/dDywhqdRsX65fqLqepjkbMP6B7BQWl33XGP9EfTFmfXD/3eAHFhRM23y7?=
 =?us-ascii?Q?1+aa+5NclfZp5EL6Uf7DeHWmECqU+ADg9eacvEnklrZr8quLRoGUFLWh2k5X?=
 =?us-ascii?Q?eKQXMslesZsEY78hSokdF437CO+UtQMEptIcU09fF3sqXlklLPelmnhDwh3O?=
 =?us-ascii?Q?lVXi6LlulqV3zdGUkB/9wb5n3xL4psI6EE7CDfmWp1C3XTC9Q2vHWXu8TjHc?=
 =?us-ascii?Q?gUkmbGk5tmrhybPIOB3v7nMP8m4ulJj5b7uolK5fnx5+/Sgxo3ZYVmOCiMYs?=
 =?us-ascii?Q?9UnX3Us57GwDjI6zmOTMrr6Im2DjeCx6tnNAxHan8xb15WZdl48ZoXKCb5zW?=
 =?us-ascii?Q?v5cPFT5FcoQkbzsLgFHWgazprz2O3npgJen5E06P2/Ag14sZEz7VTbVasvWK?=
 =?us-ascii?Q?9nzFFxGcJRx7ciKyF0sfvyDerUPz3V124LGaQtIXPKfkWgJsvlRoX2/IZ8fj?=
 =?us-ascii?Q?6lbU/PXDG39qgatngwJmoUesVwPqMD2uej8K9GVFDSHOADpWPIbpUG11gKen?=
 =?us-ascii?Q?v5rodqXVwiL138cUSvVvrh6M1b3FgbfK7ak5LWW/l2XITujq2fbDwflbK887?=
 =?us-ascii?Q?5ahL+yquxMh2G/58n7zamZ+BGXy8rcv2uAfaLc2V+8/4H1gl+pNTOU/3y0HX?=
 =?us-ascii?Q?6zVb7zLuzPwbOaYA7+aZt/AiSoHEv10NPOsXSs4Z8BZAUnjDjSdyTIdijwrV?=
 =?us-ascii?Q?3ftsFA7lwz3ECXQGHzBqeYmTXU6oN66zymPqUT/NUdaha/ApQaTWyejwMFCe?=
 =?us-ascii?Q?oMx6S9dwaF+HxCdFmKRkDUKloFebOGSpZOHSlOs56K3O3yQXYgJXbIOo8+c7?=
 =?us-ascii?Q?PmMaKR8MhfWVva9l6M611YiLfgPmPXKlOuzDkACzgQKQu2daXjeDlxeaNLSV?=
 =?us-ascii?Q?B9vUj6VVdeSgiEG533f6izoIR/t6IEz196CJzKuc+EdVgc78rXpvpkKVeBvX?=
 =?us-ascii?Q?tzoxMVNGCgKjpPUctATgC3nT4N3LcRl3mltCBIwvON6aJ4jkef3uH/rtE/6D?=
 =?us-ascii?Q?wGNHgvwOpNNtlS9/BPu5mOXLAKYrFlivuGi4XreGJ2NaV1CVmI13Vvo+Qhq2?=
 =?us-ascii?Q?iWlfBtRWasnZBOx2TCECg+jAUGSf8GH49/ILw/yiDHORtDKdlLf/RAmPK4xt?=
 =?us-ascii?Q?c3MLnrfZb17z9oypbcBld6RBgp7i3SOesfxt5J9g5TCSrK0ohu/X0pbfUt11?=
 =?us-ascii?Q?K2czj4q1sJZyxxhhH62t83g=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <33D1EC00DB7DB748860969B12C145B2B@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: CH0PR10MB4858.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: b22faf8c-4eea-4582-4681-08d9df4cab7f
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Jan 2022 15:17:43.6139
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: /A20VBPGvXr7n380K7cZqsrH5TQJpeJgPxR1aqWlxeUzazvqJH4qhPQfQUV+0c8Id4YyuyjxKRTFbQOpA698+w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR10MB3146
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10236 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 bulkscore=0 mlxlogscore=999
 suspectscore=0 spamscore=0 adultscore=0 phishscore=0 malwarescore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2201110000 definitions=main-2201240102
X-Proofpoint-GUID: yJ9tsd6K_Vgi4ZKe2O1cb_PeXEXNTWHU
X-Proofpoint-ORIG-GUID: yJ9tsd6K_Vgi4ZKe2O1cb_PeXEXNTWHU
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org

Hi Neil-

The clean-up of NFSD thread management has been merged into v5.17-rc.
I'm wondering if you have thought more about how to implement dynamic
management.

One thing that occurred to me was that there was a changeset that
attempted to convert NFSD threads into work queues. We merged some
of that but stopped when it was found that work queue scheduling
seems to have some worst case behavior that sometimes impacts the
performance of the Linux NFS server.

The part of that work that was merged is now vestigial, and might
need to be reverted before proceeding with dynamic NFSD thread
management. For example:

 476 void svc_xprt_enqueue(struct svc_xprt *xprt)
 477 {
 478         if (test_bit(XPT_BUSY, &xprt->xpt_flags))
 479                 return;
 480         xprt->xpt_server->sv_ops->svo_enqueue_xprt(xprt);
 481 }
 482 EXPORT_SYMBOL_GPL(svc_xprt_enqueue);

The svo_enqueue_xprt method was to be different for the different
scheduling mechanisms. But to this day there is only one method
defined: svc_xprt_do_enqueue()

Do you have plans to use this or discard it? If not, I'd like to
see that it is removed eventually, as virtual functions are
more costly than they used to be thanks to Spectre/Meltdown, and
this one is invoked multiple times per RPC.


--
Chuck Lever