From:   Chuck Lever III <chuck.lever@oracle.com>
To:     Neil Brown <neilb@suse.de>
CC:     Linux NFS Mailing List <linux-nfs@vger.kernel.org>,
        Bruce Fields <bfields@fieldses.org>
Subject: Re: [PATCH 00/19 v2] SUNRPC: clean up server thread management
Thread-Topic: [PATCH 00/19 v2] SUNRPC: clean up server thread management
Thread-Index: AQHX4AnGpZ1rZjrB+EqrNrOLDJYL8awRMq8AgAHTaIA=
Date:   Wed, 24 Nov 2021 18:42:52 +0000
Message-ID: <1552D77D-45F9-43F1-B9B7-E2B68AB54A17@oracle.com>
References: <163763078330.7284.10141477742275086758.stgit@noble.brown>
 <20211123144956.GA8978@fieldses.org>
In-Reply-To: <20211123144956.GA8978@fieldses.org>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?ZeKmP3bRhNHiQjrq1Bm79IsTTYsrwWd7dFz/gNQmsIkwV43l9JbZBh5S7XAj?=
 =?us-ascii?Q?5Sknouka7eQG8V9ZyTQiPX/+r4FPZ0Lmf/J4u34Zqsjk2Mg1VltEKTFJAdTq?=
 =?us-ascii?Q?Kh1OwPLJyQGYSk/z9LgZaTz31TTwlqm8sot5lYf0CJvPq0wrqNH8qI2acqJJ?=
 =?us-ascii?Q?N4TkcvzM7ZrE4eyA3hHEelmHGAbXaSZmegzt8I9LXY9OTcAkWJauuHhDqcf7?=
 =?us-ascii?Q?pu7LwHOOFqG7uOqBu1LAEK6X58V9S13xT6Oi0pNj52AnTxDnaMrwh+2E7oJN?=
 =?us-ascii?Q?D+w7OfKPIjBS348xcq23EoRhH74pPDU1AXuoJBokuucYmAEvqfRirN0uHyLP?=
 =?us-ascii?Q?hoY+CEDoB133Oo0cafor8xRbs6fUVNL84tDfwO0P7sVSTkao0YocIeJQKpZk?=
 =?us-ascii?Q?donq+YFnGxsBsAcve7xze8IuSg+SCKUEWsz3QJiFDhrphuuZHclJEI4M3SCA?=
 =?us-ascii?Q?1qN6OZsJElN7L/8f3nWkP0vmmEZeXqqHva3T159NQkYap/JktYmIsvVR2nqM?=
 =?us-ascii?Q?S9czQzAzscstgrX5XK2A5T8RWSiltixCg6CI+KE/u0VDOqqzS1bQx1+MtVtd?=
 =?us-ascii?Q?G/bxCyRe16j6Ok1gyXrDklF/dvf1N70h1x6Z47zKfRiehIXfbaEErxGsSspk?=
 =?us-ascii?Q?/QcYMMZk571qE40OnWvlrw6NNi7cGKEix9pa6WfoO3rkWDlMVOAgn2tlCOx1?=
 =?us-ascii?Q?VkwEpBCkhCBUBXoFlU7ZpXQHJgAK0TK6aMYSD86jRYLG6+NL5g4l0AZIU93l?=
 =?us-ascii?Q?ZVDWH5/OH+ccsv8pM6f2rTKGGt0ngvZRWiZ3UsKttHLWgdjoMp2OW/SLFlTT?=
 =?us-ascii?Q?81TUmis2ELWSFR8cmAHO/dpga5rVLn57+ZfQLyo5UqMnup4IT5poX143V9aK?=
 =?us-ascii?Q?C7JKKPAxR4CSOOyUfSEWHBA+vyVLYn7vW7YCt+53bmuFHn7hyGl4fPfatpBP?=
 =?us-ascii?Q?rmOLgo9c2N3M+JUrFEHsxhLv3csqilAqWc9uBjumZEyyK3F52wjYWnpLg+NE?=
 =?us-ascii?Q?YN8BM+9Tl7iKJ2lPKmnDoP0OVkVfj263WUZJbslOHjwsmNXj/CgreV9p4E5B?=
 =?us-ascii?Q?yeIudPORIfOXCQp5Jg8eSNGOWylP1MKCqkAbuJEi5efzN+10yXSTcSB3V1vm?=
 =?us-ascii?Q?jg1PykzaAH+YatSoYqNWMlP8WzUx8s3XjU3mO7eqIzhaHjnKya9Ruy+PyVED?=
 =?us-ascii?Q?H0vywCyIgfCdDL73QB6wCEZPs/mVgrqKXQFIkHIHjI/9reXYaN8BD0yRHIsz?=
 =?us-ascii?Q?8ef7lfgS0kSNknlr2Xbb+5u+gYom1fJZf5GlwNxoPdTX+/Ua1B5PPoZw5TYg?=
 =?us-ascii?Q?xwTyoPCBqJiVJGFiqPx7tzICiKVJDt77uQiuygTkcJkVmW5IOZAZTjhqp2uA?=
 =?us-ascii?Q?Uhf1l0TBQmqwg30A34evsBB3bl13nhgLsMPH58xUeoFnXcBEHws5GAzFR5Mb?=
 =?us-ascii?Q?9CugZybDMMDeXYBBB72lpvtfXiZY40xZ3fRw2443JX1ZLh0Mh7pvYDdHhofv?=
 =?us-ascii?Q?nySNPjTnkePP3TaH13yEnqt1N+fpg2rF4FKOL618gxUgYjWDDkytaLUvonNg?=
 =?us-ascii?Q?1FUbpjvcjGRz2s2RS0pgWHVMDQ6MEWWx2Z1ONQVdCdtqR8ak1lkze7A6QWPP?=
 =?us-ascii?Q?2JyJshpcFncxVlKV1C/JVH8=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-ID: <B3B9CDF6D740BB438983FE4B83384A06@namprd10.prod.outlook.com>
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: SJ0PR10MB4688.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e7a07a95-59db-481d-5ec3-08d9af7a38bf
X-MS-Exchange-CrossTenant-originalarrivaltime: 24 Nov 2021 18:42:52.2134
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: q3dNv+dqNKTkPdCvHucb3rNiJg87qwqb/fSH8b0REis9L9vGWwkXyKtCi7QxRZXPdxDJqijNpA8XH667gXCjIw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR10MB5533
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10178 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 spamscore=0 phishscore=0 malwarescore=0
 mlxlogscore=999 bulkscore=0 mlxscore=0 suspectscore=0 adultscore=0
 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2110150000
 definitions=main-2111240097
X-Proofpoint-GUID: 2lWQcp99AVG_kfAa6p7J9RNyD8xDTHK_
X-Proofpoint-ORIG-GUID: 2lWQcp99AVG_kfAa6p7J9RNyD8xDTHK_
Precedence: bulk
List-ID: <linux-nfs.vger.kernel.org>
X-Mailing-List: linux-nfs@vger.kernel.org


> On Nov 23, 2021, at 9:49 AM, J. Bruce Fields <bfields@fieldses.org> wrote=
:
>=20
> On Tue, Nov 23, 2021 at 12:29:35PM +1100, NeilBrown wrote:
>> This is a revision of my series for cleaning up server thread
>> management.
>=20
> For what it's worth, this version now passes my usual regression tests.

Likewise, I tested with both TCP and RDMA.


> --b.
>=20
>> Currently lockd, nfsd, and nfs-callback all manage threads slightly
>> differently.  This series unifies them.
>>=20
>> Changes since first series include:
>>  - minor bug fixes
>>  - kernel-doc comments for new functions
>>  - split first patch into 3, and make the bugfix a separate patch
>>  - fix management of pool_maps so lockd can usse svc_set_num_threads
>>    safely
>>  - switch nfs-callback to not request a 'pooled' service.
>>=20
>> NeilBrown
>>=20
>>=20
>> ---
>>=20
>> NeilBrown (19):
>>      SUNRPC/NFSD: clean up get/put functions.
>>      NFSD: handle error better in write_ports_addfd()
>>      SUNRPC: stop using ->sv_nrthreads as a refcount
>>      nfsd: make nfsd_stats.th_cnt atomic_t
>>      SUNRPC: use sv_lock to protect updates to sv_nrthreads.
>>      NFSD: narrow nfsd_mutex protection in nfsd thread
>>      NFSD: Make it possible to use svc_set_num_threads_sync
>>      SUNRPC: discard svo_setup and rename svc_set_num_threads_sync()
>>      NFSD: simplify locking for network notifier.
>>      lockd: introduce nlmsvc_serv
>>      lockd: simplify management of network status notifiers
>>      lockd: move lockd_start_svc() call into lockd_create_svc()
>>      lockd: move svc_exit_thread() into the thread
>>      lockd: introduce lockd_put()
>>      lockd: rename lockd_create_svc() to lockd_get()
>>      SUNRPC: move the pool_map definitions (back) into svc.c
>>      SUNRPC: always treat sv_nrpools=3D=3D1 as "not pooled"
>>      lockd: use svc_set_num_threads() for thread start and stop
>>      NFS: switch the callback service back to non-pooled.
>>=20
>>=20
>> fs/lockd/svc.c             | 194 ++++++++++++-------------------------
>> fs/nfs/callback.c          |  12 +--
>> fs/nfsd/netns.h            |  13 +--
>> fs/nfsd/nfsctl.c           |  24 ++---
>> fs/nfsd/nfssvc.c           | 139 +++++++++++++-------------
>> fs/nfsd/stats.c            |   2 +-
>> fs/nfsd/stats.h            |   4 +-
>> include/linux/sunrpc/svc.h |  58 ++++-------
>> net/sunrpc/svc.c           | 166 ++++++++++++++-----------------
>> 9 files changed, 248 insertions(+), 364 deletions(-)
>>=20
>> --
>> Signature

--
Chuck Lever