Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Zhang, Qiang1" <qiang1.zhang@intel.com>
To:     "paulmck@kernel.org" <paulmck@kernel.org>
CC:     "frederic@kernel.org" <frederic@kernel.org>,
        "rcu@vger.kernel.org" <rcu@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH] rcu: Direct boosting gp_tasks for strict grace periods
Thread-Topic: [PATCH] rcu: Direct boosting gp_tasks for strict grace periods
Thread-Index: AQHYZmJsqZZOz0NYwkqRvRtT9o+4C60b/q+AgAC5l6CAACKMgIAGLBeA
Date:   Tue, 17 May 2022 12:32:10 +0000
Message-ID: <PH0PR11MB58802D034A41626D88F44BEEDACE9@PH0PR11MB5880.namprd11.prod.outlook.com>
References: <20220513004256.465233-1-qiang1.zhang@intel.com>
 <20220513010651.GD1790663@paulmck-ThinkPad-P17-Gen-1>
 <PH0PR11MB588012EC5E5DFD3FFDA49401DACA9@PH0PR11MB5880.namprd11.prod.outlook.com>
 <20220513141445.GF1790663@paulmck-ThinkPad-P17-Gen-1>
In-Reply-To: <20220513141445.GF1790663@paulmck-ThinkPad-P17-Gen-1>
Accept-Language: en-US
Content-Language: en-US
dlp-product: dlpe-windows
dlp-version: 11.6.401.20
dlp-reaction: no-action
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?WWmhPd88kS7gxA0jdzg9//zWg5mO8FudnQmH4Zb7qgXlp4TpdtbXI5nG6DX2?=
 =?us-ascii?Q?lPNGbW3fAZAGJQO6ox5Wtvnybqbhdt9oagglPkd3eidsTp+U6sqfXm7UmtR/?=
 =?us-ascii?Q?GYM/UWTawKKtNPuAMkZJ38WTSyopaVL0OUlYRFLgF+LFRuFB0TjMkeq0xKR6?=
 =?us-ascii?Q?9uDd07yCD9ICs2K7Y5jER5plcPyZXiqE9Eus6dt2+Ql0oQZdLcOgnyIdQhMG?=
 =?us-ascii?Q?31YE2W5/zIclNeDKJeiGjUPI8APTixn3/3n3561ZfLYS1nAjzh1ezi+pWMSb?=
 =?us-ascii?Q?J7quYbJYioPtQruVpFgfSP51X1Z2+Y7jUGs97oTZMwlvQlIdLIqFqPb/CigL?=
 =?us-ascii?Q?BduWEv4iM3POxBId7p9mV9lU2B/0C08e+6XVwpJ2iximF8Wzgtai43KCxFMM?=
 =?us-ascii?Q?DtCbTW3yI2vyVQ12T6wN+VQYXJmoTOCX5yzzOicTVsBPsHZhw4gDD9yoUYuZ?=
 =?us-ascii?Q?eZ9U5Uxb92P+qFfN3jeX2oD0JlTXnwnwGnrShlqkatprNvSCN8+z8gVsBlwO?=
 =?us-ascii?Q?H01E67j8a+uSy86NyhA61y9lnYvz8GpTPG785dDRnT+CAXlzL5NncVNY3kTV?=
 =?us-ascii?Q?lAbD4KeWjG7rvKwwE0YFiaF9sTob74KLwNMWIOs7dB30OzbnVaeYkUAKYo9g?=
 =?us-ascii?Q?7JIpdPsqEpg9eFR2ROlmV00WR64l+rg0LW0WGA9X8kE3/t9V1KRX55MSq+69?=
 =?us-ascii?Q?uiC5Tzj1ro2Qq7lmWCJ0hKmOB+I7n+09nKWLs3BLJ3sC3WjxzgOkBzLb+yFK?=
 =?us-ascii?Q?ZHKSa1HKfZC9ksZlZSvyRNm+ITYeYLL87A6NxurYEExMp5boKaxS7Y9wn8Xn?=
 =?us-ascii?Q?nznIR2P1btgJW6lNgMntKOdGl86u7EF6eUZJbcnFWpqJzFg4+IW0uKmda8Cy?=
 =?us-ascii?Q?a0VRbcStG3bq7oEeyzZvCcjNAGAvOeo8rxffhZ0U0JsRzhG5GRXFlgskewBs?=
 =?us-ascii?Q?SrrOvF6tqVXGOVusJf8Rdm99fSFaAzBgbNVBPYzptYJIUpMzYVfGzlRzv9a6?=
 =?us-ascii?Q?v7cT7WcbNtXc9sWqPygM6hcTW/WlNMUWOR+sreutC3ZeJIcZNkEAob9V0OZa?=
 =?us-ascii?Q?oRFYYdkXklGL52B21bkASox2H9hTlUDKUI4Brjt7J0VB7gnh7oUMuk2m21LD?=
 =?us-ascii?Q?XKofv0cXs8bjAlTMqQtqO50NvX2cOi1aAEbs/vBOeNo/pl7ymrBi7PQmeIVu?=
 =?us-ascii?Q?kNCuxjhGsjUNWcKIvU/RhJkLifuXLjgx5QSY2JKnksfZzrcP/MqBKGefdki4?=
 =?us-ascii?Q?N7zPkn+vNCricSm5IUnLxabYymznGUDgiibdzwoB0FmJvHjkMLNCan33xQwm?=
 =?us-ascii?Q?yy+gfkbuzKXa3vOH9ZD1A6IxFLVclSkpv6jum3BiAvfCqrDSxjUuGEetl4N2?=
 =?us-ascii?Q?4MKS32HxcL68VI8/SZ1xaK4HU2ZyenayXDBaaV6gtX9y9QJVjlGr5GgutvQL?=
 =?us-ascii?Q?GW+OMp5KAqgqAddNeAA9HLtCvgWetJrmNXHgLGPAMa3T4zYnNlySccEuTGrw?=
 =?us-ascii?Q?aTZMj/b0VVacSmR7uQf0JbAFOqQZm173qiNixHWGJxmp/T1xo9lxpyt9U8F5?=
 =?us-ascii?Q?ojclCd0HOk9rYTHZQAOdNIvSqaxeJyDrHX/1UssKNiQrprz6wz9STUT97Dbu?=
 =?us-ascii?Q?E0DGDVJKzLTWViZ8K9aG4syyUgOGle5Z9z92W5rbFCYun9GyOS9ogiQZ93Ti?=
 =?us-ascii?Q?IqCbBLyIUzEZl2gBe9657abmLnBY/mcO0X8ZsCobcRAaNCAtWvQTFrl7EJmX?=
 =?us-ascii?Q?IJeb2h7/Xw=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5880.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 824f5143-bc81-4309-8fce-08da380143ba
X-MS-Exchange-CrossTenant-originalarrivaltime: 17 May 2022 12:32:10.8613
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: nyrDMvC6qKBda/M0tIqLHxSPScSNiLFErQO8UJ1qJnRpeXw1ZDQis/hGraBJDt1PKA/m1oub5sVkNbOHBMOtFg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB1778
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-5.5 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,
        RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE,URIBL_BLACK autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, May 13, 2022 at 01:17:16PM +0000, Zhang, Qiang1 wrote:
>=20
> On Fri, May 13, 2022 at 08:42:55AM +0800, Zqiang wrote:
> > If the CONFIG_RCU_STRICT_GRACE_PERIOD option is enabled, the normal=20
> > grace period will be treated as expedited grace period, the gp_tasks=20
> > that block current grace period needs to be boosted unconditionally,=20
> > therefore this commit adds Kconfig check in rcu_initiate_boost().
> >=20
> > Signed-off-by: Zqiang <qiang1.zhang@intel.com>
>=20
> >Good eyes!  I have queued this for further review and testing, thank you=
!
> >
> >What sorts of tests did you run on it?
>=20
> Hi Paul
>=20
> I didn't think of  suitable test method, Can you provide me with a=20
> test method to test it, I will be happy to test.
>
>Here is one possibility:
>
>tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 60 --co=
nfigs "TREE01 TREE02 TREE03 TREE04 TREE05 TREE07 TREE09 TREE10" --kconfig "=
CONFIG_NR_CPUS=3D4 CONFIG_RCU_STRICT_GRACE_PERIOD=3Dy" --trust-make
>
>On a 16-CPU system, this will do eight kernel builds and run about two hou=
rs of testing.

Hi Paul

I tested according to the above command, and the results are as follows:

tools/testing/selftests/rcutorture/bin/kvm.sh --allcpus --duration 60 --con=
figs TREE01 TREE02 TREE03 TREE04 TREE05 TREE07 TREE09 TREE10 --kconfig CONF=
IG_NR_CPUS=3D4 CONFIG_RCU_STRICT_GRACE_PERIOD=3Dy --trust-make
TREE01 ------- 154236 GPs (42.8433/s) [rcu: g1986553 f0x0 total-gps=3D49693=
0]
:CONFIG_NR_CPUS=3D4: improperly set
:CONFIG_RCU_STRICT_GRACE_PERIOD=3Dy: improperly set
TREE02 ------- 410546 GPs (114.041/s) [rcu: g196373325 f0x2 total-gps=3D490=
93612] n_max_cbs: 805081
TREE03 ------- 160648 GPs (44.6244/s) [rcu: g128673793 f0x0 total-gps=3D321=
68735] n_max_cbs: 646284
TREE04 ------- 347059 GPs (96.4053/s) [rcu: g539425233 f0x2 total-gps=3D134=
856594] n_max_cbs: 205907
TREE05 ------- 360973 GPs (100.27/s) [rcu: g77594645 f0x0 total-gps=3D19398=
951] n_max_cbs: 31033
:CONFIG_RCU_FANOUT_LEAF=3D6: improperly set
TREE07 ------- 355903 GPs (98.8619/s) [rcu: g639182469 f0x0 total-gps=3D159=
795908] n_max_cbs: 259737
TREE09 ------- 305700 GPs (84.9167/s) [rcu: g9668841 f0x0 total-gps=3D24175=
06] n_max_cbs: 2498623
:CONFIG_NR_CPUS=3D4: improperly set
:CONFIG_RCU_STRICT_GRACE_PERIOD=3Dy: improperly set
TREE10 ------- 220566 GPs (61.2683/s) [rcu: g3456465 f0x0 total-gps=3D86440=
9] n_max_cbs: 507348
:CONFIG_RCU_STRICT_GRACE_PERIOD=3Dy: improperly set

Thanks
Zqiang

>
>							Thanx, Paul

> Thanks,
> Zqiang
>=20
> >
> >As usual, I could not resist the urge to wordsmith, so could you please =
check the version shown below?
> >
> >							Thanx, Paul
>=20
> ----------------------------------------------------------------------
> --
>=20
> commit 079e0f894c5d887c678f94332c1fa7287abfd6bc
> Author: Zqiang <qiang1.zhang@intel.com>
> Date:   Fri May 13 08:42:55 2022 +0800
>=20
>     rcu: Immediately boost preempted readers for strict grace periods
>    =20
>     The intent of the CONFIG_RCU_STRICT_GRACE_PERIOD Konfig option is to
>     cause normal grace periods to complete quickly in order to better cat=
ch
>     errors resulting from improperly leaking pointers from RCU read-side
>     critical sections.  However, kernels built with this option enabled s=
till
>     wait for some hundreds of milliseconds before boosting RCU readers th=
at
>     have been preempted within their current critical section.  The value
>     of this delay is set by the CONFIG_RCU_BOOST_DELAY Kconfig option,
>     which defaults to 500 milliseconds.
>    =20
>     This commit therefore causes kernels build with strict grace periods
>     to ignore CONFIG_RCU_BOOST_DELAY.  This causes rcu_initiate_boost()
>     to start boosting immediately after all CPUs on a given leaf rcu_node
>     structure have passed through their quiescent states.
>    =20
>     Signed-off-by: Zqiang <qiang1.zhang@intel.com>
>     Signed-off-by: Paul E. McKenney <paulmck@kernel.org>
>=20
> diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index=20
> 99cde4c947699..b4ab952f04ea6 100644
> --- a/kernel/rcu/tree_plugin.h
> +++ b/kernel/rcu/tree_plugin.h
> @@ -1159,7 +1159,8 @@ static void rcu_initiate_boost(struct rcu_node *rnp=
, unsigned long flags)
>  	    (rnp->gp_tasks !=3D NULL &&
>  	     rnp->boost_tasks =3D=3D NULL &&
>  	     rnp->qsmask =3D=3D 0 &&
> -	     (!time_after(rnp->boost_time, jiffies) || rcu_state.cbovld))) {
> +	     (!time_after(rnp->boost_time, jiffies) || rcu_state.cbovld ||
> +	      IS_ENABLED(CONFIG_RCU_STRICT_GRACE_PERIOD)))) {
>  		if (rnp->exp_tasks =3D=3D NULL)
>  			WRITE_ONCE(rnp->boost_tasks, rnp->gp_tasks);
>  		raw_spin_unlock_irqrestore_rcu_node(rnp, flags);