Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
From:   "Zhang, Qiang1" <qiang1.zhang@intel.com>
To:     "paulmck@kernel.org" <paulmck@kernel.org>
CC:     "frederic@kernel.org" <frederic@kernel.org>,
        "rcu@vger.kernel.org" <rcu@vger.kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH] rcu: Use IRQ_WORK_INIT_HARD() to initialize defer_qs_iw
 on PREEMPT_RT kernel
Thread-Topic: [PATCH] rcu: Use IRQ_WORK_INIT_HARD() to initialize defer_qs_iw
 on PREEMPT_RT kernel
Thread-Index: AQHYRyIUb1aC3TohkEGqUigSyiIkjazeVbAAgAP8uAA=
Date:   Wed, 6 Apr 2022 04:41:46 +0000
Message-ID: <PH0PR11MB58802019D7B5F23CEACE2D69DAE79@PH0PR11MB5880.namprd11.prod.outlook.com>
References: <20220403061440.2762522-1-qiang1.zhang@intel.com>
 <20220403154626.GO4285@paulmck-ThinkPad-P17-Gen-1>
In-Reply-To: <20220403154626.GO4285@paulmck-ThinkPad-P17-Gen-1>
Accept-Language: en-US
Content-Language: en-US
dlp-product: dlpe-windows
dlp-version: 11.6.401.20
dlp-reaction: no-action
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?NhEIP9IgXGQwEmHJWFVaN2idkNkJzgmwa3O15zvLOT16P40hYkr1ncPGWFIM?=
 =?us-ascii?Q?bfhVPlpXdkuPTqhFGti0i7xIFYmPNtyIiWQOj5F1Y4hXxCr0ibT6yuCKLxA4?=
 =?us-ascii?Q?gJUMsSExPKtwMMgvdKzq7GynZ8K4NeSvWrGvlALFo5rxyTlO1ADPjx4fvgeW?=
 =?us-ascii?Q?553+WHzlGlXK+P4P0fN+34C13Kje6i/cdUAJwEVrA36yRmSJ9OxQnRXLrxhL?=
 =?us-ascii?Q?dLdipOhg/1fh1XwIwuBky9DoLUKjhEvZInXAGUDPb2A0dfJbQVCmvIPuFZ0i?=
 =?us-ascii?Q?FZVQXbwQy8ARNGAModoOreMbBLm+gPKRMvIJ4gaYseCrUOD7W+tkbFky5rch?=
 =?us-ascii?Q?PGgiquhl3iZ2ip0b2zdniP8QdgKTdeqNhxZ3tvRIfZ9mMnTV5yDCBUt8dgoL?=
 =?us-ascii?Q?FbQapXH31G9Xka87yauFxVS1NHxfM2ms6liRM1ArdPp/zEbvfKfZgWxFy0XW?=
 =?us-ascii?Q?WlY0JeiXgo4M3NDin7zghrYQ9ZXgjKXNoTUJBdmUT5M7/x1EOt/gOMTY40rf?=
 =?us-ascii?Q?06iBrbeUqJRaV6V1bgp+LOZIvXs1xhEcuAYMZmrd75G3zmhzPUZewvLz4+mM?=
 =?us-ascii?Q?IVSEYVspAdZ8ePBwf+21t03saVUrIjjzGuHXvynccom0KGLpyKWVdhMsf84v?=
 =?us-ascii?Q?6NYLbeQRy3oDKtp3q+28kHEVp8sqwmrxXTxbYMHjA4bqThQwPQhC+Wr1rS9B?=
 =?us-ascii?Q?Wwx5PqH/2q94X7NgJZh9nZEqiP/NRR7ZAMZ95y2pcq737poU3a7qZR47ZDgR?=
 =?us-ascii?Q?fTRflvl2YQ3VWvYdfWuk5+kufA2Qc9kuwJoQ6sObLLLyIC8LklYnKUXqtIK3?=
 =?us-ascii?Q?HSXzWZwYc13Y6dhs1AC78u31o104u3uiVr0tajm0uWlqi/7W0XCwt6NQXXtA?=
 =?us-ascii?Q?R18kuFVgm0cqHMj4MM4mcFi8nmYFWbZFy4s5RiVqBia1cP5CVN8mXUvyuy04?=
 =?us-ascii?Q?/QX0ivkrX5+QI1mxHqi49oZNfig+Dnc6S55itV1vGPNnU3oG8feHK2vB5Hmj?=
 =?us-ascii?Q?F7pP/m1WrmZyg1w7udNa+nN9bw70tHHzFGyzMGpiflx81jcpbHVgckHixMLg?=
 =?us-ascii?Q?QN1gf9dEcFf5B+8Q2oKnDRIsZy70FGLsTFdgRcwDrUfs2lLBmYJVrBQw14i2?=
 =?us-ascii?Q?ubnYRxJ78sm0RRrLPnBH/OReTHxtsi+Hn45Kt5WWEmXng5I4I+iFe8kE9v+n?=
 =?us-ascii?Q?N28Ktx55J7wZY3Ay651zO1n/YsgRaF6yjQmM+j5uFEnKwDkZ51p0d9yWzXKe?=
 =?us-ascii?Q?S22gFcGrQnhe6YLIALWjNn76II/R5tl09nn5gGbvA8ybqEOXTtjSpagrYyOJ?=
 =?us-ascii?Q?gjzaq2gbWeQxuC7Le4Ye30c8n4//MXVjDKdKlgtSVSI/EbmfGma/x0SLrJN4?=
 =?us-ascii?Q?1NVWfNp7cX0yK1Cyc2jHIFT2KIj2ocgCifon+CJvuootjojdgmul4rHfG8Kq?=
 =?us-ascii?Q?y4GUZPSpGCClqXnSA8S/e+YzYuSlAT8a5FRawRn2jFgdKOy3ldX1XkYbp57c?=
 =?us-ascii?Q?weCJ2zSocVSvMROiU3ey1XDXvboLAh594ursAzsRIXmGHxfWY+GHZcmZtQ4s?=
 =?us-ascii?Q?6ivrA6Qj1wMyP2GHtXGXKaBeAACH/N71rH8Gyipc/gSfJtVIXsS1kObku7BW?=
 =?us-ascii?Q?1giDJ/z3tKCFEWLfJiEfmeTGd3+8/3taCwVqsxzfnlm3/emhplGjIUmRjE/i?=
 =?us-ascii?Q?BuMu+G2Ta1HrUfbi/QIlu8wiawTYKQiuShCX98U8EQsWYoX+ptbvignDLV05?=
 =?us-ascii?Q?h3nhh1dD+w=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB5880.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e5a7724f-dfe0-4ee2-6bec-08da1787c201
X-MS-Exchange-CrossTenant-originalarrivaltime: 06 Apr 2022 04:41:46.8149
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: P9FsRgOPh0npzjCVfEbq3FwPqAt0rd6DvFgJVvGpgvRr01Hty8vXFJGtG6Ce80lCQeDjZ2I1eoGleqDfKl4UgQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM5PR11MB1657
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=no autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On Sun, Apr 03, 2022 at 02:14:40PM +0800, Zqiang wrote:
> On non-PREEMPT_RT kernel, the init_irq_work() make the defer_qs_iw=20
> irq-work execute in interrupt context. however, on PREEMPT_RT kernel,=20
> the
> init_irq_work() make defer_qs_iq irq-work execute in rt-fifo irq_work=20
> kthreads. when system booting, and the CONFIG_RCU_STRICT_GRACE_PERIOD=20
> is enabled, there are a lot of defer_qs_iw irq-work to be processed in=20
> rt-fifo irq_work kthreads, it occupies boot CPU for long time and=20
> cause other kthread cannot get the boot CPU, the boot process occurs=20
> hang. use IRQ_WORK_INIT_HARD() to initialize defer_qs_iw irq-work, can=20
> ensure the defer_qs_iw irq-work always execute in interrupt context,=20
> whether PREEMPT_RT or non PREEMPT_RT kernel.

This is a much better justification of the need for a change, thank you!

>But it looks like I need to clarify a sentence in my previous email.
>
>Please note that you were using the debugging RCU_STRICT_GRACE_PERIOD Kcon=
fig option, so this is a potential problem as opposed to an immediate bug. =
 Yes, we must fix bugs, but it is also very important to avoid harming othe=
r workloads, which are after all the vast majority of the uses of the Linux=
 kernel.
>
>And a major purpose of things like RCU_STRICT_GRACE_PERIOD is to give us a=
dvanced warning of bugs so that we can fix them properly, without hurting o=
ther workloads.
>
>So, does this patch guarantee exactly the same performance and scalability=
 as before for !PREEMPT_RT systems?  If so, please add an explanation to th=
e commit log.
>
>Otherwise, please adjust the code to provide this guarantee.

Thanks, I have been adjusted code and resend v2.

Thanks
Zqiang

>
>							Thanx, Paul

> Signed-off-by: Zqiang <qiang1.zhang@intel.com>
> ---
>  kernel/rcu/tree_plugin.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>=20
> diff --git a/kernel/rcu/tree_plugin.h b/kernel/rcu/tree_plugin.h index=20
> 3037c2536e1f..cf7bd28af8ef 100644
> --- a/kernel/rcu/tree_plugin.h
> +++ b/kernel/rcu/tree_plugin.h
> @@ -661,7 +661,7 @@ static void rcu_read_unlock_special(struct task_struc=
t *t)
>  			    expboost && !rdp->defer_qs_iw_pending && cpu_online(rdp->cpu)) {
>  				// Get scheduler to re-evaluate and call hooks.
>  				// If !IRQ_WORK, FQS scan will eventually IPI.
> -				init_irq_work(&rdp->defer_qs_iw, rcu_preempt_deferred_qs_handler);
> +				rdp->defer_qs_iw =3D=20
> +IRQ_WORK_INIT_HARD(rcu_preempt_deferred_qs_handler);
>  				rdp->defer_qs_iw_pending =3D true;
>  				irq_work_queue_on(&rdp->defer_qs_iw, rdp->cpu);
>  			}
> --
> 2.25.1
>=20