Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 13 Apr 2022 07:07:29 -0700
From:   "Paul E. McKenney" <paulmck@kernel.org>
To:     Hillf Danton <hdanton@sina.com>
Cc:     Kalesh Singh <kaleshsingh@google.com>,
        Joel Fernandes <joel@joelfernandes.org>,
        LKML <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH v2] EXP rcu: Move expedited grace period (GP) work to RT
 kthread_worker
Message-ID: <20220413140729.GL4285@paulmck-ThinkPad-P17-Gen-1>
Reply-To: paulmck@kernel.org
References: <20220408045734.1158817-1-kaleshsingh@google.com>
 <CAEXW_YQ6_VpneJnBfhTOMr6DwJhNmvMAKDRMnpr8LxB9Gtt=Xg@mail.gmail.com>
 <20220408143444.GC4285@paulmck-ThinkPad-P17-Gen-1>
 <CAEXW_YSrGKXh5DiJyrNvmbssSXbWBkA-XUjGRdS8HtGvW1r6hw@mail.gmail.com>
 <20220408153447.GE4285@paulmck-ThinkPad-P17-Gen-1>
 <CAEXW_YT-vJmXgWPQ_1J34iTb+ZhrAgN7c-HPz7kW17HmvKzJ3A@mail.gmail.com>
 <20220408173908.GJ4285@paulmck-ThinkPad-P17-Gen-1>
 <20220409071740.6024-1-hdanton@sina.com>
 <20220413113711.1263-1-hdanton@sina.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220413113711.1263-1-hdanton@sina.com>
Precedence: bulk

On Wed, Apr 13, 2022 at 07:37:11PM +0800, Hillf Danton wrote:
> On Sat, 9 Apr 2022 08:56:12 -0700 Paul E. McKenney wrote:
> > On Sat, Apr 09, 2022 at 03:17:40PM +0800, Hillf Danton wrote:
> > > On Fri, 8 Apr 2022 10:53:53 -0700 Kalesh Singh wrote
> > > > Thanks for the discussion everyone.
> > > > 
> > > > We didn't fully switch to kthread workers to avoid changing the
> > > > behavior for users that dont need this low latency exp GPs. Another
> > > > (and perhaps more important) reason is because kthread_worker offers
> > > > reduced concurrency than workqueues which Pual reported can pose
> > > > issues on systems with a large number of CPUs.
> > > 
> > > A second ... what issues were reported wrt concurrency, given the output
> > > of grep -nr workqueue block mm drivers.
> > > 
> > > Feel free to post a URL link to the issues.
> > 
> > The issues can be easily seen by inspecting kthread_queue_work() and
> > the functions that it invokes.  In contrast, normal workqueues uses
> > per-CPU mechanisms to avoid contention, as can equally easily be seen
> > by inspecting queue_work_on() and the functions that it invokes.
> 
> The worker from kthread_create_worker() roughly matches unbound workqueue
> that can get every CPU overloaded, thus the difference in implementation
> details between kthread worker and WQ worker (either bound or unbound) can
> be safely ignored if the kthread method works, given that prioirty is barely
> a cure to concurrency issues.

Please look again, this time taking lock contention in to account,
keeping in mind that systems with several hundred CPUs are reasonably
common and that systems with more than a thousand CPUs are not unheard of.

							Thanx, Paul

> Hillf
> > 
> > Please do feel free to take a look.
> > 
> > If taking a look does not convince you, please construct some in-kernel
> > benchmarks to test the scalability of these two mechanisms.  Please note
> > that some care will be required to make sure that you are doing a valid
> > apples-to-apples comparison.
> > 
> > 							Thanx, Paul
> >