Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Wed, 4 Aug 2021 15:53:32 -0700
From:   "Paul E. McKenney" <paulmck@kernel.org>
To:     Valentin Schneider <valentin.schneider@arm.com>
Cc:     linux-kernel@vger.kernel.org, rcu@vger.kernel.org,
        linux-rt-users@vger.kernel.org, Ingo Molnar <mingo@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Steven Rostedt <rostedt@goodmis.org>,
        Daniel Bristot de Oliveira <bristot@redhat.com>,
        Frederic Weisbecker <frederic@kernel.org>,
        Josh Triplett <josh@joshtriplett.org>,
        Davidlohr Bueso <dave@stgolabs.net>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        Lai Jiangshan <jiangshanlai@gmail.com>,
        Joel Fernandes <joel@joelfernandes.org>
Subject: Re: [PATCH 2/2] rcutorture: Nudge ksoftirqd priority for RCU boost
 testing
Message-ID: <20210804225332.GB4397@paulmck-ThinkPad-P17-Gen-1>
Reply-To: paulmck@kernel.org
References: <20210803225437.3612591-1-valentin.schneider@arm.com>
 <20210803225437.3612591-3-valentin.schneider@arm.com>
 <20210803234231.GW4397@paulmck-ThinkPad-P17-Gen-1>
 <87tuk5a4yk.mognet@arm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <87tuk5a4yk.mognet@arm.com>
Precedence: bulk

On Wed, Aug 04, 2021 at 11:18:11AM +0100, Valentin Schneider wrote:
> On 03/08/21 16:42, Paul E. McKenney wrote:
> > On Tue, Aug 03, 2021 at 11:54:37PM +0100, Valentin Schneider wrote:
> >> diff --git a/kernel/rcu/rcutorture.c b/kernel/rcu/rcutorture.c
> >> index 680f66b65f14..3dd5fa75f469 100644
> >> --- a/kernel/rcu/rcutorture.c
> >> +++ b/kernel/rcu/rcutorture.c
> >> @@ -948,12 +948,26 @@ static int rcu_torture_boost(void *arg)
> >>      unsigned long endtime;
> >>      unsigned long oldstarttime;
> >>      struct rcu_boost_inflight rbi = { .inflight = 0 };
> >> +	struct task_struct *ksoftirqd = this_cpu_ksoftirqd();
> >>
> >>      VERBOSE_TOROUT_STRING("rcu_torture_boost started");
> >>
> >>      /* Set real-time priority. */
> >>      sched_set_fifo_low(current);
> >>
> >> +	/*
> >> +	 * Boost testing requires TIMER_SOFTIRQ to run at a higher priority
> >> +	 * than the CPU-hogging torture kthreads, otherwise said threads
> >> +	 * will never let timer expiry for the RCU GP kthread happen, which will
> >> +	 * prevent any boosting.
> >> +	 */
> >> +	if (current->normal_prio < ksoftirqd->normal_prio) {
> >
> > Would it make sense to add IS_ENABLED(CONFIG_PREEMPT_RT) to the above
> > condition?
> >
> 
> Hm so v5.13-rt1 has this commit:
> 
>   5e59fba573e6 ("rcutorture: Fix testing of RCU priority boosting")
> 
> which gates RCU boost torture testing under CONFIG_PREEMPT_RT. Now, AFAICT
> the TIMER_SOFTIRQ priority problem is there regardless of
> CONFIG_PREEMPT_RT, so this patch would (should?) make sense even on
> !CONFIG_PREEMPT_RT.

What rcutorture scenario TREE03 does is to boot with tree.use_softirq=0
and threadirqs.  I see your point about timers and softirq, but this
does run reliably for me.

Ah, I see why.  Commit ea6d962e80b6 ("rcutorture: Judge RCU priority
boosting on grace periods, not callbacks") includes boosting the priority
of the ksoftirqd kthreads.  But only when running rcutorture builtin,
not as a module.  Here is the code in rcu_torture_init():

		// Testing RCU priority boosting requires rcutorture do
		// some serious abuse.  Counter this by running ksoftirqd
		// at higher priority.
		if (IS_BUILTIN(CONFIG_RCU_TORTURE_TEST)) {
			for_each_online_cpu(cpu) {
				struct sched_param sp;
				struct task_struct *t;

				t = per_cpu(ksoftirqd, cpu);
				WARN_ON_ONCE(!t);
				sp.sched_priority = 2;
				sched_setscheduler_nocheck(t, SCHED_FIFO, &sp);
			}
		}

I take it that you were running rcutorture as a module?

This describes how to run it built-in, if that works for you:

https://paulmck.livejournal.com/61432.html

More specifically: https://paulmck.livejournal.com/57769.html

Alternatively, the "IS_BUILTIN(CONFIG_RCU_TORTURE_TEST)" check could be
removed in the above code, and the ksoftirqd kthreads could have their
original priority restored in rcu_torture_cleanup().

Thoughts?

							Thanx, Paul

> >> +		struct sched_param sp = { .sched_priority = 2 };
> >> +
> >> +		pr_alert("%s(): Adjusting %s priority\n", __func__, ksoftirqd->comm);
> >> +		sched_setscheduler_nocheck(ksoftirqd, SCHED_FIFO, &sp);
> >> +	}
> >> +
> >>      init_rcu_head_on_stack(&rbi.rcu);
> >>      /* Each pass through the following loop does one boost-test cycle. */
> >>      do {
> >> --
> >> 2.25.1
> >>