Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Wed, 6 May 2020 11:28:56 -0700
From:   "Paul E. McKenney" <paulmck@kernel.org>
To:     Peter Zijlstra <peterz@infradead.org>
Cc:     Alexandre Chartre <alexandre.chartre@oracle.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        LKML <linux-kernel@vger.kernel.org>, x86@kernel.org,
        Andy Lutomirski <luto@kernel.org>,
        Frederic Weisbecker <frederic@kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Sean Christopherson <sean.j.christopherson@intel.com>,
        Masami Hiramatsu <mhiramat@kernel.org>,
        Petr Mladek <pmladek@suse.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Joel Fernandes <joel@joelfernandes.org>,
        Boris Ostrovsky <boris.ostrovsky@oracle.com>,
        Juergen Gross <jgross@suse.com>,
        Brian Gerst <brgerst@gmail.com>,
        Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
        Josh Poimboeuf <jpoimboe@redhat.com>,
        Will Deacon <will@kernel.org>
Subject: Re: [patch V4 part 1 03/36] sched: Clean up scheduler_ipi()
Message-ID: <20200506182856.GD2869@paulmck-ThinkPad-P72>
Reply-To: paulmck@kernel.org
References: <20200505131602.633487962@linutronix.de>
 <20200505134058.361859938@linutronix.de>
 <f969427d-a151-2c69-a779-a2b602e39d9e@oracle.com>
 <20200506153300.GB5281@hirez.programming.kicks-ass.net>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20200506153300.GB5281@hirez.programming.kicks-ass.net>
User-Agent: Mutt/1.9.4 (2018-02-28)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, May 06, 2020 at 05:33:00PM +0200, Peter Zijlstra wrote:
> On Wed, May 06, 2020 at 02:37:19PM +0200, Alexandre Chartre wrote:
> > On 5/5/20 3:16 PM, Thomas Gleixner wrote:
> > > @@ -650,6 +655,16 @@ static inline bool got_nohz_idle_kick(vo
> > >   	return false;
> > >   }
> > > +static void nohz_csd_func(void *info)
> > > +{
> > > +	struct rq *rq = info;
> > > +
> > > +	if (got_nohz_idle_kick()) {
> > > +		rq->idle_balance = 1;
> > > +		raise_softirq_irqoff(SCHED_SOFTIRQ);
> > > +	}
> > > +}
> > > +
> > >   #else /* CONFIG_NO_HZ_COMMON */
> > >   static inline bool got_nohz_idle_kick(void)
> 
> > >   #ifdef CONFIG_NO_HZ_COMMON
> > >   		rq->last_blocked_load_update_tick = jiffies;
> > >   		atomic_set(&rq->nohz_flags, 0);
> > > +
> > > +		rq_csd_init(rq, &rq->nohz_csd, nohz_csd_func);
> > >   #endif
> > >   #endif /* CONFIG_SMP */
> > >   		hrtick_rq_init(rq);
> > > --- a/kernel/sched/fair.c
> > > +++ b/kernel/sched/fair.c
> > > @@ -10009,12 +10009,11 @@ static void kick_ilb(unsigned int flags)
> > >   		return;
> > >   	/*
> > > -	 * Use smp_send_reschedule() instead of resched_cpu().
> > > -	 * This way we generate a sched IPI on the target CPU which
> > > +	 * This way we generate an IPI on the target CPU which
> > >   	 * is idle. And the softirq performing nohz idle load balance
> > >   	 * will be run before returning from the IPI.
> > >   	 */
> > > -	smp_send_reschedule(ilb_cpu);
> > > +	smp_call_function_single_async(ilb_cpu, &cpu_rq(ilb_cpu)->wake_csd);
> > 
> > This should be nohz_csd instead of wake_csd, no? I.e.:
> > 
> >        smp_call_function_single_async(ilb_cpu, &cpu_rq(ilb_cpu)->nohz_csd);
> 
> You figured correctly. Thanks!

And this does get rid of the smp_call_function splats I was seeing
earlier, thank you!

I still see warnings of the form "leave instruction with modified stack
frame" from older complilers and of the form "undefined stack state"
from newer compilers.  I am running stock objtool versions, so I am
guessing that this is at least one reason for these warnings.

							Thanx, Paul