Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Wed, 15 Jan 2020 08:18:30 -0500
From:   Steven Rostedt <rostedt@goodmis.org>
To:     David Laight <David.Laight@ACULAB.COM>
Cc:     'Vincent Guittot' <vincent.guittot@linaro.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Viresh Kumar <viresh.kumar@linaro.org>,
        Ingo Molnar <mingo@redhat.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        linux-kernel <linux-kernel@vger.kernel.org>
Subject: Re: sched/fair: scheduler not running high priority process on idle
 cpu
Message-ID: <20200115081830.036ade4e@gandalf.local.home>
In-Reply-To: <878a35a6642d482aa0770a055506bd5e@AcuMS.aculab.com>
References: <212fabd759b0486aa8df588477acf6d0@AcuMS.aculab.com>
        <20200114115906.22f952ff@gandalf.local.home>
        <5ba2ae2d426c4058b314c20c25a9b1d0@AcuMS.aculab.com>
        <20200114124812.4d5355ae@gandalf.local.home>
        <878a35a6642d482aa0770a055506bd5e@AcuMS.aculab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, 15 Jan 2020 12:44:19 +0000
David Laight <David.Laight@ACULAB.COM> wrote:

> > Yes, even with CONFIG_PREEMPT, Linux has no guarantees of latency for
> > any task regardless of priority. If you have latency requirements, then
> > you need to apply the PREEMPT_RT patch (which may soon make it to
> > mainline this year!), which spin locks and bh wont stop a task from
> > scheduling (unless they need the same lock)  

Every time you add something to allow higher priority processes to run
with less latency you add overhead. By just adding that spinlock check
or to migrate a process to a idle cpu will add a measurable overhead,
and as you state, distros won't like that.

It's a constant game of give and take.

> 
> Running the driver bh (which is often significant) from a high priority
> worker thread instead of a softint (which isn't much different to the
> 'hardint' it is scheduled from) probably doesn't cost much (in-kernel
> process switches shouldn't be much more than a stack switch).
> That would benefit RT processes since they could be higher
> priority than the bh code.
> Although you'd probably want a 'strongly preferred' cpu for them.

BTW, I believe distros compile with "CONFIG_IRQ_FORCED_THREADING" which
means if you add to the kernel command line "threadirqs" the interrupts
will be run as threads. Which allows for even more preemption.

-- Steve