Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Tue, 14 Jan 2020 11:59:06 -0500
From:   Steven Rostedt <rostedt@goodmis.org>
To:     David Laight <David.Laight@ACULAB.COM>
Cc:     'Vincent Guittot' <vincent.guittot@linaro.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Viresh Kumar <viresh.kumar@linaro.org>,
        Ingo Molnar <mingo@redhat.com>,
        Juri Lelli <juri.lelli@redhat.com>,
        Dietmar Eggemann <dietmar.eggemann@arm.com>,
        Ben Segall <bsegall@google.com>, Mel Gorman <mgorman@suse.de>,
        linux-kernel <linux-kernel@vger.kernel.org>
Subject: Re: sched/fair: scheduler not running high priority process on idle
 cpu
Message-ID: <20200114115906.22f952ff@gandalf.local.home>
In-Reply-To: <212fabd759b0486aa8df588477acf6d0@AcuMS.aculab.com>
References: <212fabd759b0486aa8df588477acf6d0@AcuMS.aculab.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, 14 Jan 2020 16:50:43 +0000
David Laight <David.Laight@ACULAB.COM> wrote:

> I've a test that uses four RT priority processes to process audio data every 10ms.
> One process wakes up the other three, they all 'beaver away' clearing a queue of
> jobs and the last one to finish sleeps until the next tick.
> Usually this takes about 0.5ms, but sometimes takes over 3ms.
> 
> AFAICT the processes are normally woken on the same cpu they last ran on.
> There seems to be a problem when the selected cpu is running a (low priority)
> process that is looping in kernel [1].
> I'd expect my process to be picked up by one of the idle cpus, but this
> doesn't happen.
> Instead the process sits in state 'waiting' until the active processes sleeps
> (or calls cond_resched()).
> 
> Is this really the expected behaviour?????

It is with CONFIG_PREEMPT_VOLUNTARY. I think you want to recompile your
kernel with CONFIG_PREEMPT. The idea is that the RT task will continue
to run on the CPU it last ran on, and would push off the lower priority
task to the idle CPU. But CONFIG_PREEMPT_VOLUNTARY means that this
will have to wait for the running task to not be in kernel context or
hit a cond_resched() which is the "voluntary" scheduling point.

-- Steve