Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751867AbdF3Iv7 (ORCPT ); Fri, 30 Jun 2017 04:51:59 -0400 Received: from mail-qk0-f169.google.com ([209.85.220.169]:34769 "EHLO mail-qk0-f169.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751668AbdF3Iv4 (ORCPT ); Fri, 30 Jun 2017 04:51:56 -0400 Subject: Re: [PATCH V2 2/2] rt: Increase/decrease the nr of migratory tasks when enabling/disabling migration To: Ingo Molnar , Daniel Bristot de Oliveira Cc: linux-rt-users , "Luis Claudio R . Goncalves" , Clark Williams , Luiz Capitulino , Sebastian Andrzej Siewior , Thomas Gleixner , Steven Rostedt , Peter Zijlstra , LKML References: <20170630073055.yp7x55zavoxlaxfl@gmail.com> From: Daniel Bristot de Oliveira Message-ID: <2ca2100a-334b-05af-8328-025ea9cb182f@redhat.com> Date: Fri, 30 Jun 2017 10:51:49 +0200 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Thunderbird/52.1.0 MIME-Version: 1.0 In-Reply-To: <20170630073055.yp7x55zavoxlaxfl@gmail.com> Content-Type: text/plain; charset=utf-8 Content-Language: en-US Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4264 Lines: 106 On 06/30/2017 09:30 AM, Ingo Molnar wrote: > > * Daniel Bristot de Oliveira wrote: > >> There is a problem in the migrate_disable()/enable() implementation >> regarding the number of migratory tasks in the rt/dl RQs. The problem >> is the following: >> >> When a task is attached to the rt runqueue, it is checked if it either >> can run in more than one CPU, or if it is with migration disable. If >> either check is true, the rt_rq->rt_nr_migratory counter is not >> increased. The counter increases otherwise. >> >> When the task is detached, the same check is done. If either check is >> true, the rt_rq->rt_nr_migratory counter is not decreased. The counter >> decreases otherwise. The same check is done in the dl scheduler. >> >> One important thing is that, migrate disable/enable does not touch this >> counter for tasks attached to the rt rq. So suppose the following chain >> of events. >> >> Assumptions: >> Task A is the only runnable task in A Task B runs on the CPU B >> Task A runs on CFS (non-rt) Task B has RT priority >> Thus, rt_nr_migratory is 0 B is running >> Task A can run on all CPUS. >> >> Timeline: >> CPU A/TASK A CPU B/TASK B >> A takes the rt mutex X . >> A disables migration . >> . B tries to take the rt mutex X >> . As it is held by A { >> . A inherits the rt priority of B >> . A is dequeued from CFS RQ of CPU A >> . A is enqueued in the RT RQ of CPU A >> . As migration is disabled >> . rt_nr_migratory in A is not increased >> . >> A enables migration >> A releases the rt mutex X { >> A returns to its original priority >> A ask to be dequeued from RT RQ { >> As migration is now enabled and it can run on all CPUS { >> rt_nr_migratory should be decreased >> As rt_nr_migratory is 0, rt_nr_migratory under flows >> } >> } >> >> This variable is important because it notifies if there are more than one >> runnable & migratory task in the runqueue. If there are more than one >> tasks, the rt_rq is set as overloaded, and then tries to migrate some >> tasks. This rule is important to keep the scheduler working conserving, >> that is, in a system with M CPUs, the M highest priority tasks should be >> running. >> >> As rt_nr_migratory is unsigned, it will become > 0, notifying that the >> RQ is overloaded, activating pushing mechanism without need. >> >> This patch fixes this problem by decreasing/increasing the >> rt/dl_nr_migratory in the migrate disable/enable operations. >> >> Reported-by: Pei Zhang >> Reported-by: Luiz Capitulino >> Signed-off-by: Daniel Bristot de Oliveira >> Cc: Luis Claudio R. Goncalves >> Cc: Clark Williams >> Cc: Luiz Capitulino >> Cc: Sebastian Andrzej Siewior >> Cc: Thomas Gleixner >> Cc: Steven Rostedt >> Cc: Peter Zijlstra >> Cc: Ingo Molnar >> Cc: LKML >> Cc: linux-rt-users >> --- >> kernel/sched/core.c | 49 ++++++++++++++++++++++++++++++++++++++++++++----- >> 1 file changed, 44 insertions(+), 5 deletions(-) > > This second patch does not apply to the latest scheduler tree (tip:master) cleanly > - which tree is it against? Hi Ingo, migrate_disable/enable() are PREEMPT_RT specific, so the patch 2/2 is addressed only to the PREEMPT_RT patch set. I was working in the 4.11-rt tree. The first one is not -rt specific, though. Sorry for a possible miss communication... -- Daniel > Thanks, > > Ingo > -- > To unsubscribe from this list: send the line "unsubscribe linux-rt-users" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html >