Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
DMARC-Filter: OpenDMARC Filter v1.3.2 smtp.codeaurora.org 44B0CC433C8
Date:   Wed, 24 Jun 2020 19:48:44 +0530
From:   Pavan Kondeti <pkondeti@codeaurora.org>
To:     Vincent Guittot <vincent.guittot@linaro.org>
Cc:     Peter Zijlstra <peterz@infradead.org>,
        LKML <linux-kernel@vger.kernel.org>, Lei Wen <leiwen@marvell.com>
Subject: Re: Looping more in detach_tasks() when RT and CFS tasks are present
Message-ID: <20200624141844.GB25985@codeaurora.org>
References: <CAEU1=PkNtyznCms3jjp-oZHW=UAinnNKqG144VuzO5M7MLkO3Q@mail.gmail.com>
 <CAKfTPtCVF-pw0cjuivuirNpBHRXZNJLihhJtGgkuTi7VdFpWbQ@mail.gmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CAKfTPtCVF-pw0cjuivuirNpBHRXZNJLihhJtGgkuTi7VdFpWbQ@mail.gmail.com>
User-Agent: Mutt/1.5.24 (2015-08-30)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Hi Vincent,

On Wed, Jun 24, 2020 at 02:39:25PM +0200, Vincent Guittot wrote:
> Hi Pavan,
> 
> On Wed, 24 Jun 2020 at 13:42, Pavan Kondeti <pkondeti@codeaurora.org> wrote:
> >
> > Hi Vincent/Peter,
> >
> > in load_balance(), we derive env->loop_max based on rq->nr_running.
> > When the busiest rq has both RT and CFS tasks, we do more loops in
> > detach_tasks(). Is there any reason for not using
> > rq->cfs.h_nr_running?
> 
> Using cfs.h_nr_running seems fine for loop_max
> 

Thanks for taking a look.

> >
> > Lei Wen attempted to fix this before.
> > https://lore.kernel.org/lkml/1376814322-7320-2-git-send-email-leiwen@marvell.com/
> 
> The 1st part of the patch is wrong because even if h_nr_running == 1
> but nr_running > 1, we can pull the cfs thread without using active
> balance
> 

Right. When a RT and CFS tasks are packed, I have seen CFS task getting pulled
via load balancer without waking migration/X.

I was using the below patch along with some prints in detach_tasks() loop.
I will update Lei Wen's patch and resend it.

diff --git a/kernel/sched/fair.c b/kernel/sched/fair.c
index 02f323b..f042016 100644
--- a/kernel/sched/fair.c
+++ b/kernel/sched/fair.c
@@ -9509,7 +9509,7 @@ static int load_balance(int this_cpu, struct rq *this_rq,
 		 * correctly treated as an imbalance.
 		 */
 		env.flags |= LBF_ALL_PINNED;
-		env.loop_max  = min(sysctl_sched_nr_migrate, busiest->nr_running);
+		env.loop_max  = min(sysctl_sched_nr_migrate, busiest->cfs.h_nr_running);
 
 more_balance:
 		rq_lock_irqsave(busiest, &rf);

Thanks,
Pavan
-- 
Qualcomm India Private Limited, on behalf of Qualcomm Innovation Center, Inc.
Qualcomm Innovation Center, Inc. is a member of Code Aurora Forum, a Linux Foundation Collaborative Project.