On Fri, Jun 06, 2014 at 02:59:56PM +0800, [email protected] wrote:
> Hi all
>
> When computing cache hot, question-1: should we check the number of running tasks
> against the migration target runqueue, as shown by the following diff?
>
> It looks that we dont migrate task if it is buddy and the target cpu is not idle.
>
>
> --- a/kernel/sched/fair.c Fri Jun 6 12:37:37 2014
> +++ b/kernel/sched/fair.c Fri Jun 6 13:55:35 2014
> @@ -5051,7 +5050,7 @@ task_hot(struct task_struct *p, u64 now)
> /*
> * Buddy candidates are cache hot:
> */
> - if (sched_feat(CACHE_HOT_BUDDY) && this_rq()->nr_running &&
> + if (sched_feat(CACHE_HOT_BUDDY) && env->dst_rq->nr_running &&
> (&p->se == cfs_rq_of(&p->se)->next ||
> &p->se == cfs_rq_of(&p->se)->last))
> return 1;
That is indeed the effective result for normal balancing, seeing how it
pulls to the dst rq, and this_rq would be dst.
This is of course false for a number of cases these days, most obviously
the nohz idle balancing.
> But based on the comment, question-2: should we check running tasks
> against the runqueue of the given task?
> --- a/kernel/sched/fair.c Fri Jun 6 12:37:37 2014
> +++ b/kernel/sched/fair.c Fri Jun 6 14:32:34 2014
> @@ -5051,7 +5051,7 @@ task_hot(struct task_struct *p, u64 now)
> /*
> * Buddy candidates are cache hot:
> */
> - if (sched_feat(CACHE_HOT_BUDDY) && this_rq()->nr_running &&
> + if (sched_feat(CACHE_HOT_BUDDY) && task_rq(p)->nr_running &&
> (&p->se == cfs_rq_of(&p->se)->next ||
> &p->se == cfs_rq_of(&p->se)->last))
> return 1;
That does appear to make more sense indeed, seeing how buddies are pairs
of tasks, so protecting a lone task doesn't make sense.
Mike, how did you intend this code to work?
On Fri, 2014-06-06 at 12:15 +0200, Peter Zijlstra wrote:
> > --- a/kernel/sched/fair.c Fri Jun 6 12:37:37 2014
> > +++ b/kernel/sched/fair.c Fri Jun 6 14:32:34 2014
> > @@ -5051,7 +5051,7 @@ task_hot(struct task_struct *p, u64 now)
> > /*
> > * Buddy candidates are cache hot:
> > */
> > - if (sched_feat(CACHE_HOT_BUDDY) && this_rq()->nr_running &&
> > + if (sched_feat(CACHE_HOT_BUDDY) && task_rq(p)->nr_running &&
> > (&p->se == cfs_rq_of(&p->se)->next ||
> > &p->se == cfs_rq_of(&p->se)->last))
> > return 1;
>
> That does appear to make more sense indeed, seeing how buddies are pairs
> of tasks, so protecting a lone task doesn't make sense.
>
>
> Mike, how did you intend this code to work?
IIRC, this_rq()->nr_running was to say if we're idle, we don't care that
it's last/next, pull it. Not sure I'm the one who did that, but could
be, I didn't look.
-Mike
On Fri, Jun 06, 2014 at 01:16:23PM +0200, Mike Galbraith wrote:
> On Fri, 2014-06-06 at 12:15 +0200, Peter Zijlstra wrote:
>
> > > --- a/kernel/sched/fair.c Fri Jun 6 12:37:37 2014
> > > +++ b/kernel/sched/fair.c Fri Jun 6 14:32:34 2014
> > > @@ -5051,7 +5051,7 @@ task_hot(struct task_struct *p, u64 now)
> > > /*
> > > * Buddy candidates are cache hot:
> > > */
> > > - if (sched_feat(CACHE_HOT_BUDDY) && this_rq()->nr_running &&
> > > + if (sched_feat(CACHE_HOT_BUDDY) && task_rq(p)->nr_running &&
> > > (&p->se == cfs_rq_of(&p->se)->next ||
> > > &p->se == cfs_rq_of(&p->se)->last))
> > > return 1;
> >
> > That does appear to make more sense indeed, seeing how buddies are pairs
> > of tasks, so protecting a lone task doesn't make sense.
> >
> >
> > Mike, how did you intend this code to work?
>
> IIRC, this_rq()->nr_running was to say if we're idle, we don't care that
> it's last/next, pull it. Not sure I'm the one who did that, but could
> be, I didn't look.
>
commit f685ceacab07d3f6c236f04803e2f2f0dbcc5afb
Author: Mike Galbraith <[email protected]>
Date: Fri Oct 23 23:09:22 2009 +0200
sched: Strengthen buddies and mitigate buddy induced latencies
...
- if (sched_feat(CACHE_HOT_BUDDY) &&
+ if (sched_feat(CACHE_HOT_BUDDY) && this_rq()->nr_running &&
(&p->se == cfs_rq_of(&p->se)->next ||
&p->se == cfs_rq_of(&p->se)->last))
Yeah, was you ;-)
OK, so we want dst_rq. Thanks!
On Fri, 2014-06-06 at 13:34 +0200, Peter Zijlstra wrote:
> On Fri, Jun 06, 2014 at 01:16:23PM +0200, Mike Galbraith wrote:
> > On Fri, 2014-06-06 at 12:15 +0200, Peter Zijlstra wrote:
> >
> > > > --- a/kernel/sched/fair.c Fri Jun 6 12:37:37 2014
> > > > +++ b/kernel/sched/fair.c Fri Jun 6 14:32:34 2014
> > > > @@ -5051,7 +5051,7 @@ task_hot(struct task_struct *p, u64 now)
> > > > /*
> > > > * Buddy candidates are cache hot:
> > > > */
> > > > - if (sched_feat(CACHE_HOT_BUDDY) && this_rq()->nr_running &&
> > > > + if (sched_feat(CACHE_HOT_BUDDY) && task_rq(p)->nr_running &&
> > > > (&p->se == cfs_rq_of(&p->se)->next ||
> > > > &p->se == cfs_rq_of(&p->se)->last))
> > > > return 1;
> > >
> > > That does appear to make more sense indeed, seeing how buddies are pairs
> > > of tasks, so protecting a lone task doesn't make sense.
> > >
> > >
> > > Mike, how did you intend this code to work?
> >
> > IIRC, this_rq()->nr_running was to say if we're idle, we don't care that
> > it's last/next, pull it. Not sure I'm the one who did that, but could
> > be, I didn't look.
> >
>
> commit f685ceacab07d3f6c236f04803e2f2f0dbcc5afb
> Author: Mike Galbraith <[email protected]>
> Date: Fri Oct 23 23:09:22 2009 +0200
>
> sched: Strengthen buddies and mitigate buddy induced latencies
>
> ...
>
> - if (sched_feat(CACHE_HOT_BUDDY) &&
> + if (sched_feat(CACHE_HOT_BUDDY) && this_rq()->nr_running &&
> (&p->se == cfs_rq_of(&p->se)->next ||
> &p->se == cfs_rq_of(&p->se)->last))
>
> Yeah, was you ;-)
Last hunk prevents buddies from stymieing BALANCE_NEWIDLE via
CACHE_HOT_BUDDY.
Last hunk, first hunk, whatever, that's what it was for :)
-Mike