Message-ID: <1392961903.5451.58.camel@marge.simpson.net>
Subject: Re: [PATCH v2] sched: keep quiescent cpu out of idle balance loop
From: Mike Galbraith <bitbucket@online.de>
To: Lei Wen <leiwen@marvell.com>
Cc: Peter Zijlstra <peterz@infradead.org>, mingo@redhat.com,
        preeti.lkml@gmail.com, daniel.lezcano@linaro.org,
        viresh.kumar@linaro.org, xjian@marvell.com,
        linux-kernel@vger.kernel.org
Date: Fri, 21 Feb 2014 06:51:43 +0100
In-Reply-To: <1392949390-9867-1-git-send-email-leiwen@marvell.com>
References: <20140220122311.GO27965@twins.programming.kicks-ass.net>
	 <1392949390-9867-1-git-send-email-leiwen@marvell.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0
Sender: linux-kernel-owner@vger.kernel.org

On Fri, 2014-02-21 at 10:23 +0800, Lei Wen wrote: 
> Cpu which is put into quiescent mode, would remove itself
> from kernel's sched_domain, and want others not disturb its
> task running. But current scheduler would not checking whether
> that cpu is setting in such mode, and still insist the quiescent
> cpu to response the nohz load balance.

Let's isolate some CPUs.

Setup:
/-----"system" CPU0
\
  \---"rtcpus" CPUs1-3

crash> runqueues
PER-CPU DATA TYPE:
  struct rq runqueues;
PER-CPU ADDRESSES:
  [0]: ffff88022fc12c00
  [1]: ffff88022fc92c00
  [2]: ffff88022fd12c00
  [3]: ffff88022fd92c00 
crash> struct rq ffff88022fd92c00 | grep sd
  sd = 0x0,  <== yup, CPU3 is isolated bit of silicon
crash> struct rq ffff88022fd92c00 | grep rd
  rd = 0xffffffff81bffe60 <def_root_domain>,
crash> struct -x root_domain 0xffffffff81bffe60
...
  span = {{
      bits = {0xe} <== "rtcpus"
    }},
crash> struct rq ffff88022fc12c00 | grep rd
  rd = 0xffff8802242c5800, 
crash> struct -x root_domain 0xffff8802242c5800
... 
  span = {{
      bits = {0x1} <== "system"
    }},

Turn off load balancing in "system" as well now, CPU0 loses its 'sd',
and becomes an isolated island identical to "rtcpus" CPUs, and thus..

  span = {{
      bits = {0xf} <== oh darn
    }},

.."system" and "rtcpus" merge, all CPUs having NULL sd, as they are now
all remote silicon islands, but they now also share rd again, as if you
had never diddled domains in the first place.

-Mike

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/