Message-ID: <1392971691.5451.84.camel@marge.simpson.net>
Subject: Re: [PATCH v2] sched: keep quiescent cpu out of idle balance loop
From: Mike Galbraith <bitbucket@online.de>
To: Lei Wen <adrian.wenl@gmail.com>
Cc: Lei Wen <leiwen@marvell.com>, Peter Zijlstra <peterz@infradead.org>,
        mingo@redhat.com, preeti.lkml@gmail.com, daniel.lezcano@linaro.org,
        viresh.kumar@linaro.org, xjian@marvell.com,
        linux-kernel@vger.kernel.org
Date: Fri, 21 Feb 2014 09:34:51 +0100
In-Reply-To: <CALZhoSTUZpL5th8HuN_pMe7UHt1PMVO7h9_DsChyz8eW3vctvw@mail.gmail.com>
References: <20140220122311.GO27965@twins.programming.kicks-ass.net>
	 <1392949390-9867-1-git-send-email-leiwen@marvell.com>
	 <1392961903.5451.58.camel@marge.simpson.net>
	 <CALZhoSTUZpL5th8HuN_pMe7UHt1PMVO7h9_DsChyz8eW3vctvw@mail.gmail.com>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit
Mime-Version: 1.0
Sender: linux-kernel-owner@vger.kernel.org

On Fri, 2014-02-21 at 15:28 +0800, Lei Wen wrote:

> Actually, what I have experiment is as:
> 1. set top cpuset as disable load balance
> 2. set 0-2 cpus to "system", and enable its load balance
> 3. set 3 cpu to "rt" and disable load balance.

Exactly as I do, pertinent part of my cheezy script being...

        # ...and fire up the shield
        cset shield --userset=rtcpus --cpu=${START_CPU}-${END_CPU} --kthread=on

        # If cpuset wasn't previously mounted (no obnoxious systemd),
        # we just mounted it.  Find the mount point.
        if [ -z $CPUSET_ROOT ]; then
                CPUSET_ROOT=$(grep cpuset /proc/mounts|cut -d ' ' -f2)
                if [ -z $CPUSET_ROOT ]; then
                        # If it's not mounted now, bail.
                        echo EEK, cupset is not mounted!
                        exit
                else
                        # ok, check for cgroup mount
                        if [ -f ${CPUSET_ROOT}/cpuset.cpus ]; then
                                CPUSET_PREFIX=cpuset.
                        fi
                fi
        fi

        echo 0 > ${CPUSET_ROOT}/${CPUSET_PREFIX}sched_load_balance
        echo 1 > ${CPUSET_ROOT}/system/${CPUSET_PREFIX}sched_load_balance
        echo 0 > ${CPUSET_ROOT}/rtcpus/${CPUSET_PREFIX}sched_load_balance
        echo 0 > ${CPUSET_ROOT}/rtcpus/${CPUSET_PREFIX}sched_relax_domain_level

> While by this way, root span always covering [0-2] which is seen
> by cpu 0-2, as you also mentioned.
> And it is true that if I disable load balance, I would see span mask
> get them merged.
> 
> So how about below change?
> +       if (!this_rq()->sd)
> +               return;
> Suppose isolated cpu would lose its sd, could you help
> confirm it from crash too?

Yeah, isolated CPUs have no sd connectivity.  I sent Peter a patchlet
offline showing what I do to keep nohz at bay.
> Or, you think it is wrong to do merge job when system group disable
> the load balance?

I think the construction stuff works fine, and !->sd is the perfect cue
to tell various things to keep their grubby mitts off of a CPU.

-Mike

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/