Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Tue, 2 Jul 2019 19:28:51 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     subhra mazumdar <subhra.mazumdar@oracle.com>
Cc:     linux-kernel@vger.kernel.org, mingo@redhat.com, tglx@linutronix.de,
        prakash.sangappa@oracle.com, dhaval.giani@oracle.com,
        daniel.lezcano@linaro.org, vincent.guittot@linaro.org,
        viresh.kumar@linaro.org, tim.c.chen@linux.intel.com,
        mgorman@techsingularity.net, Paul Turner <pjt@google.com>
Subject: Re: [RFC PATCH 2/3] sched: change scheduler to give preference to
 soft affinity CPUs
Message-ID: <20190702172851.GA3436@hirez.programming.kicks-ass.net>
References: <20190626224718.21973-1-subhra.mazumdar@oracle.com>
 <20190626224718.21973-3-subhra.mazumdar@oracle.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20190626224718.21973-3-subhra.mazumdar@oracle.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, Jun 26, 2019 at 03:47:17PM -0700, subhra mazumdar wrote:
> The soft affinity CPUs present in the cpumask cpus_preferred is used by the
> scheduler in two levels of search. First is in determining wake affine
> which choses the LLC domain and secondly while searching for idle CPUs in
> LLC domain. In the first level it uses cpus_preferred to prune out the
> search space. In the second level it first searches the cpus_preferred and
> then cpus_allowed. Using affinity_unequal flag it breaks early to avoid
> any overhead in the scheduler fast path when soft affinity is not used.
> This only changes the wake up path of the scheduler, the idle balancing
> is unchanged; together they achieve the "softness" of scheduling.

I really dislike this implementation.

I thought the idea was to remain work conserving (in so far as that
we're that anyway), so changing select_idle_sibling() doesn't make sense
to me. If there is idle, we use it.

Same for newidle; which you already retained.

This then leaves regular balancing, and for that we can fudge with
can_migrate_task() and nr_balance_failed or something.

And I also really don't want a second utilization tipping point; we
already have the overloaded thing.

I also still dislike how you never looked into the numa balancer, which
already has peferred_nid stuff.