Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Tue, 6 Aug 2019 19:12:41 +0200
From:   Peter Zijlstra <peterz@infradead.org>
To:     Tim Chen <tim.c.chen@linux.intel.com>
Cc:     Aaron Lu <aaron.lu@linux.alibaba.com>,
        Julien Desfossez <jdesfossez@digitalocean.com>,
        "Li, Aubrey" <aubrey.li@linux.intel.com>,
        Aubrey Li <aubrey.intel@gmail.com>,
        Subhra Mazumdar <subhra.mazumdar@oracle.com>,
        Vineeth Remanan Pillai <vpillai@digitalocean.com>,
        Nishanth Aravamudan <naravamudan@digitalocean.com>,
        Ingo Molnar <mingo@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Paul Turner <pjt@google.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Linux List Kernel Mailing <linux-kernel@vger.kernel.org>,
        =?iso-8859-1?Q?Fr=E9d=E9ric?= Weisbecker <fweisbec@gmail.com>,
        Kees Cook <keescook@chromium.org>,
        Greg Kerr <kerrnel@google.com>, Phil Auld <pauld@redhat.com>,
        Valentin Schneider <valentin.schneider@arm.com>,
        Mel Gorman <mgorman@techsingularity.net>,
        Pawan Gupta <pawan.kumar.gupta@linux.intel.com>,
        Paolo Bonzini <pbonzini@redhat.com>
Subject: Re: [RFC PATCH v3 00/16] Core scheduling v3
Message-ID: <20190806171241.GQ2349@hirez.programming.kicks-ass.net>
References: <20190619183302.GA6775@sinkpad>
 <20190718100714.GA469@aaronlu>
 <CAERHkrtvLKxrpvfw04urAuougsYOWnNw4-H1vUDFx27Dvy0=Ww@mail.gmail.com>
 <20190725143003.GA992@aaronlu>
 <20190726152101.GA27884@sinkpad>
 <7dc86e3c-aa3f-905f-3745-01181a3b0dac@linux.intel.com>
 <20190802153715.GA18075@sinkpad>
 <f4778816-69e5-146c-2a30-ec42e7f1677f@linux.intel.com>
 <20190806032418.GA54717@aaronlu>
 <e1c4a7ed-822e-93cb-ff1d-6a0842db115f@linux.intel.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <e1c4a7ed-822e-93cb-ff1d-6a0842db115f@linux.intel.com>
User-Agent: Mutt/1.10.1 (2018-07-13)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Tue, Aug 06, 2019 at 10:03:29AM -0700, Tim Chen wrote:
> On 8/5/19 8:24 PM, Aaron Lu wrote:
> 
> > I've been thinking if we should consider core wide tenent fairness?
> > 
> > Let's say there are 3 tasks on 2 threads' rq of the same core, 2 tasks
> > (e.g. A1, A2) belong to tenent A and the 3rd B1 belong to another tenent
> > B. Assume A1 and B1 are queued on the same thread and A2 on the other
> > thread, when we decide priority for A1 and B1, shall we also consider
> > A2's vruntime? i.e. shall we consider A1 and A2 as a whole since they
> > belong to the same tenent? I tend to think we should make fairness per
> > core per tenent, instead of per thread(cpu) per task(sched entity). What
> > do you guys think?
> > 
> > Implemention of the idea is a mess to me, as I feel I'm duplicating the
> > existing per cpu per sched_entity enqueue/update vruntime/dequeue logic
> > for the per core per tenent stuff.
> 
> I'm wondering if something simpler will work.  It is easier to maintain fairness
> between the CPU threads.  A simple scheme may be if the force idle deficit
> on a CPU thread exceeds a threshold compared to its sibling, we will
> bias in choosing the task on the suppressed CPU thread.  
> The fairness among the tenents per run queue is balanced out by cfq fairness,
> so things should be fair if we maintain fairness in CPU utilization between
> the two CPU threads.

IIRC pjt once did a simle 5ms flip flop between siblings.