Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20190613032246.GA17752@sinkpad> <CAERHkrsMFjjBpPZS7jDhzbob4PSmiPj83OfqEeiKgaDAU3ajOA@mail.gmail.com>
 <20190619183302.GA6775@sinkpad> <20190718100714.GA469@aaronlu>
 <CAERHkrtvLKxrpvfw04urAuougsYOWnNw4-H1vUDFx27Dvy0=Ww@mail.gmail.com>
 <20190725143003.GA992@aaronlu> <20190726152101.GA27884@sinkpad>
 <7dc86e3c-aa3f-905f-3745-01181a3b0dac@linux.intel.com> <20190802153715.GA18075@sinkpad>
 <f4778816-69e5-146c-2a30-ec42e7f1677f@linux.intel.com> <20190806032418.GA54717@aaronlu>
 <CAERHkrtJ3f1ggfG7Qo-KnznGo66p0Y3E0sAfb3ki6U=ADT6__g@mail.gmail.com> <54fa27ff-69a7-b2ac-6152-6915f78a57f9@linux.alibaba.com>
In-Reply-To: <54fa27ff-69a7-b2ac-6152-6915f78a57f9@linux.alibaba.com>
From:   Vineeth Remanan Pillai <vpillai@digitalocean.com>
Date:   Tue, 6 Aug 2019 08:24:17 -0400
Message-ID: <CANaguZDPdUp3Nb7hYjEiTpJTMVrKJyw2JDKP5EEphMjV-PAYpA@mail.gmail.com>
Subject: Re: [RFC PATCH v3 00/16] Core scheduling v3
To:     Aaron Lu <aaron.lu@linux.alibaba.com>
Cc:     Aubrey Li <aubrey.intel@gmail.com>,
        Tim Chen <tim.c.chen@linux.intel.com>,
        Julien Desfossez <jdesfossez@digitalocean.com>,
        "Li, Aubrey" <aubrey.li@linux.intel.com>,
        Subhra Mazumdar <subhra.mazumdar@oracle.com>,
        Nishanth Aravamudan <naravamudan@digitalocean.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Paul Turner <pjt@google.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Linux List Kernel Mailing <linux-kernel@vger.kernel.org>,
        =?UTF-8?B?RnLDqWTDqXJpYyBXZWlzYmVja2Vy?= <fweisbec@gmail.com>,
        Kees Cook <keescook@chromium.org>,
        Greg Kerr <kerrnel@google.com>, Phil Auld <pauld@redhat.com>,
        Valentin Schneider <valentin.schneider@arm.com>,
        Mel Gorman <mgorman@techsingularity.net>,
        Pawan Gupta <pawan.kumar.gupta@linux.intel.com>,
        Paolo Bonzini <pbonzini@redhat.com>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

> >
> > I also think a way to make fairness per cookie per core, is this what you
> > want to propose?
>
> Yes, that's what I meant.

I think that would hurt some kind of workloads badly, especially if
one tenant is
having way more tasks than the other. Tenant with more task on the same core
might have immediate requirements from some threads than the other and we
would fail to take that into account. With some hierarchical management, we can
alleviate this, but as Aaron said, it would be a bit messy.

Peter's rebalance logic actually takes care of most of the runq
imbalance caused
due to cookie tagging. What we have found from our testing is, fairness issue is
caused mostly due to a Hyperthread going idle and not waking up. Aaron's 3rd
patch works around that. As Julien mentioned, we are working on a per thread
coresched idle thread concept. The problem that we found was, idle thread causes
accounting issues and wakeup issues as it was not designed to be used in this
context. So if we can have a low priority thread which looks like any other task
to the scheduler, things becomes easy for the scheduler and we achieve security
as well. Please share your thoughts on this idea.

The results are encouraging, but we do not yet have the coresched idle
to not spin
100%. We will soon post the patch once it is a bit more stable for
running the tests
that we all have done so far.

Thanks,
Vineeth