Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <cover.1559129225.git.vpillai@digitalocean.com>
 <CAERHkruDE-7R5K=2yRqCJRCpV87HkHzDYbQA2WQkruVYpG7t7Q@mail.gmail.com>
 <e8872bd9-1c6b-fb12-b535-3d37740a0306@linux.alibaba.com> <CAERHkruVthrTgGqiH=yhCspZWpMDAu0G4rNcrDTKQzgPq9eqQQ@mail.gmail.com>
 <21fda627-1d3c-12cc-6389-8c226218e2ce@linux.alibaba.com>
In-Reply-To: <21fda627-1d3c-12cc-6389-8c226218e2ce@linux.alibaba.com>
From:   Aubrey Li <aubrey.intel@gmail.com>
Date:   Fri, 31 May 2019 14:53:21 +0800
Message-ID: <CAERHkrsjTNUssmECksdJnLdDjVJ_9UpvYiHyBDJ3S1toURCbrw@mail.gmail.com>
Subject: Re: [RFC PATCH v3 00/16] Core scheduling v3
To:     Aaron Lu <aaron.lu@linux.alibaba.com>
Cc:     Vineeth Remanan Pillai <vpillai@digitalocean.com>,
        Nishanth Aravamudan <naravamudan@digitalocean.com>,
        Julien Desfossez <jdesfossez@digitalocean.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Tim Chen <tim.c.chen@linux.intel.com>,
        Ingo Molnar <mingo@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Paul Turner <pjt@google.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Linux List Kernel Mailing <linux-kernel@vger.kernel.org>,
        Subhra Mazumdar <subhra.mazumdar@oracle.com>,
        =?UTF-8?B?RnLDqWTDqXJpYyBXZWlzYmVja2Vy?= <fweisbec@gmail.com>,
        Kees Cook <keescook@chromium.org>,
        Greg Kerr <kerrnel@google.com>, Phil Auld <pauld@redhat.com>,
        Valentin Schneider <valentin.schneider@arm.com>,
        Mel Gorman <mgorman@techsingularity.net>,
        Pawan Gupta <pawan.kumar.gupta@linux.intel.com>,
        Paolo Bonzini <pbonzini@redhat.com>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Fri, May 31, 2019 at 2:09 PM Aaron Lu <aaron.lu@linux.alibaba.com> wrote:
>
> On 2019/5/31 13:12, Aubrey Li wrote:
> > On Fri, May 31, 2019 at 11:01 AM Aaron Lu <aaron.lu@linux.alibaba.com> wrote:
> >>
> >> This feels like "date" failed to schedule on some CPU
> >> on time.
> >>
> >> My first reaction is: when shell wakes up from sleep, it will
> >> fork date. If the script is untagged and those workloads are
> >> tagged and all available cores are already running workload
> >> threads, the forked date can lose to the running workload
> >> threads due to __prio_less() can't properly do vruntime comparison
> >> for tasks on different CPUs. So those idle siblings can't run
> >> date and are idled instead. See my previous post on this:
> >> https://lore.kernel.org/lkml/20190429033620.GA128241@aaronlu/
> >> (Now that I re-read my post, I see that I didn't make it clear
> >> that se_bash and se_hog are assigned different tags(e.g. hog is
> >> tagged and bash is untagged).
> >
> > Yes, script is untagged. This looks like exactly the problem in you
> > previous post. I didn't follow that, does that discussion lead to a solution?
>
> No immediate solution yet.
>
> >>
> >> Siblings being forced idle is expected due to the nature of core
> >> scheduling, but when two tasks belonging to two siblings are
> >> fighting for schedule, we should let the higher priority one win.
> >>
> >> It used to work on v2 is probably due to we mistakenly
> >> allow different tagged tasks to schedule on the same core at
> >> the same time, but that is fixed in v3.
> >
> > I have 64 threads running on a 104-CPU server, that is, when the
>
> 104-CPU means 52 cores I guess.
> 64 threads may(should?) spread on all the 52 cores and that is enough
> to make 'date' suffer.

64 threads should spread onto all the 52 cores, but why they can get
scheduled while untagged "date" can not? Is it because in the current
implementation the task with cookie always has higher priority than the
task without a cookie?

Thanks,
-Aubrey