Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Tue, 29 Oct 2019 16:34:47 -0400
From:   Julien Desfossez <jdesfossez@digitalocean.com>
To:     Dario Faggioli <dfaggioli@suse.com>
Cc:     Aaron Lu <aaron.lu@linux.alibaba.com>,
        Aubrey Li <aubrey.intel@gmail.com>,
        Tim Chen <tim.c.chen@linux.intel.com>,
        "Li, Aubrey" <aubrey.li@linux.intel.com>,
        Subhra Mazumdar <subhra.mazumdar@oracle.com>,
        Vineeth Remanan Pillai <vpillai@digitalocean.com>,
        Nishanth Aravamudan <naravamudan@digitalocean.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Paul Turner <pjt@google.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Linux List Kernel Mailing <linux-kernel@vger.kernel.org>,
        =?iso-8859-1?Q?Fr=E9d=E9ric?= Weisbecker <fweisbec@gmail.com>,
        Kees Cook <keescook@chromium.org>,
        Greg Kerr <kerrnel@google.com>, Phil Auld <pauld@redhat.com>,
        Valentin Schneider <valentin.schneider@arm.com>,
        Mel Gorman <mgorman@techsingularity.net>,
        Pawan Gupta <pawan.kumar.gupta@linux.intel.com>,
        Paolo Bonzini <pbonzini@redhat.com>
Subject: Re: [RFC PATCH v3 00/16] Core scheduling v3
Message-ID: <20191029203447.GA13345@sinkpad>
References: <eec72c2d533b7600c63de3c8001cc6ab9e915afe.camel@suse.com>
 <69cd9bca-da28-1d35-3913-1efefe0c1c22@linux.intel.com>
 <fab8eabb-1cfa-9bf6-02af-3afdff3f955d@linux.intel.com>
 <20190911140204.GA52872@aaronlu>
 <7b001860-05b4-4308-df0e-8b60037b8000@linux.intel.com>
 <20190912120400.GA16200@aaronlu>
 <CAERHkrsrszO4hJqVy=g7P74h9d_YJzW7GY4ptPKykTX-mc9Mdg@mail.gmail.com>
 <20190915141402.GA1349@aaronlu>
 <277737d6034b3da072d3b0b808d2fa6e110038b0.camel@suse.com>
 <d30d343a8d9bf2c2e2977be7ac8370f26fd4df3d.camel@suse.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <d30d343a8d9bf2c2e2977be7ac8370f26fd4df3d.camel@suse.com>
User-Agent: Mutt/1.9.4 (2018-02-28)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 29-Oct-2019 10:20:57 AM, Dario Faggioli wrote:
> > Hello,
> > 
> > As anticipated, I've been trying to follow the development of this
> > feature and, in the meantime, I have done some benchmarks.
> > 
> > I actually have a lot of data (and am planning for more), so I am
> > sending a few emails, each one with a subset of the numbers in it,
> > instead than just one which would be beyond giant! :-)
> > 

Hi Dario,

Thank you for this comprehensive set of tests and analyses !

It confirms the trend we are seeing for the VM cases. Basically when the
CPUs are overcommitted, core scheduling helps compared to noHT. But when
we have I/O in the mix (sysbench-oltp), then it becomes a bit less
clear, it depends if the CPU is still overcommitted or not. About the
2nd VM that is doing the background noise, is it enough to fill up the
disk queues or is its disk throughput somewhat limited ? Have you
compared the results if you disable the disk noise ?

Our approach for bare-metal tests is a bit different, we are
constraining a set of processes only on a limited set of cpus, but I
like your approach because it pushes more the number of processes
against the whole system. And I have no explanation for why sysbench
thread vs process is so different.

And it also confirms, core scheduling has trouble scaling with the
number of threads, it works pretty well in VMs because the number of
threads is limited by the number of vcpus, but the bare-metal cases show
a major scaling issue (which is not too surprising).

I am curious, for the tagging in KVM, do you move all the vcpus into the
same cgroup before tagging ?  Did you leave the emulator threads
untagged at all time ?

For the overhead (without tagging), have you tried bisecting the
patchset to see which patch introduces the overhead ? it is more than I
had in mind.

And for the cases when core scheduling improves the performance compared
to the baseline numbers, could it be related to frequency scaling (more
work to do means a higher chance of running at a higher frequency) ?

We are almost ready to send the v4 patchset (most likely tomorrow), it
has been rebased on v5.3.5, so stay tuned and ready for another set of
tests ;-)

Thanks,

Julien