Hi all,
I've been using a homemade test for full dynticks all that time and I regret I haven't shared
it much sooner. I've been asked for such a tool many times and getting full dynticks working
correctly is often not a piece of cake.
Here it is, a very basic test that runs a userspace loop for ten seconds and dumps a trace
of the tick, scheduler, workqueue, and other kind of kernel noise.
You can fetch from:
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git
master
I hope it helps! Feel free to send any kind of improvements if you wish.
Thanks!
On Wed, 12 Jun 2013, Frederic Weisbecker wrote:
> Here it is, a very basic test that runs a userspace loop for ten seconds and dumps a trace
> of the tick, scheduler, workqueue, and other kind of kernel noise.
Excellent. Ticks are finally off here. Also has useful instructions on how
to figure out if something was wrong. The number of >1 microsecond
disturbances went down from 110 per second to 2.
On Tue, Jun 18, 2013 at 02:20:35PM +0000, Christoph Lameter wrote:
> On Wed, 12 Jun 2013, Frederic Weisbecker wrote:
>
> > Here it is, a very basic test that runs a userspace loop for ten seconds and dumps a trace
> > of the tick, scheduler, workqueue, and other kind of kernel noise.
>
> Excellent. Ticks are finally off here. Also has useful instructions on how
> to figure out if something was wrong. The number of >1 microsecond
> disturbances went down from 110 per second to 2.
Very cool!
Frederic, is the git tree mentioned in your email permanent? If so, I
will add it to the documentation.
Thanx, Paul
On Tue, Jun 18, 2013 at 08:15:20AM -0700, Paul E. McKenney wrote:
> On Tue, Jun 18, 2013 at 02:20:35PM +0000, Christoph Lameter wrote:
> > On Wed, 12 Jun 2013, Frederic Weisbecker wrote:
> >
> > > Here it is, a very basic test that runs a userspace loop for ten seconds and dumps a trace
> > > of the tick, scheduler, workqueue, and other kind of kernel noise.
> >
> > Excellent. Ticks are finally off here. Also has useful instructions on how
> > to figure out if something was wrong. The number of >1 microsecond
> > disturbances went down from 110 per second to 2.
>
> Very cool!
>
> Frederic, is the git tree mentioned in your email permanent? If so, I
> will add it to the documentation.
Yep, it should be permanent.
Thanks!
On Tue, Jun 18, 2013 at 06:22:57PM +0200, Frederic Weisbecker wrote:
> On Tue, Jun 18, 2013 at 08:15:20AM -0700, Paul E. McKenney wrote:
> > On Tue, Jun 18, 2013 at 02:20:35PM +0000, Christoph Lameter wrote:
> > > On Wed, 12 Jun 2013, Frederic Weisbecker wrote:
> > >
> > > > Here it is, a very basic test that runs a userspace loop for ten seconds and dumps a trace
> > > > of the tick, scheduler, workqueue, and other kind of kernel noise.
> > >
> > > Excellent. Ticks are finally off here. Also has useful instructions on how
> > > to figure out if something was wrong. The number of >1 microsecond
> > > disturbances went down from 110 per second to 2.
> >
> > Very cool!
> >
> > Frederic, is the git tree mentioned in your email permanent? If so, I
> > will add it to the documentation.
>
> Yep, it should be permanent.
How about the following?
Thanx, Paul
------------------------------------------------------------------------
nohz_full: Add testing information to documentation
This commit adds information about testing nohz_full, and also emphasizes
the fact that you need a multi-CPU system to get any benefit from nohz_full.
Signed-off-by: Paul E. McKenney <[email protected]>
Cc: Frederic Weisbecker <[email protected]>
diff --git a/Documentation/timers/NO_HZ.txt b/Documentation/timers/NO_HZ.txt
index 8869758..cca122f 100644
--- a/Documentation/timers/NO_HZ.txt
+++ b/Documentation/timers/NO_HZ.txt
@@ -24,8 +24,8 @@ There are three main ways of managing scheduling-clock interrupts
workloads, you will normally -not- want this option.
These three cases are described in the following three sections, followed
-by a third section on RCU-specific considerations and a fourth and final
-section listing known issues.
+by a third section on RCU-specific considerations, a fourth section
+discussing testing, and a fifth and final section listing known issues.
NEVER OMIT SCHEDULING-CLOCK TICKS
@@ -121,14 +121,15 @@ boot parameter specifies the adaptive-ticks CPUs. For example,
"nohz_full=1,6-8" says that CPUs 1, 6, 7, and 8 are to be adaptive-ticks
CPUs. Note that you are prohibited from marking all of the CPUs as
adaptive-tick CPUs: At least one non-adaptive-tick CPU must remain
-online to handle timekeeping tasks in order to ensure that system calls
-like gettimeofday() returns accurate values on adaptive-tick CPUs.
-(This is not an issue for CONFIG_NO_HZ_IDLE=y because there are no
-running user processes to observe slight drifts in clock rate.)
-Therefore, the boot CPU is prohibited from entering adaptive-ticks
-mode. Specifying a "nohz_full=" mask that includes the boot CPU will
-result in a boot-time error message, and the boot CPU will be removed
-from the mask.
+online to handle timekeeping tasks in order to ensure that system
+calls like gettimeofday() returns accurate values on adaptive-tick CPUs.
+(This is not an issue for CONFIG_NO_HZ_IDLE=y because there are no running
+user processes to observe slight drifts in clock rate.) Therefore, the
+boot CPU is prohibited from entering adaptive-ticks mode. Specifying a
+"nohz_full=" mask that includes the boot CPU will result in a boot-time
+error message, and the boot CPU will be removed from the mask. Note that
+this means that your system must have at least two CPUs in order for
+CONFIG_NO_HZ_FULL=y to do anything for you.
Alternatively, the CONFIG_NO_HZ_FULL_ALL=y Kconfig parameter specifies
that all CPUs other than the boot CPU are adaptive-ticks CPUs. This
@@ -232,6 +233,29 @@ scheduler will decide where to run them, which might or might not be
where you want them to run.
+TESTING
+
+So you enable all the OS-jitter features described in this document,
+but do not see any change in your workload's behavior. Is this because
+your workload isn't affected that much by OS jitter, or is it because
+something else is in the way? This section helps answer this question
+by providing a simple OS-jitter test suite, which is available on branch
+master of the following git archive:
+
+git://git.kernel.org/pub/scm/linux/kernel/git/frederic/dynticks-testing.git
+
+Clone this archive and follow the instructions in the README file.
+This test procedure will produce a trace that will allow you to evaluate
+whether or not you have succeeded in removing OS jitter from your system.
+If this trace shows that you have removed OS jitter as much as is
+possible, then you can conclude that your workload is not all that
+sensitive to OS jitter.
+
+Note: this test requires that your system have at least two CPUs.
+We do not currently have a good way to remove OS jitter from single-CPU
+systems.
+
+
KNOWN ISSUES
o Dyntick-idle slows transitions to and from idle slightly.
On Tue, 18 Jun 2013, Paul E. McKenney wrote:
> +online to handle timekeeping tasks in order to ensure that system
> +calls like gettimeofday() returns accurate values on adaptive-tick CPUs.
> +(This is not an issue for CONFIG_NO_HZ_IDLE=y because there are no running
> +user processes to observe slight drifts in clock rate.) Therefore, the
> +boot CPU is prohibited from entering adaptive-ticks mode. Specifying a
> +"nohz_full=" mask that includes the boot CPU will result in a boot-time
> +error message, and the boot CPU will be removed from the mask. Note that
> +this means that your system must have at least two CPUs in order for
> +CONFIG_NO_HZ_FULL=y to do anything for you.
These are some important point here. Very good. Thanks.
On Tue, Jun 18, 2013 at 11:17:33AM -0700, Paul E. McKenney wrote:
> On Tue, Jun 18, 2013 at 06:22:57PM +0200, Frederic Weisbecker wrote:
> > On Tue, Jun 18, 2013 at 08:15:20AM -0700, Paul E. McKenney wrote:
> > > On Tue, Jun 18, 2013 at 02:20:35PM +0000, Christoph Lameter wrote:
> > > > On Wed, 12 Jun 2013, Frederic Weisbecker wrote:
> > > >
> > > > > Here it is, a very basic test that runs a userspace loop for ten seconds and dumps a trace
> > > > > of the tick, scheduler, workqueue, and other kind of kernel noise.
> > > >
> > > > Excellent. Ticks are finally off here. Also has useful instructions on how
> > > > to figure out if something was wrong. The number of >1 microsecond
> > > > disturbances went down from 110 per second to 2.
> > >
> > > Very cool!
> > >
> > > Frederic, is the git tree mentioned in your email permanent? If so, I
> > > will add it to the documentation.
> >
> > Yep, it should be permanent.
>
> How about the following?
>
> Thanx, Paul
>
> ------------------------------------------------------------------------
>
> nohz_full: Add testing information to documentation
>
> This commit adds information about testing nohz_full, and also emphasizes
> the fact that you need a multi-CPU system to get any benefit from nohz_full.
>
> Signed-off-by: Paul E. McKenney <[email protected]>
> Cc: Frederic Weisbecker <[email protected]>
Nice!
Acked-by: Frederic Weisbecker <[email protected]>