Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Date:   Wed, 24 Jan 2018 10:46:08 -0500
From:   Luiz Capitulino <lcapitulino@redhat.com>
To:     Frederic Weisbecker <frederic@kernel.org>
Cc:     Ingo Molnar <mingo@kernel.org>,
        LKML <linux-kernel@vger.kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Chris Metcalf <cmetcalf@mellanox.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Christoph Lameter <cl@linux.com>,
        "Paul E . McKenney" <paulmck@linux.vnet.ibm.com>,
        Wanpeng Li <kernellwp@gmail.com>,
        Mike Galbraith <efault@gmx.de>, Rik van Riel <riel@redhat.com>
Subject: Re: [GIT PULL] isolation: 1Hz residual tick offloading v4
Message-ID: <20180124104608.038fb212@redhat.com>
In-Reply-To: <1516320140-13189-1-git-send-email-frederic@kernel.org>
References: <1516320140-13189-1-git-send-email-frederic@kernel.org>
Organization: Red Hat
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Fri, 19 Jan 2018 01:02:14 +0100
Frederic Weisbecker <frederic@kernel.org> wrote:

> Ingo,
> 
> Please pull the sched/0hz-v2 branch that can be found at:
> 
> git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
> 	sched/0hz-v2
> 
> HEAD: 9b14d5204490f9acd03998a5e406ecadb87cddba
> 
> Changes in v4:
> 
> * Remove the nohz_offload option, just stick with the existing interface,
>   the change is transparent. Suggested by Luiz.
> 
> * Automatically pin workqueues to housekeepers.

I've been testing this series and the tick doesn't go completely away
for me: it ticks at around 8 seconds interval.

I've debugged this down to the clocksource_watchdog() timer, which is
created by clocksource_start_watchdog(). This timer cycles over all online
CPUs. I couldn't find a way to disable it. It seems to be always enabled
for x86 by CONFIG_CLOCKSOURCE_WATCHDOG since commit 6471b825c4.

Since the 1Hz tick offload worked for you, I must be missing a way
to disable this timer or the kernel is thinking my CPU has unstable
TSC (which it doesn't AFAIK).

> 
> ---
> Now that scheduler_tick() has become resilient towards the absence of
> ticks, current->sched_class->task_tick() is the last piece that needs
> at least 1Hz tick to keep scheduler stats alive.
> 
> This patchset offloads this residual 1Hz tick to workqueues. This way
> the nohz full CPUs don't have anymore tick (assuming nothing else
> requires it) as their residual 1Hz tick get handled by the housekeepers.
> 
> Nothing special is required for testing, just use the usual kernel
> parameters, say on CPUs 1-7:
> 
>         "nohz_full=1-7"
> or
>         "isolcpus=nohz_offload,domain,1-7"
> 
> Thanks,
> 	Frederic
> ---
> 
> Frederic Weisbecker (6):
>       sched: Rename init_rq_hrtick to hrtick_rq_init
>       nohz: Allow to check if remote CPU tick is stopped
>       sched/isolation: Isolate workqueues when "nohz_full=" is set
>       sched/isolation: Residual 1Hz scheduler tick offload
>       sched/nohz: Remove the 1 Hz tick code
>       sched/isolation: Tick offload documentation
> 
> 
>  Documentation/admin-guide/kernel-parameters.txt |   6 +-
>  include/linux/sched/isolation.h                 |   1 +
>  include/linux/sched/nohz.h                      |   4 -
>  include/linux/tick.h                            |   2 +
>  kernel/sched/core.c                             | 104 +++++++++++++++++-------
>  kernel/sched/idle_task.c                        |   1 -
>  kernel/sched/isolation.c                        |   8 +-
>  kernel/sched/sched.h                            |  13 +--
>  kernel/time/tick-sched.c                        |  13 +--
>  kernel/workqueue.c                              |   3 +-
>  10 files changed, 103 insertions(+), 52 deletions(-)
>