Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20180719205545.16512-24-pasha.tatashin@oracle.com>
 <tip-857baa87b6422bcfb84ed3631d6839920cb5b09d@git.kernel.org>
 <20180724195215.GA22496@roeck-us.net> <CAGM2reayvBs8rs6yHZYV9SvPz-tsNG_Ux8ypHguLLagiS2SkuA@mail.gmail.com>
 <CAGM2rebX7dq_ejNuSotV_0gJZY6pqro-KteXz9zzhDkjtSxaOw@mail.gmail.com>
 <ddb6bc3d-b764-3115-ebc7-54c70c967f24@roeck-us.net> <CAGM2reaUEOMbbtfbLY2n66jJV=ZH0Wy7GtX-tfCG62U7Uhn14g@mail.gmail.com>
In-Reply-To: <CAGM2reaUEOMbbtfbLY2n66jJV=ZH0Wy7GtX-tfCG62U7Uhn14g@mail.gmail.com>
From:   Pavel Tatashin <pasha.tatashin@oracle.com>
Date:   Tue, 24 Jul 2018 22:41:19 -0400
Message-ID: <CAGM2reYevKGh7pqvK5O7oLB5FYkuU31v7t_AMDYt0=Q+jsVEqw@mail.gmail.com>
Subject: Re: [tip:x86/timers] sched/clock: Enable sched clock early
To:     Guenter Roeck <linux@roeck-us.net>, peterz@infradead.org
Cc:     LKML <linux-kernel@vger.kernel.org>, mingo@kernel.org,
        tglx@linutronix.de, hpa@zytor.com,
        linux-tip-commits@vger.kernel.org
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

Peter,

The problem is in this stack

start_kernel
  local_irq_enable
  late_time_init
  sched_clock_init
    generic_sched_clock_init
      sched_clock_register
        WARN_ON(!irqs_disabled());

Before this work, sched_clock_init() was called prior to enabling
interrupts, but now after. So, we hit this WARN_ON() in
sched_clock_register().

The question is why do we need this warning in sched_clock_register? I
guess because we want to make this section of the code atomic:

195         new_epoch = read();  <- from here
196         cyc = cd.actual_read_sched_clock();
197         ns = rd.epoch_ns + cyc_to_ns((cyc - rd.epoch_cyc) &
rd.sched_clock_mask, rd.mult, rd.shift);
198         cd.actual_read_sched_clock = read;
199
200         rd.read_sched_clock     = read;
201         rd.sched_clock_mask     = new_mask;
202         rd.mult                 = new_mult;
203         rd.shift                = new_shift;
204         rd.epoch_cyc            = new_epoch;
205         rd.epoch_ns             = ns;
206
207         update_clock_read_data(&rd); <- to here

If we need it, we can surround the sched_clock_register() with
local_irq_disable/local_irq_enable:

diff --git a/kernel/time/sched_clock.c b/kernel/time/sched_clock.c
index cbc72c2c1fca..5015b165b55b 100644
--- a/kernel/time/sched_clock.c
+++ b/kernel/time/sched_clock.c
@@ -243,8 +243,11 @@ void __init generic_sched_clock_init(void)
         * If no sched_clock() function has been provided at that point,
         * make it the final one one.
         */
-       if (cd.actual_read_sched_clock == jiffy_sched_clock_read)
+       if (cd.actual_read_sched_clock == jiffy_sched_clock_read) {
+               local_irq_disable();
                sched_clock_register(jiffy_sched_clock_read, BITS_PER_LONG, HZ);
+               local_irq_enable();
+       }

        update_sched_clock();

Thank you,
Pavel