Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Fri, 24 Mar 2023 17:47:33 -0700
From:   "Paul E. McKenney" <paulmck@kernel.org>
To:     Feng Tang <feng.tang@intel.com>
Cc:     Thomas Gleixner <tglx@linutronix.de>,
        Waiman Long <longman@redhat.com>, linux-kernel@vger.kernel.org
Subject: Re: A couple of TSC questions
Message-ID: <acda3fb5-373a-48b1-b78e-eea18da4ccb0@paulmck-laptop>
Reply-To: paulmck@kernel.org
References: <b2f07f18-b60b-403b-90ff-937ea32963ee@paulmck-laptop>
 <ZBqOyLByOgvdAve3@feng-clx>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <ZBqOyLByOgvdAve3@feng-clx>
Precedence: bulk

On Wed, Mar 22, 2023 at 01:14:48PM +0800, Feng Tang wrote:
> Hi, Paul
> 
> On Tue, Mar 21, 2023 at 04:23:28PM -0700, Paul E. McKenney wrote:
> > Hello, Feng!
> > 
> > I hope that things are going well for you and yours!
> 
> Thanks!
> 
> > First, given that the kernel can now kick out HPET instea of TSC in
> > response to clock skew, does it make sense to permit recalibration of
> > the still used TSC against the marked-unstable HPET?
> 
> Yes, it makes sense to me. I don't know the detail of the case, if
> the TSC frequency comes from CPUID info, a recalibration against a
> third party HW timer like ACPI_PM should help here. 
> 
> A further thought is if there are really quite some case that the
> CPUID-provided TSC frequency info is not accurate, then we may need
> to enable the recalibration by default, and give a warning message
> when detecting any mismatch. 

Now that you mention it, it is quite hard to choose correctly within
the kernel.  To do it right seems to require that NTP information be
pushed into the kernel.

> > Second, we are very occasionally running into console messages like this:
> > 
> > Measured 2 cycles TSC warp between CPUs, turning off TSC clock.
> > 
> > This comes from check_tsc_sync_source() and indicates that one CPU's
> > TSC read produced a later time than a later read from some other CPU.
> > I am beginning to suspect that these can be caused by unscheduled delays
> > in the TSC synchronization code, but figured I should ask you if you have
> > ever seen these.  And of course, if so, what the usual causes might be.
> 
> I haven't seen this error myself or got similar reports. Usually it
> should be easy to detect once happened, as falling back to HPET
> will trigger obvious performance degradation.

And that is exactly what happened.  ;-)

> Could you give more detail about when and how it happens, and the
> HW info like how many sockets the platform has. 

We are in early days, so I am checking for other experiences.

> CC Thomas, Waiman, as they discussed simliar case here:
> https://lore.kernel.org/lkml/87h76ew3sb.ffs@tglx/T/#md4d0a88fb708391654e78312ffa75b481690699f

Fun!  ;-)

							Thanx, Paul