Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 22 Mar 2023 13:14:48 +0800
From:   Feng Tang <feng.tang@intel.com>
To:     "Paul E. McKenney" <paulmck@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Waiman Long <longman@redhat.com>
CC:     <linux-kernel@vger.kernel.org>
Subject: Re: A couple of TSC questions
Message-ID: <ZBqOyLByOgvdAve3@feng-clx>
References: <b2f07f18-b60b-403b-90ff-937ea32963ee@paulmck-laptop>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <b2f07f18-b60b-403b-90ff-937ea32963ee@paulmck-laptop>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?P6urGkkNna19KAj0DkUNBwWXTcrstCEUtmHkNJslnsqhmQqMy9ftSjgKrehG?=
 =?us-ascii?Q?SpktpkzxR9dB9OMQUledRml//XvcpiJP5PLEOgP2kpyQFoQ+liPd+dW/96lX?=
 =?us-ascii?Q?JTnymI57AlK1TRZq1w38+1uIuaaeX0TaDMl+W/xereJkeF10jaQqYp/Tc6tA?=
 =?us-ascii?Q?JTM3HWQu9RiYJyD4WftUDbhEL43h1fgCX6f/P5+0qw7kBi5cVHNcn+fjwdSF?=
 =?us-ascii?Q?FKHrfMtyWRFn7328yKKTwWn/AVgFjAqUG8+uvgFXG1/T4JS0LnOlqKwNkJmg?=
 =?us-ascii?Q?X6BRjUYlohI+vE3K9cUeEuPSHzZ492i/r3JYPieO+N/CWUFdljli/EkyeGyw?=
 =?us-ascii?Q?N44EMMT90tUoY8v8+lZ4uscUQhumtAJdH6Fh24tuPk/FuYy667xViFlmHj1V?=
 =?us-ascii?Q?dfL7v1x6bHbexsyIhncAKilrL3YvgKDYUlD9Vp+1JWZBzHIUQrpk0KbXKvvD?=
 =?us-ascii?Q?qV1rRfe0+ErhGfAOZeHJkovnPjoqepx7n7nvoWsmpArLeEz2YPwDZ6weE0N8?=
 =?us-ascii?Q?BiDz2ejpxideUBcZusgiogNSpsaP6lkMi3yKDuNICRVBsooBR/8IC0dNrWVg?=
 =?us-ascii?Q?IEBweuyatPwh5QIMKVozSsSuprWAOpieV4CKvncK9RlSsaHhueMLaSuPj8p7?=
 =?us-ascii?Q?gaJ71ZXGzDLNM/kOFd3oZ9K0a4vSDxeywVt53x5EM/1eohRqBivbbx4BG/BB?=
 =?us-ascii?Q?nbDw/XTIK/EUZGTObvUQrM5rv/hiWUwox/KP/I54KUTjHlsMWErIn0k4bzqr?=
 =?us-ascii?Q?WXPZAjp0D6/Q2x1whMIc764aNOErf9j6UUHgk09Exd5sQvu64mO3CxeMgIwy?=
 =?us-ascii?Q?QpVq6sZoN8U9GgNrZcHnyf2c5jZON0KPVxZnd69yDVZS1j+gFLhMpTEffMtH?=
 =?us-ascii?Q?8Uz5+fJE7Lw16rzFFTXH1eHE3jOFyfJB2MKWXEuPy1OyUnixebXFFUxzeVtC?=
 =?us-ascii?Q?RbPDLMyMnnKw7hIAg9ka/GutYvMQZrsQAON9ZFmCPLSjSMx+3CUT708R/gb1?=
 =?us-ascii?Q?C9OIEOUX0ItsjjN1dQdRPgbxIxYDnIxxaJ2pa5lbMXgSiRt8oANiMgJNlmzC?=
 =?us-ascii?Q?vVgKiMmwY9JG2GG8H1pkYVWJZ7FN+u31TrBjWVoMtt0MCy0wt+et3+rOxCQL?=
 =?us-ascii?Q?udvDQ0S8VQj+ic0m3+sWRax5N6qXOMFPjGiIAfwqzJU92LhEqioT/QZe2PUy?=
 =?us-ascii?Q?gQYfGljHI4kEsujyaPsV1w+oxAiTQib4TtCmueeiyL+YrCuaWkFOfhKZgqMa?=
 =?us-ascii?Q?IwXtU9rkDwRNGOO8Y0dr4NvZ92+2v8kv4s46J28PL0ae3ZJlUB01kCyiv4/k?=
 =?us-ascii?Q?foVG+iBAj1hH45PiVXYMBLM224HYnCYMbxNAs49BoOdzlHCZsGXIICFiT804?=
 =?us-ascii?Q?Xn1U3enZASz3+LJBa0wOBuyY1xh6ne1rBJWq/ztKnfyLdV6rq/6I9IteTXTE?=
 =?us-ascii?Q?umQ52yQdmTNk+gY2dfBG7Tp1RFOZCKUjjEBeXLKRxePlEw7z9PcqOo6kkSuQ?=
 =?us-ascii?Q?V5oGmvWNLqWOjj/C8zyFvKdQOhjfJE2lf27+vsTQ4nYTBh6aIDIzaFSAxOwf?=
 =?us-ascii?Q?gC3jvIvvHubohu884lFWtcJlTvIGRJUWxlWv6ieu?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 5657aaa6-9ecc-49f4-de3f-08db2a952224
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6304.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Mar 2023 05:20:21.6423
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: FedxohbaYUHwXfZBSFvYjlpO1Hu+ryHBjAE/rm+ApMVXFi/K46OZUY8K2kC5icfb7Vll7Gp8ATLKJ5YjBStmxw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH7PR11MB6698
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.5 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE,
        SPF_NONE,URIBL_BLOCKED autolearn=unavailable autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi, Paul

On Tue, Mar 21, 2023 at 04:23:28PM -0700, Paul E. McKenney wrote:
> Hello, Feng!
> 
> I hope that things are going well for you and yours!

Thanks!

> First, given that the kernel can now kick out HPET instea of TSC in
> response to clock skew, does it make sense to permit recalibration of
> the still used TSC against the marked-unstable HPET?

Yes, it makes sense to me. I don't know the detail of the case, if
the TSC frequency comes from CPUID info, a recalibration against a
third party HW timer like ACPI_PM should help here. 

A further thought is if there are really quite some case that the
CPUID-provided TSC frequency info is not accurate, then we may need
to enable the recalibration by default, and give a warning message
when detecting any mismatch. 

> Second, we are very occasionally running into console messages like this:
> 
> Measured 2 cycles TSC warp between CPUs, turning off TSC clock.
> 
> This comes from check_tsc_sync_source() and indicates that one CPU's
> TSC read produced a later time than a later read from some other CPU.
> I am beginning to suspect that these can be caused by unscheduled delays
> in the TSC synchronization code, but figured I should ask you if you have
> ever seen these.  And of course, if so, what the usual causes might be.

I haven't seen this error myself or got similar reports. Usually it
should be easy to detect once happened, as falling back to HPET
will trigger obvious performance degradation.

Could you give more detail about when and how it happens, and the
HW info like how many sockets the platform has. 

CC Thomas, Waiman, as they discussed simliar case here:
https://lore.kernel.org/lkml/87h76ew3sb.ffs@tglx/T/#md4d0a88fb708391654e78312ffa75b481690699f

Thanks,
Feng

> Thoughts?
> 
> 							Thanx, Paul