Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 10 Oct 2022 09:23:28 +0800
From:   Feng Tang <feng.tang@intel.com>
To:     Peter Zijlstra <peterz@infradead.org>
CC:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@intel.com>,
        "H . Peter Anvin" <hpa@zytor.com>, <x86@kernel.org>,
        <linux-kernel@vger.kernel.org>, <rui.zhang@intel.com>,
        <tim.c.chen@intel.com>, Xiongfeng Wang <wangxiongfeng2@huawei.com>,
        Yu Liao <liaoyu15@huawei.com>
Subject: Re: [PATCH] x86/tsc: Extend the watchdog check exemption to 4S/8S
 machine
Message-ID: <Y0N0ENurfliW315D@feng-clx>
References: <20221009051209.393377-1-feng.tang@intel.com>
 <Y0LGLGW7RSlklKyl@hirez.programming.kicks-ass.net>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <Y0LGLGW7RSlklKyl@hirez.programming.kicks-ass.net>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?tYJ86Ee5xfcCJTLkFMJ6HNgjlthsDQtYlP8Vw/g50nFmHrVZoV0WC+orXcbw?=
 =?us-ascii?Q?8Wue2+/J+CphOgVIvyUgSPUpFVeTOrj61a76SWIO9rayhuCb5BcdQWTbuIua?=
 =?us-ascii?Q?QlLZlPpU7JSEpWffM2GQOdm6vYgnk/b64xjS/Nbtcuhvqtf0zFXKp2whtFG3?=
 =?us-ascii?Q?FjBZvVO02MdltR3Pv0csJK9B3fcCgj8fyUHQcN4FcGpfE0wnzR0r/GuEWeLh?=
 =?us-ascii?Q?zN54iaXuqTbUsRYXTVd0lQUqhgE2k/p3mv0qubm9pX9/ccfv3ZlWERx2thV3?=
 =?us-ascii?Q?KhjWUyzjbxhy/DlMfuX48e5tDBoXuJcoZZqHJW7fl41NAoTPHLw+wwtoyqT9?=
 =?us-ascii?Q?uKJoB0N91wo+etDJyBARtc10+zSUWLjNcngGLLGgfUND/Gal3QdsWtfoPQjn?=
 =?us-ascii?Q?qnWXg0c0dOoC/vFhD/kRhtvaHtbam7HwSC8GA+lpnviSSMTqTz56RYUfdeTt?=
 =?us-ascii?Q?bgrN+aRyMEQHEw/XFqYfZniCtDTJfSywoxnEkM7KZm04O2l9LPw8hCMOGcxM?=
 =?us-ascii?Q?nYR88hdpQpy2ayc7+Vj6MMAPk8EKTgNw4P3XMOMFis0Nd88819yHMyshkNZj?=
 =?us-ascii?Q?kYSjCYoU58/yYxJYQTtzpYaVt0Z7xkULDKxJYl1Un3ak2slyjIQL+1l0CNBD?=
 =?us-ascii?Q?JrJ2zyb89DEp72Ru++G6vX1o2yFPKXRDZm1IOR9a7RKhzThevfyHkcmc1xrs?=
 =?us-ascii?Q?0RAWZkKJteS4NUUlvjmxa/jcj6GgJtYHDFhOXwJo+85OzDzL54dXQAJMXsf0?=
 =?us-ascii?Q?Oc5oDeDHSXeIsWlhM6pgT3HrhvzzX1Rf4f/8xqnTdlzxwR/sEy039LXVZ1sj?=
 =?us-ascii?Q?6j55JaRpIIGFTfiQsDxv2gIyzEZqIHkxu1tBnEvQKpLFRcIydHdA7fHFnwe9?=
 =?us-ascii?Q?lnPKbIgPEZBdVSfT7XUtN3nL6hn1CL2BU7Cbp023HQTKIgoyHfgthCxKrkN+?=
 =?us-ascii?Q?YqxXyKljcHsCRXtGzrNFDSwMaFwxY0H4IJ9NBE+BR1XnR/Rx/1bqJldLQWQh?=
 =?us-ascii?Q?ByxobbUrjkTio8O3Qb/nkphlR8ha5YoP8SyA/etEztBW+4eLk1dqQFLlWgBm?=
 =?us-ascii?Q?08PawQVZ+50clQtZIPOjPOccNkVKgasWfDUBjApUsKL/IZyRdg5To09ZGiue?=
 =?us-ascii?Q?fxcZY/I5ec0X2k7q+KR2tdn+aH/tdzY97o3daiHkA4Q+H51J1ldv3IV1uU2y?=
 =?us-ascii?Q?+JLCpjxe1HMtB/UGtCjAkKikaLFLcBonz64+r4C8IeQe6YvBDjYoswQSD9G7?=
 =?us-ascii?Q?IgCPDLnsfXZzmS6Q5unWcSK94RJlWs6OLFCCMuAYThh8t90nipb6453AO3cO?=
 =?us-ascii?Q?drA9DxJOdD+FLRP81b4MyxUkr/bmmlZyQOV70H0ndnWzADsxQ+JzC/stmmf9?=
 =?us-ascii?Q?4utEFgDdRk3hq68ePz5ERKZJCjZ5DGGx66IiPqdLEi30Ay7kymAC8e1RscHr?=
 =?us-ascii?Q?TlcbIM5l3RUd8FTWbxwox2t2P1ggu4iq/9YA/jK5jQQet7ZOClYTlx9Qgem0?=
 =?us-ascii?Q?7UkLpgF+MXP1ZpHRIbGgOBD9f5ekRQmgGcnWl9Id6sE4+ipDjR/tHgT5Po02?=
 =?us-ascii?Q?OJvD1tD9SSUW1jKrlBSj6yzy5tsToHDCW0v/cpoQ?=
X-MS-Exchange-CrossTenant-Network-Message-Id: e99770d0-e886-4d10-2542-08daaa5e162e
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6304.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Oct 2022 01:23:50.3523
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 185rPhtDcnPyZ4nRDH3CgL3lmMqe741WKoYzBCMHOqnzZPcSsEh5lc34oh4p/GuWQMtzh/nVDA2MWp5SZCUnMA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR11MB4722
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-7.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Sun, Oct 09, 2022 at 03:01:32PM +0200, Peter Zijlstra wrote:
> On Sun, Oct 09, 2022 at 01:12:09PM +0800, Feng Tang wrote:
> > There is report again that the tsc clocksource on a 4 sockets x86
> > Skylake server was wrongly judged as 'unstable' by 'jiffies' watchdog,
> > and disabled [1]. Also we got silimar reports on 8 sockets platform
> > from internal test.
> > 
> > Commit b50db7095fe0 ("x86/tsc: Disable clocksource watchdog for TSC
> > on qualified platorms") was introduce to deal with these false
> > alarms of tsc unstable issues, covering qualified platforms for 2
> > sockets or smaller ones.
> > 
> > Extend the exemption also to 4/8 sockets to fix the issue.
> > 
> > Rui also proposed another way to disable 'jiffies' as clocksource
> > watchdog [2], which can also solve this specific problem in an
> > architecture independent way, with one limitation that some tsc false
> > alarms are reported by other watchdogs like HPET in post-boot time,
> > while 'jiffies' is mostly used in boot phase before hardware
> > clocksources are initialized.
> > 
> > [1]. https://lore.kernel.org/all/9d3bf570-3108-0336-9c52-9bee15767d29@huawei.com/
> > [2]. https://lore.kernel.org/all/bd5b97f89ab2887543fc262348d1c7cafcaae536.camel@intel.com/
> > 
> > Reported-by: Yu Liao <liaoyu15@huawei.com>
> > Tested-by: Yu Liao <liaoyu15@huawei.com>
> > Signed-off-by: Feng Tang <feng.tang@intel.com>
> > ---
> >  arch/x86/kernel/tsc.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
> > index cafacb2e58cc..b4ea79cb1d1a 100644
> > --- a/arch/x86/kernel/tsc.c
> > +++ b/arch/x86/kernel/tsc.c
> > @@ -1217,7 +1217,7 @@ static void __init check_system_tsc_reliable(void)
> >  	if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC) &&
> >  	    boot_cpu_has(X86_FEATURE_NONSTOP_TSC) &&
> >  	    boot_cpu_has(X86_FEATURE_TSC_ADJUST) &&
> > -	    nr_online_nodes <= 2)
> > +	    nr_online_nodes <= 8)
> 
> So you're saying all 8 socket systems since Broadwell (?) are TSC
> sync'ed ?

No, I didn't mean that. I haven't got chance to any 8 sockets
machine, and I got a report last month that on one 8S machine,
the TSC was judged 'unstable' by HPET as watchdog.

> AFAIK there is no architectural guarantee for >4 sockets to have a sane
> TSC. If there is one, the above should be limited to architectures that
> conform. 

Thanks for the note! Yes, we should be very cautious for 8 sockets
machine. Will limit the max sockets to 4, which was also originally
suggested by Thomas.

Thanks,
Feng