Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 5 Jun 2023 14:28:35 +0800
From:   Feng Tang <feng.tang@intel.com>
To:     "Paul E. McKenney" <paulmck@kernel.org>
CC:     Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@intel.com>,
        "H . Peter Anvin" <hpa@zytor.com>,
        Peter Zijlstra <peterz@infradead.org>, <x86@kernel.org>,
        <linux-kernel@vger.kernel.org>, <rui.zhang@intel.com>,
        <tim.c.chen@intel.com>, Xiongfeng Wang <wangxiongfeng2@huawei.com>,
        <liaoyu15@huawei.com>
Subject: Re: [PATCH v1 2/2] x86/tsc: Extend watchdog check exemption to
 4-Sockets platform
Message-ID: <ZH2Ak28tT3tVlWOD@feng-clx>
References: <20221021062131.1826810-1-feng.tang@intel.com>
 <20221021062131.1826810-2-feng.tang@intel.com>
 <06df410c-2177-4671-832f-339cff05b1d9@paulmck-laptop>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <06df410c-2177-4671-832f-339cff05b1d9@paulmck-laptop>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?lRbhxGQTOHnTRQJ7y6GtUDzpozzFWRrsasUCPNFsndMeMIGgssfUJYDjPKPB?=
 =?us-ascii?Q?tVxNr1/b3TahXwsyD2Q3LJdaZ8I8pUqxX41ueGeGjvDxuMisP7eHaG/ro9a4?=
 =?us-ascii?Q?cIbO7ezF7WGof9g6t490ddnV3qWBudZ6UgNdjoWXO9cvdj9sKSG6LlrytCQU?=
 =?us-ascii?Q?GNgqjzP5iO2vyXiyWEXyhH3y5/dTxMMhUo/qgrHZvBeNquxQOwUFGDgveEgX?=
 =?us-ascii?Q?07dOLdkODu6MfihmkRXJvtwRiMbhSKQn6LGIvxyifNtfeCwS82z8tmm0J+2T?=
 =?us-ascii?Q?bXHB/yQng/JldRTrVKy0s/KNFtuos7ylFc0nZL5lq8HMrqzyBjdvl4qLljhZ?=
 =?us-ascii?Q?dB2KHjxHxeXdoN29C8xWY9I4tvjkA3kJ80QKruHOCbejMgQ2zPxi2DsofIjn?=
 =?us-ascii?Q?fhBfOJ7yGINMlUwzmC/yPfumtU++nmdZQkx+nuQiXThvDvUMdZaR1wJZF9bh?=
 =?us-ascii?Q?wgJXvUi1I20sCkR6Th/kgQv+QAV0ia6hIwUdAQJLdpPsOOSKMvT4x8mogYzt?=
 =?us-ascii?Q?1JpspGCLshfUpHv8seUv40SPi3Jp/GTqrkUTusdRfgjbyASBrhWY9zolMs40?=
 =?us-ascii?Q?9h2OjSwhkqNCaAGtrksHYN2PnhH5slYWXXtp0hs42kU/Stji2UFOx6r47CA0?=
 =?us-ascii?Q?EEA9Tp+XiiK7ouLB/Drk5V6AkAegGxJC2u2AtI/y4A25w1s54K4/PQIZFhQP?=
 =?us-ascii?Q?y3IFfj4D3tNnL71Le0VSOuOYGSpyj73hJlOevmlsjdp2nY8rhjMVmcugf76H?=
 =?us-ascii?Q?SGF8LJdqvftK+1UVbsAdbZo/NTkRsm5fcipVsdJZd0raKQQ2U2XJzsU/m6pS?=
 =?us-ascii?Q?A2PSLnxPAqn09Ozq2ZUUWUYmZvGJHHaawUZ+e/niZXWKZJgFf7eeGc7HR6Db?=
 =?us-ascii?Q?OO6i6ZKP8bJcuMr4Cb3FhMWa2Rh0EejpITKuDK/6QiJfu9Fa+peqSKt3+I7b?=
 =?us-ascii?Q?IOSilKF5JDiWrrq7V+UEJXwC5cNaUs4ZK2S9YrcIJHhlV05mykO8UZBRiBiB?=
 =?us-ascii?Q?SBkk8iqYLz/lEZfgq9hVSuM9Hb8HC5HONjH3yx4gxaqzJHaaMJKsgoI/6Jy7?=
 =?us-ascii?Q?LWVAIRgwaJr74XdwB0VMNiOMJ640VYpw+xGQOeEADRFyPApFIODoeXS7Q+hj?=
 =?us-ascii?Q?DKHskKXG7O0RJgXv4FWUcGEhpeRzXtrsUhxrZjKnJL/Vm0lQ1s5+txgs7Mzd?=
 =?us-ascii?Q?eGudwc5WcHPv1ugkAdNNoTIYLEfPPXOEqQCxRexY+IR9rP4qcITJwuS4hh8+?=
 =?us-ascii?Q?T/X8jVapnH8VReIUdgUjpyK7zA81zQnuE9BFFj2yU5eCMNewuVwcuornNgw2?=
 =?us-ascii?Q?/iRqE4oUFHqU8NpoNu9HRLdbAK/lVyyl0wLWVXHHTd2ZpcrMP1lj884+5dZo?=
 =?us-ascii?Q?3Is0DyFYjpm3qDJu5b5RhTZKtbiTpVqcv/MuslLIQY5bnnTV1/+cOWoLHSW6?=
 =?us-ascii?Q?i0/HOgrOvpfo35gJ/8JJjvQFrO83YzfDNXNfjLPOm78BBaJA/GupTyh2VJBs?=
 =?us-ascii?Q?0+WkrqdG/473A79byFODxY6hm7Y8fkR+zYJzEFxzton+JwgIpJnHZzxCWfaZ?=
 =?us-ascii?Q?6zZ9vqSzRGssjjQEGwZ3lIA7pYO2yorvZ/QQdAeK?=
X-MS-Exchange-CrossTenant-Network-Message-Id: ff347f64-a4c7-4f8f-b419-08db658f1a95
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6304.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jun 2023 06:35:50.8499
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: NxYzD5alZA/J83eMHgLbjbrxcGTj40fMaWNdHR6SPTxYHhAFcPtqyJBvtPw7pBSOIUJzp5XKhnNu+ZnQvgTYhw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR11MB4818
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Jun 02, 2023 at 11:00:55AM -0700, Paul E. McKenney wrote:
> On Fri, Oct 21, 2022 at 02:21:31PM +0800, Feng Tang wrote:
> > There is report again that the tsc clocksource on a 4 sockets x86
> > Skylake server was wrongly judged as 'unstable' by 'jiffies' watchdog,
> > and disabled [1].
> > 
> > Commit b50db7095fe0 ("x86/tsc: Disable clocksource watchdog for TSC
> > on qualified platorms") was introduce to deal with these false
> > alarms of tsc unstable issues, covering qualified platforms for 2
> > sockets or smaller ones.
> > 
> > Extend the exemption to 4 sockets to fix the issue.
> > 
> > We also got similar reports on 8 sockets platform from internal test,
> > but as Peter pointed out, there was tsc sync issues for 8-sockets
> > platform, and it'd better be handled architecture by architecture,
> > instead of directly changing the threshold to 8 here.
> > 
> > Rui also proposed another way to disable 'jiffies' as clocksource
> > watchdog [2], which can also solve this specific problem in an
> > architecture independent way, with one limitation that there are
> > also some tsc false alarms which were reported by other hardware
> > watchdogs like HPET/PMTIMER, while 'jiffies' watchdog is mostly
> > used in kernel boot phase.
> > 
> > [1]. https://lore.kernel.org/all/9d3bf570-3108-0336-9c52-9bee15767d29@huawei.com/
> > [2]. https://lore.kernel.org/all/bd5b97f89ab2887543fc262348d1c7cafcaae536.camel@intel.com/
> > 
> > Reported-by: Yu Liao <liaoyu15@huawei.com>
> > Signed-off-by: Feng Tang <feng.tang@intel.com>
> 
> We have a number of four-socket systems whose TSCs seem to be reliable.
> We do see issues where high memory load forces the TSC to be marked
> unstable, but that is because those systems are using an older kernel.

Thanks for sharing the info.

> 
> If the TSCs do start to misbehave, I will of course let you all know.

That will be very helpful! I don't have much access to 4 socket machines.

> But in the meantime:
> 
> Reviewed-by: Paul E. McKenney <paulmck@kernel.org>
> 
> The previous patch that changes the definition of "socket" I have no
> opinion on.  I must let you guys work that out.  However, I do note that
> this patch can be rebased so as to no longer depend on that patch.

During previous discussion, Thomas and Peter mentioned they only
saw real TSC synchronization issue on some old generation 8
socket/package machine.

I can separate this and send out for review. Meanwhile I'll rework
on the 1/2 patch and test more.

Thanks,
Feng

> 						Thanx, Paul
> 
> > ---
> >  arch/x86/kernel/tsc.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/arch/x86/kernel/tsc.c b/arch/x86/kernel/tsc.c
> > index 178448ef00c7..356f06287034 100644
> > --- a/arch/x86/kernel/tsc.c
> > +++ b/arch/x86/kernel/tsc.c
> > @@ -1400,7 +1400,7 @@ static int __init init_tsc_clocksource(void)
> >  	if (boot_cpu_has(X86_FEATURE_CONSTANT_TSC) &&
> >  	    boot_cpu_has(X86_FEATURE_NONSTOP_TSC) &&
> >  	    boot_cpu_has(X86_FEATURE_TSC_ADJUST) &&
> > -	    logical_packages <= 2)
> > +	    logical_packages <= 4)
> >  		clocksource_tsc.flags &= ~CLOCK_SOURCE_MUST_VERIFY;
> >  
> >  	/*
> > -- 
> > 2.34.1
> >