Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 19 Jun 2023 18:42:06 +0800
From:   Feng Tang <feng.tang@intel.com>
To:     Peter Zijlstra <peterz@infradead.org>
CC:     "Zhang, Rui" <rui.zhang@intel.com>,
        "Chen, Tim C" <tim.c.chen@intel.com>,
        "bp@alien8.de" <bp@alien8.de>,
        "dave.hansen@linux.intel.com" <dave.hansen@linux.intel.com>,
        "hpa@zytor.com" <hpa@zytor.com>,
        "mingo@redhat.com" <mingo@redhat.com>,
        "tglx@linutronix.de" <tglx@linutronix.de>,
        "x86@kernel.org" <x86@kernel.org>,
        "paulmck@kernel.org" <paulmck@kernel.org>,
        "Woodhouse, David" <dwmw@amazon.co.uk>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [Patch v2 2/2] x86/tsc: use logical_packages as a better
 estimation of socket numbers
Message-ID: <ZJAw/ipOybjHNfeh@feng-clx>
References: <20230613052523.1106821-1-feng.tang@intel.com>
 <20230613052523.1106821-2-feng.tang@intel.com>
 <20230615092021.GE1683497@hirez.programming.kicks-ass.net>
 <d97fe59e47de77a36746107e4070e44ed46bf4d1.camel@intel.com>
 <20230616080231.GZ4253@hirez.programming.kicks-ass.net>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20230616080231.GZ4253@hirez.programming.kicks-ass.net>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?8SrfVljtmNi5xw1UKXn03wfGi/YP932sCFBJf9Ba3Ue0mblhRX2HeWTnZheq?=
 =?us-ascii?Q?aXQ2dxc9otf2xouPIqsT/PgiufHqk9cieI0KR8xvD3OoETqu6Zb2ubpQBX74?=
 =?us-ascii?Q?sKadD927zx106qIyu3IzqfXAs5Y78JSFv1mjgBeBEDXB5EiVv+KhEhyHLxJC?=
 =?us-ascii?Q?PgaD4PmaCdwGlFJ2b08Krz8ABNrChqb0FEPixXS8eRkPoGGGSjgthWH3d/bt?=
 =?us-ascii?Q?lfgcU+O6DqfIUZ60bM3yVymoNhcRKhE4FhHX3J9cFgQsfMESZTmV7Ob3S++b?=
 =?us-ascii?Q?AGfmfMIr9E6t3jYVZ0tNTYuE+QeLBQmu8lImsRae1v6ZEow3Gu6AIISlG/OZ?=
 =?us-ascii?Q?Dudz2ngj7MUcMIu2ZwDLWcFQqViPnWfKTjadrtqwZ+rFwsxqfGlSUXtdweOl?=
 =?us-ascii?Q?sbGoWEe6TN5SmZ2V3goXxG9ChVFgXryLBpEoag2lyirH1E0fC+R54YMBWKrl?=
 =?us-ascii?Q?lQDZZxMQJVLyTMqBlzmeLqXfkQceOniIIgx+v2K0Gnx7BJ6WT8QsGjeIHquX?=
 =?us-ascii?Q?KryFoDfFBQuwXoO8Wby0TL9M4yufQBJMPonaKHVZQ5asEpD1eYnAe3A1IwgE?=
 =?us-ascii?Q?Mtvprc0WOXK6jysyx+Kb03h/uvNIzeGOlnobvlSUgWx2AM75OI20YvZCu9Qx?=
 =?us-ascii?Q?aITQrKUP6VGZN0esqPY1yJcjHl5JWLIPxWaydRIMUISH5wOzJGNIcrBeKScl?=
 =?us-ascii?Q?H9pmoCXQe1ohmJRUmhHE8Iej38kbn8lD8kwSzYf0UmWSP3qKSTpPzfIFSeZ9?=
 =?us-ascii?Q?H6jTTpEclZi0VtwR5KCXZUoCdnQ37N/KOFtDOL44iRIY/AUKtoeDNtcfvGJm?=
 =?us-ascii?Q?VXGTu39PGsvFgvrBj4CRUg+TpHNjJeqtShHETIzupZBwBzNVPeWa8FDxdoCD?=
 =?us-ascii?Q?0ho1GkFU7x0Dk606Egtd/rw90Y0YJT49kpKBmYWoCHc/dgsKBj8j2bEHvnx9?=
 =?us-ascii?Q?byI+H2Zk4mtslBuy78F2WpbJMkorjMg8mPlwfhLiUWjILzegCbcAJDIxEAGr?=
 =?us-ascii?Q?9nIGDIgUiGTvPNU9xyhYzNURUSDf+doCtZj1ZICbjXImTg+w/LFiREnUBcGK?=
 =?us-ascii?Q?ihKoqMiOoz6aWyTB5huKZWwKC324xEygqE6H+5eRxRFl1K+Y6e8wCqNA3cL2?=
 =?us-ascii?Q?7VMsBTE8W/Qz6OnFG9pnwmqgx5GqIfPWWdDroO0fcGd+2oSDL1SICnoqc7/T?=
 =?us-ascii?Q?/2NnzMok3rC1UIYU91n33J8b/yCUjCSv90+kXXiaSH4V3TIg5O89Yzc0Yljw?=
 =?us-ascii?Q?7J8HgFSkMD4jq6FPij3n8a5JgQhZ7qgh/RvkyATOUqnQIbHOpp77z70W7cIa?=
 =?us-ascii?Q?bTZfZyJA0fE861X5j5aClQTbQ60lE7yB99jjvuanl67JnFjjmECrAWBANIE4?=
 =?us-ascii?Q?A5n2GaU2t3dOP1h7MDSzeA353ETzi16rodz2wvdxOLzzRjRVtnmETuX7vkJD?=
 =?us-ascii?Q?8ZBpkxNrNWT1Y9qjo78rDBGaueAoRCh5frzIO3HfLElgRkhqfOubs1o+OE7T?=
 =?us-ascii?Q?3GWEcPD7EfMAnbOiy8qX03jQBAq+mKu4ZUGRUPt/vhf1YFnDwj+qexGsVrPl?=
 =?us-ascii?Q?k9UnYUa2rV7SFnRAeZzYLlXXhmpBYWOXRH7RKwTi?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 162e1b80-ea1b-4332-e5c0-08db70b2db88
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6304.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Jun 2023 10:49:29.4252
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 9k3qVTRH9ycwS1LokgaMvHuVjhTawFGrqFP3FwEEd9NbvRhaaz9jvDQDBWyFpKygs5LSC73ekBYs8+mOV4NQJA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR11MB6890
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Jun 16, 2023 at 10:02:31AM +0200, Peter Zijlstra wrote:
> On Fri, Jun 16, 2023 at 06:53:21AM +0000, Zhang, Rui wrote:
> > On Thu, 2023-06-15 at 11:20 +0200, Peter Zijlstra wrote:
> 
> > > So I have at least two machines where I boot with 'possible_cpus=#'
> > > because the BIOS MADT is reporting a stupid number of CPUs that
> > > aren't
> > > actually there.
> > 
> > Does the MADT report those CPUs as disabled but online capable?
> > can you send me a copy of the acpidmp?
> 
> Sent privately, it's a bit big.
> 
> > I had a patch to parse MADT and count the number of physical packages
> > by decoding all the valid APICIDs in MADT.
> > I'm wondering if the patch still works on this machine.
> 
> I can certainly give it a spin; it has IPMI serial-over-ethernet that
> works. Brilliant dev machine.
> 
> > > So I think I'm lucky and side-stepped this nonsense, but if someone
> > > were
> > > to use nr_cpus= for this same purpose, they get screwed over and get
> > > the
> > > watchdog. Sad day for them I suppose.
> > 
> > what if using package_count_from_MADT?
> 
> So I'm thinking that if you cap possible_mask the actual logical
> packages is the right number.
> 
> Suppose you have a machine with 8 sockets, but limit possible_mask to
> only 1 socket. Then TSC will actually be stable, it doesn't matter you
> have 7 idle sockets that are not synchronized.
> 
> Then again, perhaps if you limit it to 2 sockets you're still in
> trouble, I'm not entirely sure how the TSC sync stuff comes apart on
> these large systems.

I had the similar thought. For this case, the defensive way is to keep
the watchdog for 'nr_cpus=' and 'possible_cpus=' setup, and if the
specific setup has no TSC sync issue, people can add one more parameter
'tsc=reliable' to skip the watchdog, while aggressive way is to ignore
the 2 cmdline parameters as the above case is really rare.

Again, as you mentioned, I can't find a perfect solution to cover all
kinds of setup and broken firmware. But at least 'logical_packages' is
much better than 'nr_online_nodes' :)

Thanks,
Feng