Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Mon, 24 Oct 2022 15:37:18 +0800
From:   Feng Tang <feng.tang@intel.com>
To:     Dave Hansen <dave.hansen@intel.com>,
        Zhang Rui <rui.zhang@intel.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, Borislav Petkov <bp@alien8.de>,
        "H . Peter Anvin" <hpa@zytor.com>,
        "Peter Zijlstra" <peterz@infradead.org>, <x86@kernel.org>
CC:     <linux-kernel@vger.kernel.org>, <tim.c.chen@intel.com>,
        Xiongfeng Wang <wangxiongfeng2@huawei.com>,
        <liaoyu15@huawei.com>
Subject: Re: [PATCH v1 1/2] x86/tsc: use logical_package as a better
 estimation of socket numbers
Message-ID: <Y1ZArguS/rVEjXOb@feng-clx>
References: <20221021062131.1826810-1-feng.tang@intel.com>
 <f27e4b3f858890c657df9a7d6f34dc2d60b89757.camel@intel.com>
 <63dca468-c94d-844a-5b19-09c03cf84911@intel.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <63dca468-c94d-844a-5b19-09c03cf84911@intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?K3J7TMkm4QElXwxo/la3yvXAFINkJZbR1eeFrI1qByWmUzmHnwaEkq8y3tLP?=
 =?us-ascii?Q?lSq3OCVw8fe0Fq5CQ+iNWLJA09J8qDQDzERvhXNDhFnfFL6ju01QtBFgLJ2m?=
 =?us-ascii?Q?eVjbjM/fA7ONhK3rV+GKlCakYy5mvyJsJK6G/A16IBK6PeBWi1bJHMbjA0kw?=
 =?us-ascii?Q?jY55LVDpohegrlSd6HPkejBXpgkf1C29PaehkLhn3IbPj098A5qbxnNGi4zc?=
 =?us-ascii?Q?BbOUu/uN2KEwj1kK2FrJZ003rT/XJ8XX+op/iff5nbgzEnj4n5YKWPG1ceCd?=
 =?us-ascii?Q?Q/0055LU1Ib8MjaKzaIgS6t1+OjcsOiks1eABrRdTHojs0tDxnDpYLEl4wec?=
 =?us-ascii?Q?Xh6X16sguQfnWZVvM+OZIs+uIWyNQbBFit4Ov39aIIov0/s2ou5yxUTMyZRH?=
 =?us-ascii?Q?jVfz/qJcsYcBSsA3c2+Ia/Sh9TJHn1FxtA6cBFJz7ZaaZrOzuvsHDh41I/IN?=
 =?us-ascii?Q?0P/Off3g9TDYS0WIF08LD92TgsJeUs9dNRCfaePHQNIS8UrKZHIzTjfZziWY?=
 =?us-ascii?Q?xfr27d8XHLIZQ9DOsEHJHcgYkAb7g/J9KVPghuGtNd9DpqIGcJczMVswEzVc?=
 =?us-ascii?Q?hrFETV1HHDSx91dY96NIji0DscnLFvU26PsTmX5QGuqzjTZzzgGMcabx6OnR?=
 =?us-ascii?Q?erpCb8gzNGOZ3n5GtDO+DVxXSQbzDnZq2MnVLgx6itl7Z9hnLP2f/AJw9XVS?=
 =?us-ascii?Q?WdpksKP+5OKAtRaXRcpAOieuoyjVUQ95ZfD8ILq17X/xeBR8OQtkuliQuyxh?=
 =?us-ascii?Q?aullwDAedEeqePLVh0OAoUidI632zMIBw9Yh1Nj8E+1AoBFuydIGKanFuz7Q?=
 =?us-ascii?Q?bv9vReU8R2gIC0Sdn1lhpICHmST5bGKJ8m28yc+lsIziMs99TzHuwGlL4D5k?=
 =?us-ascii?Q?rbWmIzlhRobO8d1KPfgOyfLF7CVNkSCuPWSz9AS7i8GSVVewS6MFfaPv5LnP?=
 =?us-ascii?Q?IBqLWXhD6f5RUGbjM5Q48BnSFcUj5TfUA7YR1meBPXc8Ryt0Ba7GY2SuRnOP?=
 =?us-ascii?Q?ucZz+DRupC9dV8/Y7yLt1Sxkk/NsStXg+yM42GbyDHhtV8ZMWei6nVgGfDab?=
 =?us-ascii?Q?3IIvEht7wDF0FtHT05KDTIzLaJium9PutE4NNdnYrLlFkKIOx+W1AOihmTEK?=
 =?us-ascii?Q?OJ+qi7aGe/sMXxq0tr5+FmBJ1581Ie95w+EngJZl6z8/0aYyVzOl1184EbN9?=
 =?us-ascii?Q?NlL6K6XH4PZBMODCdyzuPkW6fv6OvqUVCVt5NF2yUYBOqKUQdTLDzhplyYZd?=
 =?us-ascii?Q?s7rchEJponP7hiM1e+ZXFqk1rDVJhu3Y3/c+idig9uy6+AZ6y/b2cmvqPdWb?=
 =?us-ascii?Q?8KNQupK/nTgYBzKFHDjjjM3m2swQF2IHDMuYt6lQlz/m7e1Mq8M0qVtRbfH+?=
 =?us-ascii?Q?AYQfMdEk1REGFDT7lm/T7XbfNVfBdy9jpX+E58GHY3Pdd4yDhld7POBrUG2q?=
 =?us-ascii?Q?pPuG3/7LPQvXqZxw7+4mqN1JkeL5bkK4ea2x4leNazyvEmwHRMEIhYityFVH?=
 =?us-ascii?Q?Jri5LorgsRmjbwMLFEeip4IIRNubF3k81oXY7Fzaum/fefopZMOsiNqxnWb7?=
 =?us-ascii?Q?RAr5KHoV10XIGD8QerPK2sS6XR/XH5Z/JBUhbybe?=
X-MS-Exchange-CrossTenant-Network-Message-Id: e8607278-719c-4308-bf16-08dab5929af1
X-MS-Exchange-CrossTenant-AuthSource: MN0PR11MB6304.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 24 Oct 2022 07:37:29.7309
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: co4tZHahL7bbYA4O9U7DBlxQQ/Z1h/DiBxRiUsPgvZMjVnKuxW9563cm0zqWNqAXOzyqCuLWgV9wfQV6tsttKg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR11MB5204
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.9 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Fri, Oct 21, 2022 at 09:21:02AM -0700, Dave Hansen wrote:
> On 10/21/22 08:00, Zhang Rui wrote:
> > I checked the history of '__max_logical_packages', and realized that
> > 
> > 1. for topology_max_packages()/'__max_logical_packages', the divisor
> >    'ncpus' uses cpu_data(0).booted_cores, which is based on the
> >    *online* CPUs. So when using kernel cmdlines like maxcpus=/nr_cpus=,
> >    '__max_logical_packages' can get over-estimated.
> > 
> > 2. for 'logical_packages', it equals the number of different physical
> >    Package IDs for all *online* CPUs. So with kernel cmdlines like
> >    nr_cpus=/maxcpus=, it can gets under-estimated.

Thanks for raising this new case! For 'maxcpus=' cmdline parameter, I
run this on 4-Sockets and 2-Sockets platform, and found the
'logical_packages' is still correct if the 'maxcpus >= (total_cpus/2 + 1)',
but gets wrong for smaller numbers.

SRAT parsing solution [1]. can solve 'maxcpus' case, but it will fail
for other user cases like sub-numa cluster or 'numa=off' case

IIUC, 'maxcpus' are likely for debug purpose, I think 'logical_packages'
is the better option for socket number estimation in the several
existing solutions.

> For instance, I can live with the implementation being a bit goofy when
> kernel commandlines are in play.  We can pr_info() about those cases.

Something like adding

pr_info("Watchdog for TSC is disabled for this platform while estimating
	the socket number is %d, if the real socket number is bigger than
	4 (may due to some tricks like 'maxcpus=' cmdline parameter, please
	add 'tsc=watchdog' to cmdline as well\n", logical_packages);

and adding a new 'tsc=watchdog' option to force watchdog on (might be
over-complexed?)

[1]. https://lore.kernel.org/lkml/Y0UgeUIJSFNR4mQB@feng-clx/

Thanks,
Feng