Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <69a3f699-257a-1579-3ae6-236dd19b5381@amd.com>
Date:   Thu, 22 Sep 2022 11:14:45 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.13.0
Subject: Re: [PATCH] ACPI: processor_idle: Skip dummy wait for processors
 based on the Zen microarchitecture
Content-Language: en-US
To:     Dave Hansen <dave.hansen@intel.com>, linux-kernel@vger.kernel.org
Cc:     rafael@kernel.org, lenb@kernel.org, linux-acpi@vger.kernel.org,
        linux-pm@vger.kernel.org, dave.hansen@linux.intel.com,
        bp@alien8.de, tglx@linutronix.de, andi@lisas.de, puwen@hygon.cn,
        mario.limonciello@amd.com, peterz@infradead.org,
        rui.zhang@intel.com, gpiccoli@igalia.com,
        daniel.lezcano@linaro.org, ananth.narayan@amd.com,
        gautham.shenoy@amd.com, Calvin Ong <calvin.ong@amd.com>,
        stable@vger.kernel.org, regressions@lists.linux.dev
References: <20220921063638.2489-1-kprateek.nayak@amd.com>
 <2b6143b6-9db4-05bc-1e8d-c5d129126f99@intel.com>
From:   K Prateek Nayak <kprateek.nayak@amd.com>
In-Reply-To: <2b6143b6-9db4-05bc-1e8d-c5d129126f99@intel.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?OHN1QzZUZHRvYVE1ZnRmWUlBdDhsaytKUmhOdC8rRU8wcllSZ29TeXNBSThn?=
 =?utf-8?B?a1djS0lzYi9jT1hINC9qaGQyUFdFVFdRbmdUZFgzQWUwYnFrVWlTaXNuNlJ6?=
 =?utf-8?B?c0J2NXFaMzhpbHI0b1VxQmFNcUIxYXE2aThXZ0pocWtaWkJCSDJzaXJ5SzRR?=
 =?utf-8?B?SlpGM2E2dkVPZmJ0dVVyS2JTM2F2MFY2Zkx6dVNxWWMyR2tkSkJzdFVNZUxE?=
 =?utf-8?B?Vnc5VElLdlJTVi9oWEZ4ajlUaG14eVRGYlRhbUFFN01DWm4zMHZQaDJ5RDRk?=
 =?utf-8?B?eTM2Zy9KVC8yWGZSNjlXSHdSTnpCTGZmRG9DY094aXhkNDV4UVQyVFF3OFV1?=
 =?utf-8?B?UkxDK29tcmorUDd6YzROVUR6TWk5c3diekZ5aTVHQ0JSOXp6dkhNSTlBL0JO?=
 =?utf-8?B?V1ZsU0dHTHMxUzFDUTdONVBhMWwvaVVwL0FGd2xsdHVuUmdmUHZlWmF4UGhj?=
 =?utf-8?B?YitsNVlnWG1JeFloUWgzdGRMbzFQWThQVGZCaWNrVzByd1BZNTdPUFY5Mml5?=
 =?utf-8?B?MHhLOEE1TEFKdUx1UzFmbUxqM0RzMlZkZHhheFJPNlo0WFBJZnpmejd6RUZk?=
 =?utf-8?B?SHVOR2xlWmFuODdBSE1tUFZJQ1J4d2pqVHova3NuekVVdnV2UEJueU5pWTNZ?=
 =?utf-8?B?cExlSE9KYkF4SDhtdU9mSFlYS3hNTjE1cWJMSi81ZzYxRTNwVloxY1pwbTJh?=
 =?utf-8?B?WmhJcVlabW1qOWZlLzZMeExHK2xsRFlpNXhYWjUyZmRjZHFZMWRCSC9qaE1L?=
 =?utf-8?B?NmlkVm5YdFc0QzBURmN2N2F3YUNwWW8wOHM4OE5sT29jL3ZJOVkrekdWRURB?=
 =?utf-8?B?U2tSbStpRVpqK0NydmJ0VWdXMW9sVE44YzVQSWQ2VHZxaXdocEYvZk9RU1Fr?=
 =?utf-8?B?K3QrTjZHNmd4NlFybklXbmpKbkgwT1l2MWRYeFJRaEZhVGtiZVRjZTEvSXp6?=
 =?utf-8?B?d1lydmgxVlYvVWlZNEZJcjN5RzZ3YW1mOVhySElYaHFucDBIamxuZXFraE13?=
 =?utf-8?B?NWh5LzZwVFkzUkVnWGxVNE9qK3JyQ3M1QjNubURWenI5NXFCUkhDd0s0VmpY?=
 =?utf-8?B?dk5XMnVOS0gwOC9mVk1XMzFydFgyQnpzUzRCUjBWWndqVVVXVTV3akZaQ3M1?=
 =?utf-8?B?dDVWejhPNHQzb2FBeS84d3BvWnNtTUNlYWdVdzNJWk1IN3F0bzJSR0FmSmNI?=
 =?utf-8?B?ZnNtN1NCSkpSWitOREdzbUdlWUp1VnlueUlzc1hmU3M1R0QvckNSendNOWpv?=
 =?utf-8?B?TGE1WldMSUpSLzlzaG5FQmM4SFdsWlZHVmJIb0JqbkZWamRLbG54NWUyU2JU?=
 =?utf-8?B?My91cWVjeG1laVlQa0hkdFFKY21Md2Q5WG9NclRzTUtZSm1GNG1rTVg4ZCsx?=
 =?utf-8?B?Vkc4aDY5OE53MXpvbzYwbHB0TWIzS0xsRjBZb3hGNlBEYUFQUU0xNmNTdmtO?=
 =?utf-8?B?NTE0bHZHQlN1amlQQkdyaWRvZG1qQndoUzdKM1BEL0pwWHZVRVg2YnJlMjFF?=
 =?utf-8?B?YW1aYmJsZ2U3Q0xnQkZpUitoREtycXVoQ0ZpSnhnNnlQQWNzeW15YzJUei9x?=
 =?utf-8?B?V0R1YUJpdlRWanhOaEhhNng5S3RMcmdBVGt6WUtHUWJVLzJ5QkRma3VkeUlG?=
 =?utf-8?B?MVh5cnN1TzBsMHYxQnA0TnJWQ1U4dlg2UjFUbVo0azhXS0o4OThtekNVcjVm?=
 =?utf-8?B?R05oN3d3ZzdZdDEyRW4xSERsK1Rwaldwa0ZjZC9UMTlIMVg2R3Q1TmdRalUw?=
 =?utf-8?B?MkQxLzZHSDVCelFIQ1VUYzI2SHlyMkNNcElCeVF0bnRmR3Z6N1hxU28rd3Rn?=
 =?utf-8?B?aitUOUpUU1p0U09GdzZtcWdzT0U5cFdiU3pheExpU3dVT1J5WHhReU1nVnpr?=
 =?utf-8?B?MGJHMG5Tc084MzZxa1NuTmhKMEJueFJjK3ZEZnNjT3owYTZJOW1tcm9JWk1x?=
 =?utf-8?B?OEo0cmhJMElwa3NJVURlcUxoeUZidnVDbjFrTFpkeTdNMlMwbXdwQW1yaktI?=
 =?utf-8?B?SEp5VEVjMUw5NzBKa1lQYUpXdWp5MWJVY09zci9Vd1l2UDNXL0Q5RXpvWmQx?=
 =?utf-8?B?YlZQZkF1NE40KzBHNWo2SWIyWC95UzYxYVM2UUxNM2QwcEtjU3JBWk9WUTkz?=
 =?utf-8?Q?0ale68ecDzeYkhOdhAZDuds1S?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 14fb7e2a-e373-4fa3-e315-08da9c5d9a05
X-MS-Exchange-CrossTenant-AuthSource: SN1PR12MB2381.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Sep 2022 05:45:06.1002
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: MS71s26Lq5RuAtQ1IxnK/Nb+/zk7edwJYQjLqr8B6tPwIVs1ZiaRDJoEcyUiO0mRnUlVWVIonVwzL2YJ4fahcA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MN0PR12MB5811
X-Spam-Status: No, score=-5.8 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hello Dave,

On 9/21/2022 7:45 PM, Dave Hansen wrote:
> On 9/20/22 23:36, K Prateek Nayak wrote:
>> +	/*
>> +	 * No delay is needed if we are in guest or on a processor
>> +	 * based on the Zen microarchitecture.
>> +	 */
>> +	if (boot_cpu_has(X86_FEATURE_HYPERVISOR) || boot_cpu_has(X86_FEATURE_ZEN))
>>  		return;
> 
> In the end, the delay is because of buggy, circa 2006 chipsets?  So, we
> use a CPU vendor specific check to approximate that the chipset is
> recent and not affected by the bug?  If so, is there no better way to
> check for a newer chipset than this?

Elsewhere in the thread, people have noted that the faulty chipsets seem to
go all the way back to pre-2002. Andreas's comment was added in 2006 but we
have no way of knowing if it is limited only to chipsets prior to 2006. If
anyone can confirm a clean cut-off point when this was no longer required,
perhaps we can limit this dummy wait to the older chipsets by annotating
them with a X86_BUG_STPCLK quirk.

> 
> Do X86_FEATURE_ZEN CPUs just have unusually painful
> inl(acpi_fadt.xpm_tmr_blk.address) implementations?

Yes. The issue becomes more pronounced with increased core counts when many
cores exit from C2 simultaneously. The core density is especially high on
X86_FEATURE_ZEN chipsets, none of which require a dummy wait op to ensure
correct behavior. Hence, we used the feature check to skip it.

> Is that why we
> noticed all of a sudden?
> 

We saw run-to-run variance in tbench with 128 clients as a part of our
scheduler regression runs. Originally we attributed it to the tbench
threads not being spread around uniformly, but the problem persisted even
when we ensured that the initial task placement spreads them out. Further
analysis showed that significant time was spent in exit from C2 in the
bad runs.

--
Thanks and Regards,
Prateek