Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1;
Message-ID: <d981aa59-8aec-4929-9633-9ec7af81b099@amd.com>
Date:   Wed, 20 Sep 2023 15:11:54 -0500
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH V7 0/7] amd-pstate preferred core
Content-Language: en-US
To:     Oleksandr Natalenko <oleksandr@natalenko.name>,
        Huang Rui <ray.huang@amd.com>, Meng Li <li.meng@amd.com>
Cc:     linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org,
        x86@kernel.org, linux-acpi@vger.kernel.org,
        Shuah Khan <skhan@linuxfoundation.org>,
        linux-kselftest@vger.kernel.org,
        Nathan Fontenot <nathan.fontenot@amd.com>,
        Deepak Sharma <deepak.sharma@amd.com>,
        Alex Deucher <alexander.deucher@amd.com>,
        Shimmer Huang <shimmer.huang@amd.com>,
        Perry Yuan <Perry.Yuan@amd.com>,
        Xiaojian Du <Xiaojian.Du@amd.com>,
        Viresh Kumar <viresh.kumar@linaro.org>,
        Borislav Petkov <bp@alien8.de>,
        "Rafael J . Wysocki" <rafael.j.wysocki@intel.com>
References: <20230918081407.756858-1-li.meng@amd.com>
 <5973628.lOV4Wx5bFT@natalenko.name>
 <ce377dda-e1ce-4553-b9b8-125620b8b2d7@amd.com>
 <12290212.O9o76ZdvQC@natalenko.name>
From:   Mario Limonciello <mario.limonciello@amd.com>
In-Reply-To: <12290212.O9o76ZdvQC@natalenko.name>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?TEFXQzhwQTdRQ0VDL1QyRE5TKzBFbXdRdTJnUFQxR0F4L0s3M3JURVZwWldD?=
 =?utf-8?B?Yzh0SXRLN2VHajBKcTZ4RXFoTVE1WmllYVMxQ0dEK0JCclp1SnJzSUVIdnNj?=
 =?utf-8?B?NUpYR0dwelJLOE9OZFRyV0pWckQzL1RMZFlsRzJQZEsxMlJJbDY3dkkzK1ZU?=
 =?utf-8?B?R1BJd2NVYXlVVGZML0FSTCs5OXV5clhsU2tDR25iUUNQelZBV0NYbWJuZlVu?=
 =?utf-8?B?cXZKVnVidlR3R2x3SVVhOTAvc2xHcWs1VjZ3WGdMYVpMUUVBNFJHeWYzQUNB?=
 =?utf-8?B?d0lackF6K2lQREhkczd3ZmNmeThYYmtmTUdUb1R6aWlLdGJrRTBGZ3JZWVY1?=
 =?utf-8?B?V0tPNkk4dEVrZGZ4Z0dTaWZ1NVhaK01TZzVPYXFxRUE5VEd5bHVGOWs2bTNx?=
 =?utf-8?B?dTkwSDJsMHlKZG5RcVUvcWlERFFLTlNWSVplQWM4ZENqQUxwcmdGN3R4UjFj?=
 =?utf-8?B?RUxMb2NLQkpZT2tpNExRK29yc2ZPK3JqMzhDWHRScEF6M3NTZ3BGWFJMaVpI?=
 =?utf-8?B?TWd1LzRJRFY3bmlkM2cyWk9xTWdWOEFrSnNBYU1qeXJyQ1VwN3JvSFpSU0p5?=
 =?utf-8?B?NS82TGM1YXgybVlXcXBEaWJZSHhUelZsUU5aWFpCWE1zcTAvdG5XQ0dSOUNG?=
 =?utf-8?B?OU1aU2xoNXVWMmhwWnp4ZkdEUWhvUm0wcVBwVWpjZ0VVazhvbHp5dDFZNVZj?=
 =?utf-8?B?Q200WmFyQkkvYjNjUmNsUk02aXJRWnhJcFNIMEE2QXRuZ2dURFIzM2paY3Bi?=
 =?utf-8?B?Sno5T1BvMzBYclJKZjd2Q3I5NkFrYURsUzh3Q3k1WGpKTXhLSUNVMXFOYjBM?=
 =?utf-8?B?OVppakwwc0FBMGh5NEhyV01uak9CTSt1YWJGMjgza21TUnE2V3B5bUQ5UFky?=
 =?utf-8?B?NHY5NHlNUUhKYUcrZ1VoOHljS1IxU2NKNG8veVkwWjk4czZpQ1FKcnB2V09G?=
 =?utf-8?B?WHF4ckpZOVJ2NjhRWWNmbGdRRStnNGM5a2UxZHVzcXU4WnFISDVaeWZoWlVG?=
 =?utf-8?B?eEdTUkJlTzBraTZhVTQ0dVIzdFRyMTF2Qk9uMHJsaW1iVHFRN2VYaGFEaThX?=
 =?utf-8?B?NVY3Z3ZPQXZZbmpmck5EcW9IOFJPVXhETkZVSEJhcEttTVh5VDE1aElQMEdH?=
 =?utf-8?B?OUYyd1VxZGtqNUtwd0J5WVIzWk5lR0N3QkgxUk9zd0RUbzArc2YxZUo4MXl6?=
 =?utf-8?B?RWh5aHZQUEpuc2daaGdoZVFXSEVod1FXb1pVQ2lveWdWdGJmV3Bjd0haS2tN?=
 =?utf-8?B?cnRhSFB3ZnBqOXcxS28yWFpsRzBOT2FkQndLK21XNmJ1MkZnQ2lOSDhCZk0x?=
 =?utf-8?B?cVppeU04dEhVOTlBN0lLREY1cWZzZVBOKzBGb0tBOGt2WTBBb0ljSmpEdDlj?=
 =?utf-8?B?dmxscUNBTnNLZ2k0RWhyYTlLakU2MFhBMVVOYnMzT1lPQnBxY0VEQXFLQkg5?=
 =?utf-8?B?aitrYUlGanhNeEtpNUkrVmNQVTZ6clBML1hubm5XNVVlZ29PNXlaYmQ0MDBP?=
 =?utf-8?B?UmFkQ0NxdDRIcEQyalNGV3A5bitmemxWSTM5VDVacUphci81VjdEUnYvaENW?=
 =?utf-8?B?UUN0dlhHNUlJZG5pSnNZTUR4OFVlVGdZNk8zVHE1anlKSzEzRnQxR3dnL1Ba?=
 =?utf-8?B?R244WlVqUlU3UktZaE9MN1plaEZMUXkwaE84cG43NkV0UGdYZzlzREZWOHQ2?=
 =?utf-8?B?VHRHbjd5VXljeVVscjlVQXBKWXIxVlljQjk4UVJQZTNldWZoeGZnT05lR1E1?=
 =?utf-8?B?RTBhTUNJQ2g0Y0VZRkkzR0hHQTZZSjJBTGlLUXQ2YldVY0l4cEFiK0pHTWFk?=
 =?utf-8?B?eitXcWt2U2FuMzMzMGVycExzM2ZPb1BLeWYrb3ZmSzFrdEQvRERYc0JCKzh4?=
 =?utf-8?B?NFYyTTNHNXFOc0FEejBGMVVsQUt4OEwvZWFPUHZ4bk42Wi9QUnluVzJad1Rm?=
 =?utf-8?B?SlhROGphSDlsM1ZVbmtnVE9wWlg4bHN3TThPa01JNTVTaHhEY0l0R0dVczFM?=
 =?utf-8?B?TWpmNE9MSzBNKzB2Q2thTDh6c25xWVNMenNlSVBNY1MrRHMvckZXZk1FMmo1?=
 =?utf-8?B?UTRYbUVJUTZndDVlQzFwbC9acTFoMXpsMEFmRy9rU01zenpvNnE5bHphdEFv?=
 =?utf-8?Q?gTktFdvw8EJVxGJcvieh6weeR?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: c7cd9f14-0499-4393-47e8-08dbba15d763
X-MS-Exchange-CrossTenant-AuthSource: MN0PR12MB6101.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Sep 2023 20:11:57.5903
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 4wcys+QOxMFRgXQFxLCj/jw+Oh3cXaBxr2SF7lehEWxeLQYOi0qFyBd7Bf2VA1UZyzGrORah7aGTK4fM0V311Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB4353
X-Spam-Status: No, score=-0.9 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED,
	DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,SPF_HELO_NONE,SPF_PASS autolearn=unavailable
	autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on morse.vger.email
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org
X-Greylist: Sender passed SPF test, not delayed by milter-greylist-4.6.4 (morse.vger.email [0.0.0.0]); Wed, 20 Sep 2023 13:12:10 -0700 (PDT)

On 9/20/2023 14:34, Oleksandr Natalenko wrote:
> Hello.
> 
> On středa 20. září 2023 18:56:09 CEST Mario Limonciello wrote:
>>> When applied on top of v6.5.3 this breaks turbo on my 5950X after suspend/resume cycle. Please see the scenario description below.
>>>
>>> If I boot v6.5.3 + this patchset, then `turbostat` reports ~4.9 GHz on core 0 where `taskset -c 0 dd if=/dev/zero of=/dev/null` is being run.
>>>
>>> After I suspend the machine and then resume it, and run `dd` again, `turbostat` reports the core to be capped to a stock frequency of ~3.4 GHz. Rebooting the machine fixes this, and the CPU can boost again.
>>>
>>> If this patchset is reverted, then the CPU can turbo after suspend/resume cycle just fine.
>>>
>>> I'm using `amd_pstate=guided`.
>>>
>>> Is this behaviour expected?
>>
>> To help confirm where the issue is, can I ask you to do three
>> experiments with the patch series applied:
>>
>> 1) 'amd_pstate=active' on your kernel command line.
> 
> The issue is reproducible. If I toggle the governor in cpupower to `powersave` and back to `performance`, boost is restored.
> 
>> 2) 'amd_pstate=active amd_prefcore=disable' on your kernel command line.
> 
> The issue is not reproducible.
> 
>> 3) 'amd_pstate=guided amd_prefcore=disable' on your kernel command line.
> 
> The issue is not reproducible.
> 
> I should also mention that in my initial configuration I use `amd_pstate=guided` and `schedutil`. If I switch to `performance` after suspend-resume cycle, the boost is restored. However, if I switch back to `schedutil`, the freq is capped.
> 
> Does this info help?
> 

Yeah, it matches my expectations for this issue you reported.
Thanks!

Jassmine can dig into a fix for another spin of this series.

>> Looking through the code, I anticipate from your report that it
>> reproduces on "1" but not "2" and "3".
>>
>> Meng,
>>
>> Can you try to repro?
>>
>> I think that it's probably a call to amd_pstate_init_prefcore() missing
>> from amd_pstate_cpu_resume() and also amd_pstate_epp_resume().
>