Received-SPF: pass (google.com: domain of linux-kernel+bounces-208468-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <cfd0a622-89bc-4303-a972-4b5c8380eb76@amd.com>
Date: Mon, 10 Jun 2024 20:47:42 +0530
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 0/6] Add per-core RAPL energy counter support for AMD CPUs
To: Oleksandr Natalenko <oleksandr@natalenko.name>, peterz@infradead.org,
 mingo@redhat.com, acme@kernel.org, namhyung@kernel.org,
 mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org,
 irogers@google.com, adrian.hunter@intel.com, kan.liang@linux.intel.com,
 tglx@linutronix.de, bp@alien8.de, dave.hansen@linux.intel.com,
 x86@kernel.org, kees@kernel.org, gustavoars@kernel.org
Cc: linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
 linux-hardening@vger.kernel.org, ananth.narayan@amd.com,
 gautham.shenoy@amd.com, kprateek.nayak@amd.com, ravi.bangoria@amd.com,
 sandipan.das@amd.com, linux-pm@vger.kernel.org
References: <20240610100751.4855-1-Dhananjay.Ugwekar@amd.com>
 <2733323.mvXUDI8C0e@natalenko.name>
Content-Language: en-US
From: Dhananjay Ugwekar <Dhananjay.Ugwekar@amd.com>
In-Reply-To: <2733323.mvXUDI8C0e@natalenko.name>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?emErU2hsRnBpd0RUSkRzMElHVWp5czV6T2pxMVQ5N1VSblIveWt6bGxrRGl5?=
 =?utf-8?B?R0Q3aGp6elF0cm5JMnliNVlDbmRlK0xGUUk4WXVxcmdoTXhOekQycWt0L3ZP?=
 =?utf-8?B?OEVaY3VMNTluM0c2L1B0b3ZyVTB6aU04aDY3ZDlRRklmUlg1MS9CaWtMc0xW?=
 =?utf-8?B?L1lMTlNmNHArTUQ5WlY2RC9XeWZ3M3BCVGpOejMrcy9uUFJNam9kTTlsU29t?=
 =?utf-8?B?RWxuS24rM1NXc3dtdWYzTVkvZk45UjlZZVVIRWRtbzlxNlRXWmdGSVgwS1Rt?=
 =?utf-8?B?OEc3YzI0b3g1a3paeUVheS9hOW1mMi9jMHdBMkdiUFREVW9KRjBpQjRUWGQy?=
 =?utf-8?B?eEZSV2xOUkZnVm9XUWZHSDR3V0pmYllHWkYzemFtdXc2TSs0ZmVEQW5NTWx1?=
 =?utf-8?B?emh0bjZ0YkFCZmtCRTNFOFJBWkpjN2N3aFdnNUk4V0I3RUZZRGpKNUQ2QnM4?=
 =?utf-8?B?eXFPcVVQL0xyTXJjY2N2TjhnU28vaTNVL1JLdERxSUEwcVd1VldJdENMR3Ew?=
 =?utf-8?B?NmVvSDBBZVFZS2F4TWlQZXo1UVdOcTZ6UGF2SlczTE5nM3BlcHBma1Blc1hR?=
 =?utf-8?B?aG9VOFUweXgrWHRrWVFzOTFGOWp1QytiY0VDQzFFUzM3b2p3eG9hL1BFV2pQ?=
 =?utf-8?B?WkJCSHF2VmZYZG5XMzBUbnlzMU8xVU9EQjdnTGM5bWQ2OHRza21LQW14TU9F?=
 =?utf-8?B?a2NhMjk4a0k5Zlp5NFJuOUw3MUsvTnEvTmlEZlRzVzd2cXlOc2FtSU5UZ0xS?=
 =?utf-8?B?bUs3UFhiVDQ4SnZDTjNrRmtBM2g1eWpDREN2MTUwaGtqWGU1cFF4NFlKeG12?=
 =?utf-8?B?bWx3V1JIZ0t2KzNHZmgwN2V4VWhRL0RmRUtHTk1KWjAySXlUMVVuK0dZWHhQ?=
 =?utf-8?B?YkNEZzF6SDVwYjNrYzhESldRZ0hLd0FETjlaRzRiYUFQKzVQc2kwTlBRZXR5?=
 =?utf-8?B?NmdNVnpoOWFiTUJ5Qk53Y0ZNTi8rWlNXcGI4TE5iSWdRRThDMFZKS0xFWnhT?=
 =?utf-8?B?NFQxTEowOUVNeVB5blhtYUlRangrbmFDdTd6Wk5sc2NZQm14dSt5SXlHZXph?=
 =?utf-8?B?c2IyVlRSNG9EN3laa3B1RGwvdFhRWXhGWWdFdytaS3RJRWhGZWZacENIMXhz?=
 =?utf-8?B?YzZLaHdldEVBaHNqODExd2NHQ28zcCtaQTIxYmV5QnhnY1RDSm5NN1NLbTZS?=
 =?utf-8?B?Z1YzT3UxZTk4eS9kNnIwL0N3SGl2aVBZcDNIWE54dUdSdllJdVdpVmtUK25s?=
 =?utf-8?B?QXRnVWtQUWMvdFFlRWo0aWFJU0llWHVISnZqbUl3eUdmMk5qNmZ3US9YTnZy?=
 =?utf-8?B?TU0yYklBTHNnVXBBL0ZGU2VSVGdjSXByWEd1c2liREhxS0pqaVhoVHNCR3Rj?=
 =?utf-8?B?TzR3TDFjSzNsNlo2S0tjWndIbXNMTkxkVXVLS1BQWVhwZVZISUZtOHMvRzVG?=
 =?utf-8?B?WE9CV2ZGQmxrT2JTelFmQ2RzUTZOSTk2SW5TeS9odmsrckQ2QjN5WnRDRWZr?=
 =?utf-8?B?bEdINHhwN2dYSG9vbm5vaVd3MFh6cm54dXh1TFZUTDcvUnpnOTZVQlZyZmp6?=
 =?utf-8?B?QmE3VHpWUTNHQnhwcVd5cmRRTThuNzVnMFpxMmx3aFBEeXJSUDB3SW5sa0FE?=
 =?utf-8?B?K1JCNEx3WUNhUENsUHRGaWM4a2ZZTDlwR1Z2ZzhGR0hpQ2xUSHdSVm5aRGhz?=
 =?utf-8?B?cjhhTjZWbFVMb1V5OWlhS0hiYmVCSXpNZy8vQ1p3NytDSGhqc295UTd4MHhk?=
 =?utf-8?B?dE1SV3o1UkdraHhwSmlWY2pleWVHMjlqSUhWQmMrMVRWdVdiU05kbVlKb0hU?=
 =?utf-8?B?ZGp2bzhrVHFFM3NzRXhUU0pSZDNnSTV3WmsxZ3FSVjBQdUtvVmpUbVdFV2c2?=
 =?utf-8?B?UE15SjAxV0FmVXBjUFB2WDhLT3pZUm1WWnlqUGlmMlN2WTc2UE0zLzB3b0pm?=
 =?utf-8?B?WWNpYzB1azdlbFVMci84bVNHanlKMUE1a05BQzRiQW1kL0tPd2pQNkpNcTNO?=
 =?utf-8?B?MlN4TnoyRlRpSlRZVGRIUFZ0emp3QjdVTStWcVVuVGMyMW9yM3hBMWROVGZE?=
 =?utf-8?B?cUZ2eXB6NmoxYXVXN0VMM3dhSWFIZ0NqVW45TzFsWk1yei90bCtlVmNJSTZX?=
 =?utf-8?Q?OEOEs0yvyHSTICvWjm9VQ+rbz?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 46497c75-56ae-4485-4790-08dc8960812b
X-MS-Exchange-CrossTenant-AuthSource: LV8PR12MB9207.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Jun 2024 15:17:56.0282
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: y+licGruIJjNcYR+3Qc5ETDO/Sn+jxy72X9WCeYZHq2iDiI/mwr4Pnxo7Bx8eXinhdbCz0OFZrH8SqnfVvGFsA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SA0PR12MB4400

Hello Oleksandr,

On 6/10/2024 7:58 PM, Oleksandr Natalenko wrote:
> Hello.
> 
> On pondělí 10. června 2024 12:07:45, SELČ Dhananjay Ugwekar wrote:
>> Currently the energy-cores event in the power PMU aggregates energy
>> consumption data at a package level. On the other hand the core energy
>> RAPL counter in AMD CPUs has a core scope (which means the energy 
>> consumption is recorded separately for each core). Earlier efforts to add
>> the core event in the power PMU had failed [1], due to the difference in 
>> the scope of these two events. Hence, there is a need for a new core scope
>> PMU.
>>
>> This patchset adds a new "power_per_core" PMU alongside the existing
>> "power" PMU, which will be responsible for collecting the new
>> "energy-per-core" event.
>>
>> Tested the package level and core level PMU counters with workloads
>> pinned to different CPUs.
>>
>> Results with workload pinned to CPU 1 in Core 1 on an AMD Zen4 Genoa 
>> machine:
>>
>> $ perf stat -a --per-core -e power_per_core/energy-per-core/ sleep 1
>>
>>  Performance counter stats for 'system wide':
>>
>> S0-D0-C0         1          0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C1         1          5.72 Joules power_per_core/energy-per-core/
>> S0-D0-C2         1          0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C3         1          0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C4         1          0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C5         1          0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C6         1          0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C7         1          0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C8         1          0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C9         1          0.02 Joules power_per_core/energy-per-core/
>> S0-D0-C10        1          0.02 Joules power_per_core/energy-per-core/
>>
>> [1]: https://lore.kernel.org/lkml/3e766f0e-37d4-0f82-3868-31b14228868d@linux.intel.com/
>>
>> This patchset applies cleanly on top of v6.10-rc3 as well as latest 
>> tip/master.
>>
>> Dhananjay Ugwekar (6):
>>   perf/x86/rapl: Fix the energy-pkg event for AMD CPUs
>>   perf/x86/rapl: Rename rapl_pmu variables
>>   perf/x86/rapl: Make rapl_model struct global
>>   perf/x86/rapl: Move cpumask variable to rapl_pmus struct
>>   perf/x86/rapl: Add wrapper for online/offline functions
>>   perf/x86/rapl: Add per-core energy counter support for AMD CPUs
>>
>>  arch/x86/events/rapl.c | 311 ++++++++++++++++++++++++++++++-----------
>>  1 file changed, 233 insertions(+), 78 deletions(-)
>>
>>
> 
> With my CPU:
> 
>   Model name:             AMD Ryzen 9 5950X 16-Core Processor
> 
> and this workload:
> 
> $ taskset -c 1 dd if=/dev/zero of=/dev/null
> 
> the following result is got:
> 
> $ sudo perf stat -a --per-core -e power_per_core/energy-per-core/ sleep 1
> 
>  Performance counter stats for 'system wide':
> 
> S0-D0-C0              1               1,70 Joules power_per_core/energy-per-core/
> S0-D0-C1              1               8,83 Joules power_per_core/energy-per-core/
> S0-D0-C2              1               0,17 Joules power_per_core/energy-per-core/
> S0-D0-C3              1               0,33 Joules power_per_core/energy-per-core/
> S0-D0-C4              1               0,14 Joules power_per_core/energy-per-core/
> S0-D0-C5              1               0,33 Joules power_per_core/energy-per-core/
> S0-D0-C6              1               0,25 Joules power_per_core/energy-per-core/
> S0-D0-C7              1               0,19 Joules power_per_core/energy-per-core/
> S0-D0-C8              1               0,66 Joules power_per_core/energy-per-core/
> S0-D0-C9              1               1,71 Joules power_per_core/energy-per-core/
> S0-D0-C10             1               0,38 Joules power_per_core/energy-per-core/
> S0-D0-C11             1               1,69 Joules power_per_core/energy-per-core/
> S0-D0-C12             1               0,22 Joules power_per_core/energy-per-core/
> S0-D0-C13             1               0,11 Joules power_per_core/energy-per-core/
> S0-D0-C14             1               0,49 Joules power_per_core/energy-per-core/
> S0-D0-C15             1               0,37 Joules power_per_core/energy-per-core/
> 
>        1,002409590 seconds time elapsed
> 
> If it is as expected, please add my:
> 
> Tested-by: Oleksandr Natalenko <oleksandr@natalenko.name>

We can see that after you affined the workload to cpu 1, energy 
consumption of core 1 is considerably higher than the other cores, 
which is as expected, will add your tested-by in next version.

P.S: I'm assuming here that cpu 1 is part of core 1 in your system, 
please let me know if that assumption is wrong.

Thanks for testing the patch!

Regards,
Dhananjay

> 
> Thank you.
>