Message-ID: <7d31ec11-cc4d-b6ed-8656-d23257e5af95@amd.com>
Date:   Thu, 2 Mar 2023 00:02:53 +0530
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Subject: Re: [PATCH] perf/x86/rapl: Enable Core RAPL for AMD
Content-Language: en-US
To:     Oleksandr Natalenko <oleksandr@natalenko.name>,
        Peter Zijlstra <peterz@infradead.org>,
        Ingo Molnar <mingo@redhat.com>,
        Arnaldo Carvalho de Melo <acme@kernel.org>,
        Mark Rutland <mark.rutland@arm.com>,
        Alexander Shishkin <alexander.shishkin@linux.intel.com>,
        Jiri Olsa <jolsa@kernel.org>,
        Namhyung Kim <namhyung@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        Borislav Petkov <bp@alien8.de>,
        Dave Hansen <dave.hansen@linux.intel.com>, hpa@zytor.com
Cc:     x86@kernel.org, linux-perf-users@vger.kernel.org,
        linux-kernel@vger.kernel.org, gautham.shenoy@amd.com,
        ananth.narayan@amd.com
References: <20230217161354.129442-1-wyes.karny@amd.com>
 <12153455.O9o76ZdvQC@natalenko.name>
From:   Wyes Karny <wyes.karny@amd.com>
In-Reply-To: <12153455.O9o76ZdvQC@natalenko.name>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?OUd1YnNic0JldWhKWDc5TDdtQWwwMGZaYmhLRS9ZMWY0bk8xYlErZk5PT1Iz?=
 =?utf-8?B?NGFGem5RWHdwYVlhMmhGakhCTW02M0N5UXRrS0xOMkxQSlI3SSt1WThGY0tI?=
 =?utf-8?B?SENwY0lqRWtKMDhkWFg3VlM0MGJRQ3NOWkZScWE2WjRKdWJXQmhXVXpjSUJs?=
 =?utf-8?B?V1NWNUNHbDZ5cXFEL2FOdWQzTk9sTUNKcmhpYlo5a2tuOFJIT25QL1ZFeWxv?=
 =?utf-8?B?ektYWFBlazNFWVZFbk5GTVZON2lNKzVIZFF1YlR1NFFtdlJpbTFPVms5Vkxy?=
 =?utf-8?B?MTBHUjRGcDZPLzdUMTQ4NDRJd3ZBZHY5M2VEUVFaVVpXSFBGWFp2aWVOUzE5?=
 =?utf-8?B?QlJwREozTXZvSVY3Qmd4aDVwbUdmY1pnNXJyUFgxZEZqcE84OFo5Qlo2U3o4?=
 =?utf-8?B?YzIzWGcyM2ZycTlSYnF1ZE1DWXZVSDdsbVBjVDFCZ3RjbVplN0JxN0RBSzhy?=
 =?utf-8?B?ZFpyR1NyNFF0TXR5RUg1dUhUWGxINVZMMjJwUFhUUzdVTzdCSTZDZGtvamg1?=
 =?utf-8?B?NDBNUGFJTjJycVArN0UzZzkzVVpLNk1XdFA1ZmtLZmxUUmRqMHZ3R2s2MVdM?=
 =?utf-8?B?cHlvcm1HSTdpSzZML1VVelRXeUIycys5VEVGWTBDOEp3amxNMmJMWVI5ZDdz?=
 =?utf-8?B?bHFrOEZGWHZJTDNUdWVyQzl2U0x5dEFiWmp2QjZJR3JMSFUrdy95N1hPRFIr?=
 =?utf-8?B?S1hicUZiT0V6UzVLQjhFUWlYNWE2NFNvWnhBUE9ybC9wMnp5cGJvS3F2R1Ez?=
 =?utf-8?B?cXZDYmJsRG9qZVMxQ3FxMnVTb0JFMVhSOXJjUGQyQ1E4cHRkVTZtdEwvWHRx?=
 =?utf-8?B?NDVCQmhKd2xvdjhpQkhWYmtTMCtQTHUzaVNhd2xCTTFpZm1PdTlkdWFzanhC?=
 =?utf-8?B?RHBjdDRaV2xKWG9IMUNCNWRLZUJscHdwV043Zm5PSWljSHdDVWhscURiTzF2?=
 =?utf-8?B?MW8ydlp3eDR5ZC9uY3BoeDFndVk3Ynluak5OUG1nbFAxSFVlTWxubkVjQmE2?=
 =?utf-8?B?bXJNc2ZRSXhka0xVMnVFOVRYeUpyR1NKVnE0MHlQYUxWRzdCbWMwa1VCY1NN?=
 =?utf-8?B?OERkRnI2MlVIaVRhSnVueENJWFI1bjQ5QTRWMEVIczR5VXY1RHVSOXowK2JM?=
 =?utf-8?B?eE9KeEdqbHFQUmNQcWtWeUl0a3NjUDgwUWJ4cnlNYUt1dTVVc1Q1cnZxb1pr?=
 =?utf-8?B?VVBPMmpzL1JHckpuempuYXYwVFpiQWg1ZTNvSEtraDg3NmNqWEpMazN5citn?=
 =?utf-8?B?SUZOZXZsb1g2azE0b1NTWkpBY1hmQXNvVUNSMDNkaVBzaW05aml3VG8xOXl0?=
 =?utf-8?B?TDMxSFZtWTNQdEJ5YlNHa3BoZytISDQrSnVPbmsyNE5qd3pUZm9Wc1RsbEp2?=
 =?utf-8?B?bEpzSHMraGV3Y1NrTnNmME5tWVRqY2VXdVFrY01kV25JbFZyaTIvNzVIS0l6?=
 =?utf-8?B?TWsvMFZ6TldpR2NISjRnSFF2cTlEOVFCVGs0bUFQNnFzZE1YWkc5emp0Njls?=
 =?utf-8?B?ZktXTVZ6eGM5ZSszK09oUk1yL2I3dm02ZTJkeFF2anIvZm40VjMvNnZFOHVG?=
 =?utf-8?B?K2NNNnNOdm15d3R1bUcrRFJUL1JmU09pRzhxaThVb3hlYU9PMnFEL1hTSGZv?=
 =?utf-8?B?ZXN6MUpiNi9rdVVJNXh2RHVQbU9mR2FaV1dkdUdQM1dtRE1la0l2dUVPNTcy?=
 =?utf-8?B?VzBsL1RWaUJBTHpXQ2llZGZFcDJ6Q2YwVmE2em5zRDRESGN1UXlza0pIZnov?=
 =?utf-8?B?MHlSeFdSeis5cFlnNFIwc3UwN0RrdEw2cUFwcnZsU0VZc2lGMEtZZkFGWVhi?=
 =?utf-8?B?aE1YSVUvS3JBSkVFZnpmTzZpWXVOTVBZUzBOVEtEUStUTFZBazRDRUcvbDdv?=
 =?utf-8?B?b0dEaTd0cDFIZnRVMExYYTdNWEhIK2krQ3YrRndWNEN1Y1JYc3hUemI0Zzlt?=
 =?utf-8?B?bFpIUlpwVTdwaDJVT1dWWHdSelBQVzRjWjFOdzV5dEJiTEplMy9HeGRKNk05?=
 =?utf-8?B?dnlOcU9sQ2dmWFEyQWt5VGlkZFJiUDlEY3BsNTdjR2FTMForUWVtekRKNUR3?=
 =?utf-8?B?S056TlJHMUNNR25VZWZwRVVYb25VVC9xelhXbVBZeURGRmp3dGJnWlA5R2My?=
 =?utf-8?Q?wxYrPLKt+D4oLWAPn420uicXx?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 9a17a85c-9afc-43a9-1bf8-08db1a8367e2
X-MS-Exchange-CrossTenant-AuthSource: BY5PR12MB3876.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 01 Mar 2023 18:33:09.2403
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 5ZPG4ZPKU4LSRMsGKhPpzdsfKWcM5DkQ9UW6ZTqB3yzPA6W4pgI1o2SGJ/SldKjTwnk1rEGXhuf1NZg9AvBKXA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: LV2PR12MB5774
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Hi Oleksandr,

On 2/19/2023 5:22 PM, Oleksandr Natalenko wrote:
> Hello.
> 
> On pátek 17. února 2023 17:13:54 CET Wyes Karny wrote:
>> AMD processors support per-package and per-core energy monitoring
>> through RAPL counters which can be accessed by users running in
>> supervisor mode.
>>
>> Core RAPL counters gives power consumption information per core.  For
>> AMD processors the package level RAPL counter are already exposed to
>> perf. Expose the core level RAPL counters also.
>>
>> sudo perf stat -a --per-core -C 0-127 -e power/energy-cores/
>>
>> Output:
>> S0-D0-C0           2               8.73 Joules power/energy-cores/
>> S0-D0-C1           2               8.73 Joules power/energy-cores/
>> S0-D0-C2           2               8.73 Joules power/energy-cores/
>> S0-D0-C3           2               8.73 Joules power/energy-cores/
>> S0-D0-C4           2               8.73 Joules power/energy-cores/
>>
>> Signed-off-by: Wyes Karny <wyes.karny@amd.com>
>> ---
>>  arch/x86/events/rapl.c | 5 +++--
>>  1 file changed, 3 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/x86/events/rapl.c b/arch/x86/events/rapl.c
>> index 52e6e7ed4f78..d301bbbc3b93 100644
>> --- a/arch/x86/events/rapl.c
>> +++ b/arch/x86/events/rapl.c
>> @@ -537,7 +537,7 @@ static struct perf_msr intel_rapl_spr_msrs[] = {
>>   * - want to use same event codes across both architectures
>>   */
>>  static struct perf_msr amd_rapl_msrs[] = {
>> -	[PERF_RAPL_PP0]  = { 0, &rapl_events_cores_group, 0, false, 0 },
>> +	[PERF_RAPL_PP0]  = { MSR_AMD_CORE_ENERGY_STATUS, &rapl_events_cores_group, test_msr, false, RAPL_MSR_MASK },
>>  	[PERF_RAPL_PKG]  = { MSR_AMD_PKG_ENERGY_STATUS,  &rapl_events_pkg_group,   test_msr, false, RAPL_MSR_MASK },
>>  	[PERF_RAPL_RAM]  = { 0, &rapl_events_ram_group,   0, false, 0 },
>>  	[PERF_RAPL_PP1]  = { 0, &rapl_events_gpu_group,   0, false, 0 },
>> @@ -764,7 +764,8 @@ static struct rapl_model model_spr = {
>>  };
>>  
>>  static struct rapl_model model_amd_hygon = {
>> -	.events		= BIT(PERF_RAPL_PKG),
>> +	.events		= BIT(PERF_RAPL_PP0) |
>> +			  BIT(PERF_RAPL_PKG),
>>  	.msr_power_unit = MSR_AMD_RAPL_POWER_UNIT,
>>  	.rapl_msrs      = amd_rapl_msrs,
>>  };
>>
> 
> With this patch:
> 
> ```
> $ lscpu | grep 'Model name'
> Model name:                      AMD Ryzen 9 5950X 16-Core Processor
> 
> $ sudo perf stat -a --per-core -C 0-15 -e power/energy-cores/ -- dd if=/dev/zero of=/dev/null bs=1M count=100000
> 100000+0 records in
> 100000+0 records out
> 104857600000 bytes (105 GB, 98 GiB) copied, 1,59252 s, 65,8 GB/s
> 
> Performance counter stats for 'system wide':
> 
> S0-D0-C0           1               1,56 Joules power/energy-cores/
> S0-D0-C1           1               1,56 Joules power/energy-cores/
> S0-D0-C2           1               1,56 Joules power/energy-cores/
> S0-D0-C3           1               1,56 Joules power/energy-cores/
> S0-D0-C4           1               1,56 Joules power/energy-cores/
> S0-D0-C5           1               1,56 Joules power/energy-cores/
> S0-D0-C6           1               1,56 Joules power/energy-cores/
> S0-D0-C7           1               1,56 Joules power/energy-cores/
> S0-D0-C8           1               1,56 Joules power/energy-cores/
> S0-D0-C9           1               1,56 Joules power/energy-cores/
> S0-D0-C10          1               1,56 Joules power/energy-cores/
> S0-D0-C11          1               1,56 Joules power/energy-cores/
> S0-D0-C12          1               1,56 Joules power/energy-cores/
> S0-D0-C13          1               1,56 Joules power/energy-cores/
> S0-D0-C14          1               1,56 Joules power/energy-cores/
> S0-D0-C15          1               1,56 Joules power/energy-cores/
> 
> 1,593982452 seconds time elapsed
> ```
> 
> Hence,
> 
> Tested-by: Oleksandr Natalenko <oleksandr@natalenko.name>

I haven't put your tested-by tag in v2 as I've added a new patch there.
Please let me know if v2 works well for you and if I can add tested-by on v2.
 
> 
> Thank you.
> 

-- 
Thanks & Regards,
Wyes