Received-SPF: pass (google.com: domain of linux-kernel+bounces-4862-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <05bc5720-d066-42fe-a3f9-d933a88e5935@amd.com>
Date: Tue, 19 Dec 2023 13:38:07 +0530
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/3] perf/x86/amd/lbr: Use freeze based on availability
To: Stephane Eranian <eranian@google.com>
Cc: linux-perf-users@vger.kernel.org, linux-kernel@vger.kernel.org,
 x86@kernel.org, peterz@infradead.org, mingo@redhat.com, acme@kernel.org,
 mark.rutland@arm.com, alexander.shishkin@linux.intel.com, jolsa@kernel.org,
 namhyung@kernel.org, adrian.hunter@intel.com, tglx@linutronix.de,
 bp@alien8.de, irogers@google.com, mario.limonciello@amd.com,
 ravi.bangoria@amd.com, ananth.narayan@amd.com
References: <cover.1702833179.git.sandipan.das@amd.com>
 <1d9106579c7781746ca39860bda8061c56d6dc48.1702833179.git.sandipan.das@amd.com>
 <CABPqkBQ0Zn_orR_9FnHA7Y1pNHAzG0E=86MkdMjOtGfKXDp29g@mail.gmail.com>
Content-Language: en-US
From: Sandipan Das <sandipan.das@amd.com>
In-Reply-To: <CABPqkBQ0Zn_orR_9FnHA7Y1pNHAzG0E=86MkdMjOtGfKXDp29g@mail.gmail.com>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?NkVva2QrWU8wbXFvU0dHWll4NzdqeWhLaHRDTnRpSkk1aTJoNml6cGxFMjVE?=
 =?utf-8?B?TGU3Q2FPOHJCQlBIRGZDc2s2eDJ4aGdnbW5PL3A1MEd4bGd1RUlSS2ZuSFEw?=
 =?utf-8?B?MDg4YTFZT2NYOWFYbEpxVnY2VFhMSm5VV2VxSkxhL1hGL05Rd0RuaHhLcjAw?=
 =?utf-8?B?NTRDemlQT1VhcnAyUTJHZWNiZHkyVHZVaFZwRG1Mai9Fd1NXV0xCeXB2Q1Ri?=
 =?utf-8?B?dnZmYVBYWGlHOC92a2VhdU1uNTdjWWV5ZENRK3hxOXV2RUFwLzRkV3BucEJY?=
 =?utf-8?B?QzlXNUxIcHQxUm5jZFJrN1h5RkRZZlNQODJIUmgzcHFkNm83T04rZjUzTjNK?=
 =?utf-8?B?cmpVQ2Q0V3ZlUlE5UlJHelFoSk96VGlKT3ZTU2J2Z01Ka09hZkdUaTR3ZFZ4?=
 =?utf-8?B?dVpHSzNrNlV3WWNOWHdDeTFHWmphdkZFRFNJL2Fjcmw3NHRnVkFPQWxCNlh0?=
 =?utf-8?B?MWNZSEl2aDZFTFdNZzJjaFhzSU5iRXFrc0hONGkwTnRDQ2NPWWJUQ2tSMTN1?=
 =?utf-8?B?bkxMOGZoVWwxNUplS0ZVUDAybmtHWjNlZlpNMzI5cmxCVG8wcng2RXJGUnpM?=
 =?utf-8?B?RGNUYTdRRnhvLytyY2Eyb2g3aThBUXFxZk1lbnl6dWJ1R3QwYmE2TWdXbHZj?=
 =?utf-8?B?aUtPMnN2bzM1NitKSWk5YktndVBSVklCK1F3ais3VldURTI0KzF5cTdXZTlF?=
 =?utf-8?B?U2NQNTdSMmdMVTlqc2FYVzlxbXpEZHpJM1c4dmxvZGRmcHk4SDUwcjU1YjJZ?=
 =?utf-8?B?K08rTHc5eG8rVldXUG1uaVZNNkJxZjFVZlhvekxzRThGUW1vc1lUSytjeVBI?=
 =?utf-8?B?NHNyMW5QakFhWTFKSjlHSU10SE5yc3UyQ1NuRHNvNVBJZmdMc1R1TzJVZUhq?=
 =?utf-8?B?cjdzcHVtSE5rWm5jMURiUWFJTUF1OCtHbUF4MnJ1Z28xaWZhemEwdk9OWjJU?=
 =?utf-8?B?VHpGY05Qck43SzhTZ1hnc0swVzVWMzF0NTJxSXo4N3FDMUhMOHRjV2lDdnow?=
 =?utf-8?B?aHR1UEtlc2VrVEJRcjYxNFpMVGx6NlQwdGIvemtJYStEWjg4TzhSWmx2QWF6?=
 =?utf-8?B?OEJqR0ZWRnUwT0JtRXVucEJsQ1c3WVROLzdNVkd5bUNVRW5heEtTU21FZHpG?=
 =?utf-8?B?MlJlTGtkQklnQzBGc3pBdWE3NmJwQUlOenBMb0ZjTjAxbk4wZmFSUmUrZkNt?=
 =?utf-8?B?QXI3NTJDTW0wVkgxYTd0b0JQRzRGczVmajJSQlVzVFpFUG9scm9tTzJNQXUy?=
 =?utf-8?B?cWZsTnd0TGJqWkdtcEtzYURHTVlveWdjbHZiS1JodzZyQjNUcUxIUkFmdDdM?=
 =?utf-8?B?dTRWeHgvZ0JWanQwRC9CRkdiSWRHTTllQVMvRFpka1hnYitVLzczaTY5dFlR?=
 =?utf-8?B?RktJeWV0Y0NmWjExc09POE1tT0JuQURLcUh5RDNseGlMTkVGZkVQdDdWaWdt?=
 =?utf-8?B?VWtJaU5XMURKYUtjTkkvWERXVm1FR3BKVGZpTGtjRnlldzNHM1pYMm9QR3ZF?=
 =?utf-8?B?d3JXR2RpZTVEeDJaUk1MRVpDdklqRG1pbFlHeGZyVHUrTUlGYVRxeDE0QWY5?=
 =?utf-8?B?N3kzTEFPbnMrU1NuNG9DNVFzV3Rpc1U5UDdSQjBWN3BKVWJuMkJaTkxZaG5G?=
 =?utf-8?B?dURtRy83V1greldJajFERSswOTIrclVHZGg5RGJ4MHNzaCs1WXFIVlJzYStE?=
 =?utf-8?B?UlcwYkZ6M1RSaS95ZFlaVkFjOVIzdTNOWFpqL3Myc0UwZkFKaEpVYjdnT0JL?=
 =?utf-8?B?UnVDZGxTZ2ZiT1lyU0tGWHE5c2xtN3VmVFlGczBoVnJJeVNpR0lqOFo4WHUv?=
 =?utf-8?B?bHRzVnR1S3ByZnVLWEo4c2cvcnMvQXZGVEpsSTZWaXFtSnViZTc0Rm5YODlo?=
 =?utf-8?B?OFpqT25POFZNNnBCU1VYRDhnaW9WUHQ0ZlgwVTJzQk1xT1l2RUpTaFBoYklX?=
 =?utf-8?B?YmhGY0wxd1ZpMVl3OGd6OURhZm45U2dWMWFtdTl1ZkVad2oxUU1taGYveXRT?=
 =?utf-8?B?eTMvQTlCd3JJN0RRRWJMSkh3alJCT0FVckVsRjZVYWZvcXFXVkxYZ0orRUhT?=
 =?utf-8?B?cUJPdGdFc2dDZFJ0VWR3NUQ0VXJzaXdJeHhJTFZFOUxmRmhDeDNxcUMySzlv?=
 =?utf-8?Q?3yMNEt6KIpP9L0vC/5QE3dNla?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 78df9e48-c0e4-4714-e327-08dc00698c02
X-MS-Exchange-CrossTenant-AuthSource: PH7PR12MB5712.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 19 Dec 2023 08:07:30.2073
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: sVNV2DGF/VwmbH8g5Gve9u8dpoiLP+k9KJJt9WPUcpJ7fHc+/LLU+4b0k2c5xbRyIyv8pky04Z6yKeVoe/+KnA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BL3PR12MB6546

On 12/19/2023 2:13 AM, Stephane Eranian wrote:
> On Sun, Dec 17, 2023 at 9:26 AM Sandipan Das <sandipan.das@amd.com> wrote:
>>
>> Currently, it is assumed that LBR Freeze is supported on all processors
>> which have CPUID leaf 0x80000022[EAX] bit 1 set. This is incorrect as
>> the feature availability is additionally dependent on CPUID leaf
>> 0x80000022[EAX] bit 2 being set which may not be set for all Zen 4
>> processors. Define a new feature bit for LBR and PMC freeze and set the
>> freeze enable bit (FLBRI) in DebugCtl (MSR 0x1d9) conditionally.
> 
> 
> Is this new feature bit visible to users?
> I think it is useful to know whether or not LBR freeze is supported.
> Imagine I want to do kernel FDO, then the user-only LBR trick to freeze LBR
> does not work and I need actual LBR freeze support.
> 
> Thanks.
> 

Agreed. Will make it a visible flag in /proc/cpuinfo.

- Sandipan

>> It should still be possible to use LBR without freeze for profile-guided
>> optimization of user programs by using an user-only branch filter during
>> profiling. When the user-only filter is enabled, branches are no longer
>> recorded after the transition to CPL 0 upon PMI arrival. When branch
>> entries are read in the PMI handler, the branch stack does not change.
>>
>> E.g.
>>
>>   $ perf record -j any,u -e ex_ret_brn_tkn ./workload
>>
>> Fixes: ca5b7c0d9621 ("perf/x86/amd/lbr: Add LbrExtV2 branch record support")
>> Signed-off-by: Sandipan Das <sandipan.das@amd.com>
>> Cc: stable@vger.kernel.org
>> ---
>>  arch/x86/events/amd/core.c         |  4 ++--
>>  arch/x86/events/amd/lbr.c          | 16 ++++++++++------
>>  arch/x86/include/asm/cpufeatures.h |  2 +-
>>  arch/x86/kernel/cpu/scattered.c    |  1 +
>>  4 files changed, 14 insertions(+), 9 deletions(-)
>>
>> diff --git a/arch/x86/events/amd/core.c b/arch/x86/events/amd/core.c
>> index 4ee6390b45c9..ffdfaee08b08 100644
>> --- a/arch/x86/events/amd/core.c
>> +++ b/arch/x86/events/amd/core.c
>> @@ -905,8 +905,8 @@ static int amd_pmu_v2_handle_irq(struct pt_regs *regs)
>>         if (!status)
>>                 goto done;
>>
>> -       /* Read branch records before unfreezing */
>> -       if (status & GLOBAL_STATUS_LBRS_FROZEN) {
>> +       /* Read branch records */
>> +       if (x86_pmu.lbr_nr) {
>>                 amd_pmu_lbr_read();
>>                 status &= ~GLOBAL_STATUS_LBRS_FROZEN;
>>         }
>> diff --git a/arch/x86/events/amd/lbr.c b/arch/x86/events/amd/lbr.c
>> index eb31f850841a..110e34c59643 100644
>> --- a/arch/x86/events/amd/lbr.c
>> +++ b/arch/x86/events/amd/lbr.c
>> @@ -400,10 +400,12 @@ void amd_pmu_lbr_enable_all(void)
>>                 wrmsrl(MSR_AMD64_LBR_SELECT, lbr_select);
>>         }
>>
>> -       rdmsrl(MSR_IA32_DEBUGCTLMSR, dbg_ctl);
>> -       rdmsrl(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg);
>> +       if (cpu_feature_enabled(X86_FEATURE_AMD_LBR_PMC_FREEZE)) {
>> +               rdmsrl(MSR_IA32_DEBUGCTLMSR, dbg_ctl);
>> +               wrmsrl(MSR_IA32_DEBUGCTLMSR, dbg_ctl | DEBUGCTLMSR_FREEZE_LBRS_ON_PMI);
>> +       }
>>
>> -       wrmsrl(MSR_IA32_DEBUGCTLMSR, dbg_ctl | DEBUGCTLMSR_FREEZE_LBRS_ON_PMI);
>> +       rdmsrl(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg);
>>         wrmsrl(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg | DBG_EXTN_CFG_LBRV2EN);
>>  }
>>
>> @@ -416,10 +418,12 @@ void amd_pmu_lbr_disable_all(void)
>>                 return;
>>
>>         rdmsrl(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg);
>> -       rdmsrl(MSR_IA32_DEBUGCTLMSR, dbg_ctl);
>> -
>>         wrmsrl(MSR_AMD_DBG_EXTN_CFG, dbg_extn_cfg & ~DBG_EXTN_CFG_LBRV2EN);
>> -       wrmsrl(MSR_IA32_DEBUGCTLMSR, dbg_ctl & ~DEBUGCTLMSR_FREEZE_LBRS_ON_PMI);
>> +
>> +       if (cpu_feature_enabled(X86_FEATURE_AMD_LBR_PMC_FREEZE)) {
>> +               rdmsrl(MSR_IA32_DEBUGCTLMSR, dbg_ctl);
>> +               wrmsrl(MSR_IA32_DEBUGCTLMSR, dbg_ctl & ~DEBUGCTLMSR_FREEZE_LBRS_ON_PMI);
>> +       }
>>  }
>>
>>  __init int amd_pmu_lbr_init(void)
>> diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
>> index 4af140cf5719..9790e906d5e5 100644
>> --- a/arch/x86/include/asm/cpufeatures.h
>> +++ b/arch/x86/include/asm/cpufeatures.h
>> @@ -97,7 +97,7 @@
>>  #define X86_FEATURE_SYSENTER32         ( 3*32+15) /* "" sysenter in IA32 userspace */
>>  #define X86_FEATURE_REP_GOOD           ( 3*32+16) /* REP microcode works well */
>>  #define X86_FEATURE_AMD_LBR_V2         ( 3*32+17) /* AMD Last Branch Record Extension Version 2 */
>> -/* FREE, was #define X86_FEATURE_LFENCE_RDTSC          ( 3*32+18) "" LFENCE synchronizes RDTSC */
>> +#define X86_FEATURE_AMD_LBR_PMC_FREEZE ( 3*32+18) /* "" AMD LBR and PMC Freeze */
>>  #define X86_FEATURE_ACC_POWER          ( 3*32+19) /* AMD Accumulated Power Mechanism */
>>  #define X86_FEATURE_NOPL               ( 3*32+20) /* The NOPL (0F 1F) instructions */
>>  #define X86_FEATURE_ALWAYS             ( 3*32+21) /* "" Always-present feature */
>> diff --git a/arch/x86/kernel/cpu/scattered.c b/arch/x86/kernel/cpu/scattered.c
>> index 0dad49a09b7a..a515328d9d7d 100644
>> --- a/arch/x86/kernel/cpu/scattered.c
>> +++ b/arch/x86/kernel/cpu/scattered.c
>> @@ -49,6 +49,7 @@ static const struct cpuid_bit cpuid_bits[] = {
>>         { X86_FEATURE_BMEC,             CPUID_EBX,  3, 0x80000020, 0 },
>>         { X86_FEATURE_PERFMON_V2,       CPUID_EAX,  0, 0x80000022, 0 },
>>         { X86_FEATURE_AMD_LBR_V2,       CPUID_EAX,  1, 0x80000022, 0 },
>> +       { X86_FEATURE_AMD_LBR_PMC_FREEZE,       CPUID_EAX,  2, 0x80000022, 0 },
>>         { 0, 0, 0, 0, 0 }
>>  };
>>
>> --
>> 2.34.1
>>