Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
From:   "Brown, Len" <len.brown@intel.com>
To:     "Chen, Yu C" <yu.c.chen@intel.com>,
        "linux-pm@vger.kernel.org" <linux-pm@vger.kernel.org>,
        Len Brown <lenb@kernel.org>
CC:     Artem Bityutskiy <artem.bityutskiy@linux.intel.com>,
        "Rafael J. Wysocki" <rafael@kernel.org>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: RE: [PATCH 1/2] tools/power turbostat: Support thermal throttle count
 print
Thread-Topic: [PATCH 1/2] tools/power turbostat: Support thermal throttle
 count print
Thread-Index: AQHX17vmo9Yl0WxUik2a7/rHYiG/gayOKpfQ
Date:   Fri, 11 Feb 2022 02:14:12 +0000
Message-ID: <PH0PR11MB4887077D94A30A5F757F01B4E0309@PH0PR11MB4887.namprd11.prod.outlook.com>
References: <20211112115159.725357-1-yu.c.chen@intel.com>
In-Reply-To: <20211112115159.725357-1-yu.c.chen@intel.com>
Accept-Language: en-US
Content-Language: en-US
dlp-version: 11.6.200.16
dlp-reaction: no-action
dlp-product: dlpe-windows
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?vNSepIwKNL5jE67ceIIwxvFLb96n9yguKMas9S3EoOJPi6dg/luwcZt8b6eG?=
 =?us-ascii?Q?mEpXVvH/TzXQfiiDiWobXEwa8JOcPLe4o/64NdgqNIvXWHaQSxvLu83tmxHy?=
 =?us-ascii?Q?RnwgL84rZSwL5m+pdoo64Vc5OObns+EVhgkOWHHu2MZum3PCsakPhA9mBpeB?=
 =?us-ascii?Q?ntY8gEekjRS2rJUEV55G4AaoiVZ+dHXeJVIkQj1c4Q2womYoR1dFgGY74yV9?=
 =?us-ascii?Q?x1vpsA4+qH82cbZX1GrlW8C/Muw2gINnke3EE0LDgBZ+S/IFKOo0+KZBRJmm?=
 =?us-ascii?Q?Z7W/NV9CXTWfYF4W1anRGirVZKfuAQPEguBsqYFzUcgNOsdkMLPLrXAs501b?=
 =?us-ascii?Q?U3156uN7z8U508lgwNzdBbp4qjMD/BpyiAttv5a5UbpnQZKRz1qzDl+Mvclg?=
 =?us-ascii?Q?xNb+p6Bh8UWSXz2S4aG06TCzfJg9ZbxMO862po3hjVTWuJ5wTZWW7wKGiJqF?=
 =?us-ascii?Q?ofYkX+ZkpLGCXZ9M0v2u5m+Fy0JcXoMnXnAupq0GYg1FgNdL6nVBQkKfYiRN?=
 =?us-ascii?Q?HoygqzxE++aXSfoOvOW3muB/vUyxAezl8oDvPGX6BfmMNYSb62Xt+skQXzbv?=
 =?us-ascii?Q?mI/4wtoNXs7mFA14wxT1cJ04S9ieH0jwbWKpuIUFzxDgjkGqGEwwjUaJAvPF?=
 =?us-ascii?Q?W7rgZQyvi1dp5UpbRu+j3XkVMfFW505e+yS/XJN3Xnsj/9ehTjSxiyvOToC9?=
 =?us-ascii?Q?cqTGXAWo/w+mpTh/c5FNjOrSdZfxBqVyPUPvpKGjExhU4XL4dUlkpEvV09nC?=
 =?us-ascii?Q?Kh3IMyB7EPjpDIfj9+96jPzgOYBl/VfxjrpT8nt8CM1BVAg8TsbPB+pCclsv?=
 =?us-ascii?Q?7j/GrSICgVmY7SGFaO34zLMycBUs1TCWgHNRIGTFV0oXpNIcacDE5HIzszXe?=
 =?us-ascii?Q?CAPi0UuZjaxg8nuvglfeElP+tqomlrTw8JaoXi1h9cBG2iCqOkd+urgtsAys?=
 =?us-ascii?Q?Q3HBZW7bFQN7Z9L2KUemV6ROD5VhTGqEbncJeYZ1fkCKoXcIs6OgiMZIIq2J?=
 =?us-ascii?Q?O47EBxvtx4kxlEQ5HIlFe3n8DS/VLr9Zb0rNoxd2vnbxmjfRna+pA74LLE1o?=
 =?us-ascii?Q?LxLh+IhJzuhL81z0gRiKLTp36YHWcfDLyW1gdvPt4WsXlSLie6dLNOZ0LUcu?=
 =?us-ascii?Q?30mU7qpg0GMtjMiJx3+YzEHQZ0QzGzEkNpKvpZCMItyCfWRUjasNubOtPMlu?=
 =?us-ascii?Q?ZTtwm7nH2Ob6HZrsLoCRwEPwODOahYONkKVQjSLCehbmOClbygd34wEKT3vt?=
 =?us-ascii?Q?x+XMxwaPR0Isdtw7gwJcjIrS23ZODdNcU/NLS6azJ0gya1cq/z4889woIvTO?=
 =?us-ascii?Q?nO0MAMV2NtkKOeC116AVYnArMiMsbiaBVWkPWdfeiuef18BOiGo7ogTe1zgF?=
 =?us-ascii?Q?kEqSMjNFn4bpsehcsnGERuh4L9cB/GGZpk7YBEBmhXXrWfwbBaicrl+UIhjN?=
 =?us-ascii?Q?AN0JQGxTTtOdx1pVyBWUZ8AeZ05ujsxlKUPN5DWn2M9QsEIaJHyEBSCV8OGS?=
 =?us-ascii?Q?VnuSzLmVqEtRC6A6dd0YVoQljzHEKjNnWpqs2O9id+LkiSfRGy7EhG9fSQSR?=
 =?us-ascii?Q?BBGRMIaCicIX779IYrpOFiUGd736HGj5vZOj+hcYrzZMcfUbkkod1t05WZK1?=
 =?us-ascii?Q?BQ=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: PH0PR11MB4887.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 37d07c02-6565-4a92-2d97-08d9ed04325c
X-MS-Exchange-CrossTenant-originalarrivaltime: 11 Feb 2022 02:14:12.8771
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: 8AMdUtBXXsCbv3pfZEQWP5t/IrRi0+igLsnBSG2NkqMDLR1Ey5d9oU56r3A2hYqjAOil2muFHXtlyKLenUaSmA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB5594
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,SPF_HELO_NONE,
        SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

What is the scope of this counter -- per-core or per-CPU?

-----Original Message-----
From: Chen, Yu C <yu.c.chen@intel.com>=20
Sent: Friday, November 12, 2021 6:52 AM
To: linux-pm@vger.kernel.org; Len Brown <lenb@kernel.org>
Cc: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>; Rafael J. Wysocki =
<rafael@kernel.org>; Brown, Len <len.brown@intel.com>; linux-kernel@vger.ke=
rnel.org; Chen, Yu C <yu.c.chen@intel.com>
Subject: [PATCH 1/2] tools/power turbostat: Support thermal throttle count =
print

The turbostat data is collected by end user for power evaluationit. However=
 it looks like we are missing enough thermal context there. Already a coupl=
e of time we found that power management developer asking something like th=
is:
grep -r . /sys/devices/system/cpu/cpu*/thermal_throttle/*

Print the per core thermal throttle count so as to get suffificent thermal =
context.

turbostat -i 5 -s Core,CPU,CoreThr
Core	CPU	CoreThr
-	-	104
0	0	61
0	4
1	1	0
1	5
2	2	104
2	6
3	3	7
3	7

Suggested-by: Artem Bityutskiy <artem.bityutskiy@linux.intel.com>
Signed-off-by: Chen Yu <yu.c.chen@intel.com>
---
 tools/power/x86/turbostat/turbostat.c | 43 +++++++++++++++++++++++++++
 1 file changed, 43 insertions(+)

diff --git a/tools/power/x86/turbostat/turbostat.c b/tools/power/x86/turbos=
tat/turbostat.c
index 47d3ba895d6d..b7908f1ee969 100644
--- a/tools/power/x86/turbostat/turbostat.c
+++ b/tools/power/x86/turbostat/turbostat.c
@@ -205,6 +205,7 @@ struct core_data {
 	unsigned int core_temp_c;
 	unsigned int core_energy;	/* MSR_CORE_ENERGY_STAT */
 	unsigned int core_id;
+	unsigned long long core_throt_cnt;
 	unsigned long long counter[MAX_ADDED_COUNTERS];  } *core_even, *core_odd;
=20
@@ -610,6 +611,7 @@ struct msr_counter bic[] =3D {
 	{ 0x0, "Die" },
 	{ 0x0, "GFXAMHz" },
 	{ 0x0, "IPC" },
+	{ 0x0, "CoreThr" },
 };
=20
 #define MAX_BIC (sizeof(bic) / sizeof(struct msr_counter)) @@ -666,6 +668,=
7 @@ struct msr_counter bic[] =3D {
 #define	BIC_Die		(1ULL << 50)
 #define	BIC_GFXACTMHz	(1ULL << 51)
 #define	BIC_IPC		(1ULL << 52)
+#define	BIC_CORE_THROT_CNT	(1ULL << 53)
=20
 #define BIC_TOPOLOGY (BIC_Package | BIC_Node | BIC_CoreCnt | BIC_PkgCnt | =
BIC_Core | BIC_CPU | BIC_Die )  #define BIC_THERMAL_PWR ( BIC_CoreTmp | BIC=
_PkgTmp | BIC_PkgWatt | BIC_CorWatt | BIC_GFXWatt | BIC_RAMWatt | BIC_PKG__=
 | BIC_RAM__) @@ -872,6 +875,9 @@ void print_header(char *delim)
 	if (DO_BIC(BIC_CoreTmp))
 		outp +=3D sprintf(outp, "%sCoreTmp", (printed++ ? delim : ""));
=20
+	if (DO_BIC(BIC_CORE_THROT_CNT))
+		outp +=3D sprintf(outp, "%sCoreThr", (printed++ ? delim : ""));
+
 	if (do_rapl && !rapl_joules) {
 		if (DO_BIC(BIC_CorWatt) && (do_rapl & RAPL_PER_CORE_ENERGY))
 			outp +=3D sprintf(outp, "%sCorWatt", (printed++ ? delim : "")); @@ -101=
1,6 +1017,7 @@ int dump_counters(struct thread_data *t, struct core_data *c=
, struct pkg_data *p
 		outp +=3D sprintf(outp, "c6: %016llX\n", c->c6);
 		outp +=3D sprintf(outp, "c7: %016llX\n", c->c7);
 		outp +=3D sprintf(outp, "DTS: %dC\n", c->core_temp_c);
+		outp +=3D sprintf(outp, "cpu_throt_count: %016llX\n",=20
+c->core_throt_cnt);
 		outp +=3D sprintf(outp, "Joules: %0X\n", c->core_energy);
=20
 		for (i =3D 0, mp =3D sys.cp; mp; i++, mp =3D mp->next) { @@ -1225,6 +123=
2,10 @@ int format_counters(struct thread_data *t, struct core_data *c, str=
uct pkg_data
 	if (DO_BIC(BIC_CoreTmp))
 		outp +=3D sprintf(outp, "%s%d", (printed++ ? delim : ""), c->core_temp_c=
);
=20
+	/* Core throttle count */
+	if (DO_BIC(BIC_CORE_THROT_CNT))
+		outp +=3D sprintf(outp, "%s%lld", (printed++ ? delim : ""),=20
+c->core_throt_cnt);
+
 	for (i =3D 0, mp =3D sys.cp; mp; i++, mp =3D mp->next) {
 		if (mp->format =3D=3D FORMAT_RAW) {
 			if (mp->width =3D=3D 32)
@@ -1311,6 +1322,7 @@ int format_counters(struct thread_data *t, struct cor=
e_data *c, struct pkg_data
 	if (DO_BIC(BIC_PkgWatt))
 		outp +=3D
 		    sprintf(outp, fmt8, (printed++ ? delim : ""), p->energy_pkg * rapl_e=
nergy_units / interval_float);
+
 	if (DO_BIC(BIC_CorWatt) && !(do_rapl & RAPL_PER_CORE_ENERGY))
 		outp +=3D
 		    sprintf(outp, fmt8, (printed++ ? delim : ""), p->energy_cores * rapl=
_energy_units / interval_float); @@ -1467,6 +1479,7 @@ void delta_core(stru=
ct core_data *new, struct core_data *old)
 	old->c6 =3D new->c6 - old->c6;
 	old->c7 =3D new->c7 - old->c7;
 	old->core_temp_c =3D new->core_temp_c;
+	old->core_throt_cnt =3D new->core_throt_cnt;
 	old->mc6_us =3D new->mc6_us - old->mc6_us;
=20
 	DELTA_WRAP32(new->core_energy, old->core_energy); @@ -1626,6 +1639,7 @@ v=
oid clear_counters(struct thread_data *t, struct core_data *c, struct pkg_d=
ata
 	c->mc6_us =3D 0;
 	c->core_temp_c =3D 0;
 	c->core_energy =3D 0;
+	c->core_throt_cnt =3D 0;
=20
 	p->pkg_wtd_core_c0 =3D 0;
 	p->pkg_any_core_c0 =3D 0;
@@ -1710,6 +1724,7 @@ int sum_counters(struct thread_data *t, struct core_d=
ata *c, struct pkg_data *p)
 	average.cores.mc6_us +=3D c->mc6_us;
=20
 	average.cores.core_temp_c =3D MAX(average.cores.core_temp_c, c->core_temp=
_c);
+	average.cores.core_throt_cnt =3D MAX(average.cores.core_throt_cnt,=20
+c->core_throt_cnt);
=20
 	average.cores.core_energy +=3D c->core_energy;
=20
@@ -1987,6 +2002,26 @@ void get_apic_id(struct thread_data *t)
 		fprintf(outf, "cpu%d: BIOS BUG: apic 0x%x x2apic 0x%x\n", t->cpu_id, t->=
apic_id, t->x2apic_id);  }
=20
+int get_core_throt_cnt(int cpu, unsigned long long *cnt) {
+	char path[128 + PATH_BYTES];
+	unsigned long long tmp;
+	FILE *fp;
+	int ret;
+
+	sprintf(path, "/sys/devices/system/cpu/cpu%d/thermal_throttle/core_thrott=
le_count", cpu);
+	fp =3D fopen(path, "r");
+	if (!fp)
+		return -1;
+	ret =3D fscanf(fp, "%lld", &tmp);
+	if (ret !=3D 1)
+		return -1;
+	fclose(fp);
+	*cnt =3D tmp;
+
+	return 0;
+}
+
 /*
  * get_counters(...)
  * migrate to cpu
@@ -2129,6 +2164,9 @@ int get_counters(struct thread_data *t, struct core_d=
ata *c, struct pkg_data *p)
 		c->core_temp_c =3D tj_max - ((msr >> 16) & 0x7F);
 	}
=20
+	if (DO_BIC(BIC_CORE_THROT_CNT))
+		get_core_throt_cnt(cpu, &c->core_throt_cnt);
+
 	if (do_rapl & RAPL_AMD_F17H) {
 		if (get_msr(cpu, MSR_CORE_ENERGY_STAT, &msr))
 			return -14;
@@ -5572,6 +5610,11 @@ void process_cpuid()
 	else
 		BIC_NOT_PRESENT(BIC_CPU_LPI);
=20
+	if (!access("/sys/devices/system/cpu/cpu0/thermal_throttle/core_throttle_=
count", R_OK))
+		BIC_PRESENT(BIC_CORE_THROT_CNT);
+	else
+		BIC_NOT_PRESENT(BIC_CORE_THROT_CNT);
+
 	if (!access(sys_lpi_file_sysfs, R_OK)) {
 		sys_lpi_file =3D sys_lpi_file_sysfs;
 		BIC_PRESENT(BIC_SYS_LPI);
--
2.25.1