Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Ankur Arora <ankur.a.arora@oracle.com>
To:     linux-kernel@vger.kernel.org, linux-mm@kvack.org, x86@kernel.org
Cc:     mingo@kernel.org, bp@alien8.de, luto@kernel.org,
        akpm@linux-foundation.org, mike.kravetz@oracle.com,
        jon.grimm@amd.com, kvm@vger.kernel.org, konrad.wilk@oracle.com,
        boris.ostrovsky@oracle.com, Ankur Arora <ankur.a.arora@oracle.com>
Subject: [PATCH v2 05/14] x86/cpuid: add X86_FEATURE_MOVNT_SLOW
Date:   Wed, 20 Oct 2021 10:02:56 -0700
Message-Id: <20211020170305.376118-6-ankur.a.arora@oracle.com>
In-Reply-To: <20211020170305.376118-1-ankur.a.arora@oracle.com>
References: <20211020170305.376118-1-ankur.a.arora@oracle.com>
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?qywhtWxTqBggR6jadk+/hncKRxLkFYiaQH4imAE8Tr7bagDR6gqOhj/SrgyR?=
 =?us-ascii?Q?TgzcyFv+1Z8Rh80iJ+wxZ8ImNsTWo0+I/OFQGOd9iUxDNUjPc3ekTeyQ7OEs?=
 =?us-ascii?Q?JqFWAH/iZ3OIpSfdwiu4EAM1hGYCaamJxzE0Ytzq27y3DZUzUf8AyXqJZ+az?=
 =?us-ascii?Q?E4TYKWZaO4+EW49kdZpdABSAcEVFwbTsMJygDGPYg3WS2+1ChtUmj4EtbqMm?=
 =?us-ascii?Q?opbg3oLw3BJWg7BxizS6giSbjrjXxuWcWxieG7t9YCMxetxxFilAW+da7xlU?=
 =?us-ascii?Q?qxcdU1MYz9R+N3X4kETd2GtRyWlKcy7AD2pyehrCrnBWA9bics8P8uZbIiOo?=
 =?us-ascii?Q?J0LoupumXvdPkiZFz3oowHoFQaDDX4So4LGB5c/7BbergFA8b8NMn97ZPVZv?=
 =?us-ascii?Q?RVV1DS0610cNgSlcxI0oQIjtkHq/G1hjnrROvWFiCNTCSYa0+7cgKDuf5AOV?=
 =?us-ascii?Q?kNhkguIjRnsZ9Ri1cRCpep0eUmg9hgeV4+Axyc//X64NptSZShtI1lV5Sh3h?=
 =?us-ascii?Q?558kmlL9e1kQNdYMp4CufQjqoVSu043tGjWaYO+QLgZDjZaP69niMdz0+ML4?=
 =?us-ascii?Q?bPFl2T34fGSbHybjilLcqcIl29Q8fhiNbBuTnDGDAMC12qyQKc89W8sdkTWW?=
 =?us-ascii?Q?VWzdKkMx6m+j0RJ5VOIDYnqidWlkYgSuB7uWOvy0VE0NkIvz8MrmoTkQHiBD?=
 =?us-ascii?Q?1N2D/RdcLHI7zpokf4WfNg8JlW6dnnIurEsXEXkSeUnzkJifUtRs7O+AdGZU?=
 =?us-ascii?Q?BrXcCOmyrjPbzDbo6Fx86BrzSGuQ174kUWqPjXbNm5NgxuMy+IbfZQv/FW3n?=
 =?us-ascii?Q?Qqt0gHfyAkic7PCvdL6eLGE1HYbJd5TrwTkkh4NHMVbatWnLmcudJ6+Lb1EM?=
 =?us-ascii?Q?p4c25O7P60ov6mIWfh4UjMIbu7mLeGtLpOxGhq2AeGv8W3zZEjb+q2xkEeWE?=
 =?us-ascii?Q?lODlH/USWnVp5a2yPxCtMHHG+ATub8gKnLnUXb65oeIfG6L75Sxv8HueC0Ix?=
 =?us-ascii?Q?/n8R8Cq8HvfjcFchU5pAoZHclwqYTApjXm8jgLTnZFtyk6A40U1y8Uz44OC2?=
 =?us-ascii?Q?O0zpuN3Bb/+DJkhO/K69Fu++nua4nz02OPGK4Is7eYRbrbXFav4wXXuCsSzG?=
 =?us-ascii?Q?K1rn0HJPm2YNZvILpaKwO7CBZC5gnzXQG9Qgdqnw9+pb3B1g1tl0B9UcEXgN?=
 =?us-ascii?Q?SvV4I1jskyrmTFzpiJZlPwibDhyBoF413ujZvwO5I7EV0J/hHymHCzEEVgsk?=
 =?us-ascii?Q?+PavIYAQ0YRrCsKZm+dx+nNAnYjx1/xxXGQswBRfg7sfT3MxtZoYZQ2utryM?=
 =?us-ascii?Q?o+5B8gL2n35/zl0wiuqDmESvTbHGjKFRcES/YSVJVIQkbbGEXsXuR8YToHLT?=
 =?us-ascii?Q?iy/3zledIQreI4L+4AJ+CNiLLXXo11z86RK3LJOqKfsyxVAWh+JNvXyvmnvo?=
 =?us-ascii?Q?SYitjxU7AMWQgcrD6Tbklp7Hrg8+GsOi?=
X-OriginatorOrg: oracle.com
X-MS-Exchange-CrossTenant-Network-Message-Id: dfa51988-de29-4053-4799-08d993ebd019
X-MS-Exchange-CrossTenant-AuthSource: CO6PR10MB5409.namprd10.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Oct 2021 17:05:27.0089
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 4e2c6054-71cb-48f1-bd6c-3a9705aca71b
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: ankur.a.arora@oracle.com
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CO1PR10MB4577
X-Proofpoint-Virus-Version: vendor=nai engine=6300 definitions=10143 signatures=668683
X-Proofpoint-Spam-Details: rule=notspam policy=default score=0 mlxlogscore=999 suspectscore=0
 malwarescore=0 bulkscore=0 phishscore=0 adultscore=0 spamscore=0
 mlxscore=0 classifier=spam adjust=0 reason=mlx scancount=1
 engine=8.12.0-2109230001 definitions=main-2110200096
X-Proofpoint-GUID: I1FTLLqN1gywCPHo1sRsT0WJjUjXaZo6
X-Proofpoint-ORIG-GUID: I1FTLLqN1gywCPHo1sRsT0WJjUjXaZo6
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Enabled on microarchitectures where MOVNT is slower for
bulk page clearing than the standard cached clear_page()
idiom.

Also add check_movnt_quirks() where we would set this.

Signed-off-by: Ankur Arora <ankur.a.arora@oracle.com>
---
 arch/x86/include/asm/cpufeatures.h |  1 +
 arch/x86/kernel/cpu/amd.c          |  2 ++
 arch/x86/kernel/cpu/bugs.c         | 15 +++++++++++++++
 arch/x86/kernel/cpu/cpu.h          |  2 ++
 arch/x86/kernel/cpu/intel.c        |  1 +
 5 files changed, 21 insertions(+)

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.h
index d0ce5cfd3ac1..69191f175c2c 100644
--- a/arch/x86/include/asm/cpufeatures.h
+++ b/arch/x86/include/asm/cpufeatures.h
@@ -294,6 +294,7 @@
 #define X86_FEATURE_PER_THREAD_MBA	(11*32+ 7) /* "" Per-thread Memory Bandwidth Allocation */
 #define X86_FEATURE_SGX1		(11*32+ 8) /* "" Basic SGX */
 #define X86_FEATURE_SGX2		(11*32+ 9) /* "" SGX Enclave Dynamic Memory Management (EDMM) */
+#define X86_FEATURE_MOVNT_SLOW		(11*32+10) /* MOVNT is slow. (see check_movnt_quirks()) */
 
 /* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */
 #define X86_FEATURE_AVX_VNNI		(12*32+ 4) /* AVX VNNI instructions */
diff --git a/arch/x86/kernel/cpu/amd.c b/arch/x86/kernel/cpu/amd.c
index 2131af9f2fa2..5de83c6fe526 100644
--- a/arch/x86/kernel/cpu/amd.c
+++ b/arch/x86/kernel/cpu/amd.c
@@ -915,6 +915,8 @@ static void init_amd(struct cpuinfo_x86 *c)
 	if (c->x86 >= 0x10)
 		set_cpu_cap(c, X86_FEATURE_REP_GOOD);
 
+	check_movnt_quirks(c);
+
 	/* get apicid instead of initial apic id from cpuid */
 	c->apicid = hard_smp_processor_id();
 
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index ecfca3bbcd96..4e1558d22a5f 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -84,6 +84,21 @@ EXPORT_SYMBOL_GPL(mds_idle_clear);
  */
 DEFINE_STATIC_KEY_FALSE(switch_mm_cond_l1d_flush);
 
+void check_movnt_quirks(struct cpuinfo_x86 *c)
+{
+#ifdef CONFIG_X86_64
+	/*
+	 * Check if MOVNT is slower than the model specific clear_page()
+	 * idiom (movq/rep-stosb/rep-stosq etc) for bulk page clearing.
+	 * (Bulk is defined here as LLC-sized or larger.)
+	 *
+	 * Condition this check on CONFIG_X86_64 so we don't have
+	 * to worry about any CONFIG_X86_32 families that don't
+	 * support SSE2/MOVNT.
+	 */
+#endif /* CONFIG_X86_64*/
+}
+
 void __init check_bugs(void)
 {
 	identify_boot_cpu();
diff --git a/arch/x86/kernel/cpu/cpu.h b/arch/x86/kernel/cpu/cpu.h
index 95521302630d..72e3715d63ea 100644
--- a/arch/x86/kernel/cpu/cpu.h
+++ b/arch/x86/kernel/cpu/cpu.h
@@ -83,4 +83,6 @@ extern void update_srbds_msr(void);
 
 extern u64 x86_read_arch_cap_msr(void);
 
+void check_movnt_quirks(struct cpuinfo_x86 *c);
+
 #endif /* ARCH_X86_CPU_H */
diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.c
index 8321c43554a1..36a2f8e88b74 100644
--- a/arch/x86/kernel/cpu/intel.c
+++ b/arch/x86/kernel/cpu/intel.c
@@ -666,6 +666,7 @@ static void init_intel(struct cpuinfo_x86 *c)
 		c->x86_cache_alignment = c->x86_clflush_size * 2;
 	if (c->x86 == 6)
 		set_cpu_cap(c, X86_FEATURE_REP_GOOD);
+	check_movnt_quirks(c);
 #else
 	/*
 	 * Names for the Pentium II/Celeron processors
-- 
2.29.2