Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Message-ID: <8798a3cc-6e64-6966-d5ad-fadca79d92ba@synaptics.com>
Date:   Mon, 7 Feb 2022 16:46:42 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Content-Language: en-US
To:     Catalin Marinas <catalin.marinas@arm.com>,
        Will Deacon <will@kernel.org>
Cc:     linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org
From:   Jisheng Zhang <Jisheng.Zhang@synaptics.com>
Subject: [PATCH] arm64: Add read_mostly declaration/definition to irq stack
 ptr
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?N2lQTGd2ZGwwT3J3RGhvUzU0cUwwNHlUQmw1YzNDc3d1SXF0MXM1dFo1cGFT?=
 =?utf-8?B?WmorbTM3dzlVSDRtN2R4MlJ3TU1WdU1mZnZyei9haVJwY09ZREEvL3ZlWEtC?=
 =?utf-8?B?RmVQRGxUeS81SUpvTFpGNlFIZnliMG1Rdy9ZYnpWVWZzbGQ2aXpnOGhvOVg0?=
 =?utf-8?B?Z0pKeHplMUgxbmxjc095ekI1TWgrNndHY1BCbVd4azRsNFJsZVdKdE5MNTVF?=
 =?utf-8?B?dXdOMHpLelVoSEEzSGI2RUQ1dTJvMXdDcEtVMVFQaTljNzNoRkc4MlUwelZz?=
 =?utf-8?B?V1RjSWYxcDZKSDMvYyt6Z1VCbzBhNmdzMUhlcGhUaERuVWhuWDNzZFB0NTN6?=
 =?utf-8?B?MDlaT283bTd2NGZURVovTFdOSXhOZXJIMk1xNFBwZGxrRUgzRDRJMXQ4OVFO?=
 =?utf-8?B?bDVmRXJWeERpY3h5Nm9qU3ErdERZaVJVejYwM2p6ZEplZUtpc09iNWRwak80?=
 =?utf-8?B?V2JkQktaSUlJbDkyeVlOUUlPZ2Z5aXFIOVRoRUVHanpoZWNTV094OVVjNElE?=
 =?utf-8?B?MFN5WW0xbmJTeFgyNFNrekNoNGZiZ0k2WFpSMHRXZ1Q2djJ3cStFalo4N2Fq?=
 =?utf-8?B?SHFQTERGYUZlY0RyV0JCakQ0bjZMU0Y5TDRYWTNkV3JWaHBwa2JxNGordHFZ?=
 =?utf-8?B?SUJxZ09WZGJQT2pjSzJZaTV2YVBxaVYzL0wwN0VEVlFtVHFYU1Z6SGwwSFJj?=
 =?utf-8?B?b0czbjZXbHlxbVdidDdDSmhsemQ1aXl0U2p0dVlFVnlwd0oyR0N1MVJ1bkVT?=
 =?utf-8?B?VU0wQWE4aHB2U04zS2sxbmFTUWtQMnRBRGU4WHJjek9UWWlVM1lacnYyTzhE?=
 =?utf-8?B?OXpQamNOTGlJZExRMXdOU0ZTNENLZVFLOVNaNmxKSko0NzNJU05FYWdaeDhB?=
 =?utf-8?B?dWNSSW1qZEE5QktFdndVTWhkek1qN1ZtM0t6eTh0VDZPY3prZGduRXJHZ2tq?=
 =?utf-8?B?NGhHK0NlY3B5a3UwRlRiZnZWYjdwZTBwcWxkWSt0d2JLbk9FV0RNZWRqd2Mr?=
 =?utf-8?B?bmJ2SUpPQUdwdmpDRW1aZnQ1ek45Sy9FMHRGTi9PUkcvZEF0NU1HK1I2REVC?=
 =?utf-8?B?QXNoMUNpc3FETEJTbXlsd3FkVTh0NVgrVG55bGk0Z3RudVFqRzU2L0tNbGJ2?=
 =?utf-8?B?SFZJajRwRmlhdEQ0c3JjdDFxbFFmY0w3c21qdXJROVJKNTBnWmlQbmNlbFVy?=
 =?utf-8?B?ZHovelBDZWZZRnFVWkN4Z2NGZmQ1RW8zRXArdTdiZktXZGZyaUZ1UHg3aGJV?=
 =?utf-8?B?RmVFZWhRWTA0bUVLZnN2OXBIOEpJbG9ENnhGcVAxT2xCRURYOEV6aGJuSy9w?=
 =?utf-8?B?NEdlc2w5OUl1MS9IcU9lQ2lkVDRHTVdGSldlbi80UmRyVGVWUGd6S3RkUGZQ?=
 =?utf-8?B?MFZQRVgvSklZRGRnZEtLUVhQdzZCZDM1TVdvVzNFL1RKbTBOMkZPQWQxWlYw?=
 =?utf-8?B?OEZ4dENzR2hHQW9aQ01kYU5TMFlZYjVpam9xQlFrSjFrZEQzRnVPdU1FVXRz?=
 =?utf-8?B?Ukh2ZjluSGw0emhxZWxhU2cxL2lKV0hWVTVtWlJEWlJIMkY2emR0ZUxUV2Rt?=
 =?utf-8?B?dmFVbjhlMFd2UklPdE5XV2ljL24xdWRmTUpZeG40ZE9qajFjY2F5Q0RJWmRW?=
 =?utf-8?B?VW5PVjBYd1JwdjdLSFZJRTJKUzBHcDljNTBnY1hpV1FMSTFTRkcxRHRJSjM5?=
 =?utf-8?B?d051WmRXSzR5QTRHN3UvT2dTL2tVYjA3VG9mY0NZa21ZWUhMcjRjYXY2dk1Q?=
 =?utf-8?B?WGZadlFieUdXNHg0TUNrTkg3eG5xczhNZnpiYVlWQkxYU0VhZzR3eUozOGJr?=
 =?utf-8?B?TXZJd0lQL3VxNXRTUDgybGxWZlVSVXFJQ1ZMRXR3RGo5T0d0TThSNGd1RkRt?=
 =?utf-8?B?MXI1ZWl0TUhWSlNRYyszNzlIc3NRZmd1SmdzSnRvVEZFU3pLMEYwTkRqaGVr?=
 =?utf-8?B?K2FrQ0xmSUFsYjRtbzlLTC9LUVVKZUlCSlM5QTNIamtNVnEyMzFOc2szM2VT?=
 =?utf-8?B?NjZwQktSa1N4cElxamhaTVNLQUxQbTljL20rWmpqZUR2YkpxMlN6aXNlWGg5?=
 =?utf-8?B?TzZTVU1Zb0pDRXBEMXhmdkgzZ1FHNjZZdm13ZW9QUUdqQXRyc1hSSnM2dzdt?=
 =?utf-8?B?c0UyRkhMZVdWSVFOSmoxb3pXMCtLSmZzWTRWUU9DZGtkZ3ZZQnZHLzd2NWdo?=
 =?utf-8?Q?wymYXCZwp8x2v0QQ5OigX8w=3D?=
X-OriginatorOrg: synaptics.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 10b36d49-414a-4388-a5b0-08d9ea166239
X-MS-Exchange-CrossTenant-AuthSource: BN9PR03MB6058.namprd03.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 07 Feb 2022 08:46:51.0260
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 335d1fbc-2124-4173-9863-17e7051a2a0e
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: kRQItADrkUgATxS+9dNJ0UrE6j6WrNc+HULmWj71mbT5whm8Yk9SS3VGwdwRQkSpynEq3VG+yMPn9vs2nHYhkA==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MWHPR03MB2813
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED,
	DKIM_VALID,HEADER_FROM_DIFFERENT_DOMAINS,MAILING_LIST_MULTI,RDNS_NONE,
	SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no
	version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

Add "read-mostly" qualifier to irq_stack_ptr and
irq_shadow_call_stack_ptr. This is to prevent the false sharing.

Before the patch, I got below percpu layout with one defconfig:
ffffffc008723050 <mde_ref_count>:
ffffffc008723050:       00 00 00 00
....

ffffffc008723054 <kde_ref_count>:
ffffffc008723054:       00 00 00 00
....

ffffffc008723058 <irq_stack_ptr>:
        ...

ffffffc008723060 <nmi_contexts>:
        ...

ffffffc008723070 <fpsimd_last_state>:

As can be seen, the irq_stack_ptr sits with the heavy read/write percpu
vars such as fpsimd_last_state etc. at the same cacheline.

After the patch:

ffffffc008723000 <irq_stack_ptr>:
        ...

ffffffc008723008 <cpu_number>:
        ...

ffffffc008723010 <arm64_ssbd_callback_required>:
        ...

ffffffc008723018 <bp_hardening_data>:
        ...

Now, the irq_stack_ptr sits with read mostly percpu vars such as
cpu_number etc. at the same cacheline.

Signed-off-by: Jisheng Zhang <Jisheng.Zhang@synaptics.com>
---
 arch/arm64/include/asm/stacktrace.h | 2 +-
 arch/arm64/kernel/irq.c             | 7 +++----
 2 files changed, 4 insertions(+), 5 deletions(-)

diff --git a/arch/arm64/include/asm/stacktrace.h b/arch/arm64/include/asm/stacktrace.h
index e77cdef9ca29..75c142bfdffe 100644
--- a/arch/arm64/include/asm/stacktrace.h
+++ b/arch/arm64/include/asm/stacktrace.h
@@ -66,7 +66,7 @@ struct stackframe {
 extern void dump_backtrace(struct pt_regs *regs, struct task_struct *tsk,
 			   const char *loglvl);
 
-DECLARE_PER_CPU(unsigned long *, irq_stack_ptr);
+DECLARE_PER_CPU_READ_MOSTLY(unsigned long *, irq_stack_ptr);
 
 static inline bool on_stack(unsigned long sp, unsigned long size,
 			    unsigned long low, unsigned long high,
diff --git a/arch/arm64/kernel/irq.c b/arch/arm64/kernel/irq.c
index bda49430c9ea..d2e75e9bb826 100644
--- a/arch/arm64/kernel/irq.c
+++ b/arch/arm64/kernel/irq.c
@@ -26,13 +26,12 @@
 /* Only access this in an NMI enter/exit */
 DEFINE_PER_CPU(struct nmi_ctx, nmi_contexts);
 
-DEFINE_PER_CPU(unsigned long *, irq_stack_ptr);
+DEFINE_PER_CPU_READ_MOSTLY(unsigned long *, irq_stack_ptr);
 
-
-DECLARE_PER_CPU(unsigned long *, irq_shadow_call_stack_ptr);
+DECLARE_PER_CPU_READ_MOSTLY(unsigned long *, irq_shadow_call_stack_ptr);
 
 #ifdef CONFIG_SHADOW_CALL_STACK
-DEFINE_PER_CPU(unsigned long *, irq_shadow_call_stack_ptr);
+DEFINE_PER_CPU_READ_MOSTLY(unsigned long *, irq_shadow_call_stack_ptr);
 #endif
 
 static void init_irq_scs(void)
-- 
2.34.1