Received-SPF: pass (google.com: domain of linux-kernel+bounces-199530-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Received-SPF: Pass (protection.outlook.com: domain of amd.com designates
 165.204.84.17 as permitted sender) receiver=protection.outlook.com;
 client-ip=165.204.84.17; helo=SATLEXMB04.amd.com; pr=C
From: Babu Moger <babu.moger@amd.com>
To: <pbonzini@redhat.com>
CC: <babu.moger@amd.com>, <kvm@vger.kernel.org>,
	<linux-kernel@vger.kernel.org>
Subject: [PATCH] KVM: Fix Undefined Behavior Sanitizer(UBSAN) error
Date: Mon, 3 Jun 2024 12:44:13 -0500
Message-ID: <98ad0dab3a2c66834e50e6d465dcae47dd80758b.1717436464.git.babu.moger@amd.com>
Precedence: bulk
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Content-Type: text/plain
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2024 17:44:19.3994
 (UTC)
X-MS-Exchange-CrossTenant-Network-Message-Id: 56403a4d-5a4d-41f2-d837-08dc83f4cbd2
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-OriginalAttributedTenantConnectingIp: TenantId=3dd8961f-e488-4e60-8e11-a82d994e183d;Ip=[165.204.84.17];Helo=[SATLEXMB04.amd.com]
X-MS-Exchange-CrossTenant-AuthSource:
	SN1PEPF000252A3.namprd05.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Anonymous
X-MS-Exchange-CrossTenant-FromEntityHeader: HybridOnPrem
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR12MB5663

System throws this following UBSAN: invalid-load error when the very first
VM is powered up on a freshly booted host machine. Happens only with 2P or
4P (multiple sockets) systems.

{  688.429145] ------------[ cut here ]------------
[  688.429156] UBSAN: invalid-load in arch/x86/kvm/../../../virt/kvm/kvm_main.c:655:10
[  688.437739] load of value 160 is not a valid value for type '_Bool'
[  688.444760] CPU: 370 PID: 8246 Comm: CPU 0/KVM Not tainted 6.8.2-amdsos-build58-ubuntu-22.04+ #1
[  688.444767] Hardware name: AMD Corporation Sh54p/Sh54p, BIOS WPC4429N 04/25/2024
[  688.444770] Call Trace:
[  688.444777]  <TASK>
[  688.444787]  dump_stack_lvl+0x48/0x60
[  688.444810]  ubsan_epilogue+0x5/0x30
[  688.444823]  __ubsan_handle_load_invalid_value+0x79/0x80
[  688.444827]  ? srso_alias_return_thunk+0x5/0xfbef5
[  688.444836]  ? flush_tlb_func+0xe9/0x2e0
[  688.444845]  kvm_mmu_notifier_invalidate_range_end.cold+0x18/0x4f [kvm]
[  688.444906]  __mmu_notifier_invalidate_range_end+0x63/0xe0
[  688.444917]  __split_huge_pmd+0x367/0xfc0
[  688.444928]  ? srso_alias_return_thunk+0x5/0xfbef5
[  688.444931]  ? alloc_pages_mpol+0x97/0x210
[  688.444941]  do_huge_pmd_wp_page+0x1cc/0x380
[  688.444946]  __handle_mm_fault+0x8ee/0xe50
[  688.444958]  handle_mm_fault+0xe4/0x4a0
[  688.444962]  __get_user_pages+0x190/0x840
[  688.444972]  get_user_pages_unlocked+0xe0/0x590
[  688.444977]  hva_to_pfn+0x114/0x550 [kvm]
[  688.445007]  ? srso_alias_return_thunk+0x5/0xfbef5
[  688.445011]  ? __gfn_to_pfn_memslot+0x3b/0xd0 [kvm]
[  688.445037]  kvm_faultin_pfn+0xed/0x5b0 [kvm]
[  688.445079]  kvm_tdp_page_fault+0x123/0x170 [kvm]
[  688.445109]  kvm_mmu_page_fault+0x244/0xaa0 [kvm]
[  688.445136]  ? srso_alias_return_thunk+0x5/0xfbef5
[  688.445138]  ? kvm_io_bus_get_first_dev+0x56/0xf0 [kvm]
[  688.445165]  ? srso_alias_return_thunk+0x5/0xfbef5
[  688.445171]  ? svm_vcpu_run+0x329/0x7c0 [kvm_amd]
[  688.445186]  vcpu_enter_guest+0x592/0x1070 [kvm]
[  688.445223]  kvm_arch_vcpu_ioctl_run+0x145/0x8a0 [kvm]
[  688.445254]  kvm_vcpu_ioctl+0x288/0x6d0 [kvm]
[  688.445279]  ? vcpu_put+0x22/0x50 [kvm]
[  688.445305]  ? srso_alias_return_thunk+0x5/0xfbef5
[  688.445307]  ? kvm_arch_vcpu_ioctl_run+0x346/0x8a0 [kvm]
[  688.445335]  __x64_sys_ioctl+0x8f/0xd0
[  688.445343]  do_syscall_64+0x77/0x120
[  688.445353]  ? srso_alias_return_thunk+0x5/0xfbef5
[  688.445355]  ? fire_user_return_notifiers+0x42/0x70
[  688.445363]  ? srso_alias_return_thunk+0x5/0xfbef5
[  688.445365]  ? syscall_exit_to_user_mode+0x82/0x1b0
[  688.445372]  ? srso_alias_return_thunk+0x5/0xfbef5
[  688.445377]  ? do_syscall_64+0x86/0x120
[  688.445380]  ? srso_alias_return_thunk+0x5/0xfbef5
[  688.445383]  ? do_syscall_64+0x86/0x120
[  688.445388]  ? srso_alias_return_thunk+0x5/0xfbef5
[  688.445392]  ? do_syscall_64+0x86/0x120
[  688.445396]  ? srso_alias_return_thunk+0x5/0xfbef5
[  688.445400]  ? do_syscall_64+0x86/0x120
[  688.445404]  ? do_syscall_64+0x86/0x120
[  688.445407]  ? do_syscall_64+0x86/0x120
[  688.445410]  entry_SYSCALL_64_after_hwframe+0x6e/0x76
[  688.445416] RIP: 0033:0x7fdf2ed1a94f
[  688.445421] Code: 00 48 89 44 24 18 31 c0 48 8d 44 24 60 c7 04 24 10 00 00 00 48 89
		     44 24 08 48 8d 44 24 20 48 89 44 24 10 b8 10 00 00 00 0f 05 <41>
		     89 c0 3d 00 f0 ff ff 77 1f 48 8b 44 24 18 64 48 2b 04 25 28 00
[  688.445424] RSP: 002b:00007fc127bff460 EFLAGS: 00000246 ORIG_RAX: 0000000000000010
[  688.445429] RAX: ffffffffffffffda RBX: 000000000000ae80 RCX: 00007fdf2ed1a94f
[  688.445432] RDX: 0000000000000000 RSI: 000000000000ae80 RDI: 0000000000000016
[  688.445434] RBP: 00005586f80dc350 R08: 00005586f6a0af10 R09: 00000000ffffffff
[  688.445436] R10: 0000000000000001 R11: 0000000000000246 R12: 0000000000000000
[  688.445438] R13: 0000000000000001 R14: 0000000000000cf8 R15: 0000000000000000
[  688.445443]  </TASK>
[  688.445444] ---[ end trace ]---

However, VM boots up fine without any issues and operational.

The error is due to invalid assignment in kvm invalidate range end path.
There is no arch specific handler for this case and handler is assigned
to kvm_null_fn(). This is an empty function and returns void. Return value
of this function is assigned to boolean variable. UBSAN complains about
this incompatible assignment when kernel is compiled with CONFIG_UBSAN.

Fix the issue by adding a check for the null handler.

Signed-off-by: Babu Moger <babu.moger@amd.com>
---
Seems straight forward fix to me. Point me if you think otherwise. New
to this area of the code. First of all not clear to me why handler need
to be called when memory slot is not found in the hva range.
---
 virt/kvm/kvm_main.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 14841acb8b95..ee8be1835214 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -653,7 +653,8 @@ static __always_inline kvm_mn_ret_t __kvm_handle_hva_range(struct kvm *kvm,
 				if (IS_KVM_NULL_FN(range->handler))
 					break;
 			}
-			r.ret |= range->handler(kvm, &gfn_range);
+			if (!IS_KVM_NULL_FN(range->handler))
+				r.ret |= range->handler(kvm, &gfn_range);
 		}
 	}
 
-- 
2.34.1