Received-SPF: pass (google.com: domain of linux-kernel+bounces-164503-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.48.161 as permitted sender) client-ip=147.75.48.161;
Date: Tue, 30 Apr 2024 19:23:13 +0200
From: Borislav Petkov <bp@alien8.de>
To: kernel test robot <oliver.sang@intel.com>,
	Sean Christopherson <seanjc@google.com>
Cc: oe-lkp@lists.linux.dev, lkp@intel.com, linux-kernel@vger.kernel.org,
	x86@kernel.org, Ingo Molnar <mingo@kernel.org>,
	Srikanth Aithal <sraithal@amd.com>
Subject: Re: [tip:x86/alternatives] [x86/alternatives] ee8962082a:
 WARNING:at_arch/x86/kernel/cpu/cpuid-deps.c:#do_clear_cpu_cap
Message-ID: <20240430172313.GCZjEpAfUECkEZ9S5L@fat_crate.local>
References: <202404302233.f27f91b2-oliver.sang@intel.com>
Precedence: bulk
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
In-Reply-To: <202404302233.f27f91b2-oliver.sang@intel.com>

+ Sean.

On Tue, Apr 30, 2024 at 11:00:52PM +0800, kernel test robot wrote:
> 
> 
> Hello,
> 
> kernel test robot noticed "WARNING:at_arch/x86/kernel/cpu/cpuid-deps.c:#do_clear_cpu_cap" on:
> 
> commit: ee8962082a4413dba1a1b3d3d23490c5221f3b8a ("x86/alternatives: Catch late X86_FEATURE modifiers")
> https://git.kernel.org/cgit/linux/kernel/git/tip/tip.git x86/alternatives
> 
> [test failed on linux-next/master bb7a2467e6beef44a80a17d45ebf2931e7631083]
> 
> in testcase: lkvs
> version: lkvs-x86_64-b07d44a-1_20240401
> with following parameters:
> 
> 	test: xsave
> 
> 
> 
> compiler: gcc-13
> test machine: 36 threads 1 sockets Intel(R) Core(TM) i9-10980XE CPU @ 3.00GHz (Cascade Lake) with 32G memory
> 
> (please refer to attached dmesg/kmsg for entire log/backtrace)
> 
> 
> 
> If you fix the issue in a separate patch/commit (i.e. not just a new version of
> the same patch/commit), kindly add following tags
> | Reported-by: kernel test robot <oliver.sang@intel.com>
> | Closes: https://lore.kernel.org/oe-lkp/202404302233.f27f91b2-oliver.sang@intel.com
> 
> 
> [    0.055225][    T0] ------------[ cut here ]------------
> [ 0.055225][ T0] WARNING: CPU: 1 PID: 0 at arch/x86/kernel/cpu/cpuid-deps.c:118 do_clear_cpu_cap (arch/x86/kernel/cpu/cpuid-deps.c:118 (discriminator 1)) 
> [    0.055225][    T0] Modules linked in:
> [    0.055225][    T0] CPU: 1 PID: 0 Comm: swapper/1 Not tainted 6.9.0-rc3-00001-gee8962082a44 #1
> [    0.055225][    T0] Hardware name: Gigabyte Technology Co., Ltd. X299 UD4 Pro/X299 UD4 Pro-CF, BIOS F8a 04/27/2021
> [ 0.055225][ T0] RIP: 0010:do_clear_cpu_cap (arch/x86/kernel/cpu/cpuid-deps.c:118 (discriminator 1)) 
> [ 0.055225][ T0] Code: 89 c1 83 e0 07 48 c1 e9 03 83 c0 03 0f b6 14 11 38 d0 7c 08 84 d2 0f 85 b7 00 00 00 8b 15 f4 f7 7c 04 85 d2 0f 84 f2 fd ff ff <0f> 0b e9 eb fd ff ff 48 c7 c7 c0 eb 89 85 e8 41 fd ff ff 49 8d bf
> All code
> ========
>    0:	89 c1                	mov    %eax,%ecx
>    2:	83 e0 07             	and    $0x7,%eax
>    5:	48 c1 e9 03          	shr    $0x3,%rcx
>    9:	83 c0 03             	add    $0x3,%eax
>    c:	0f b6 14 11          	movzbl (%rcx,%rdx,1),%edx
>   10:	38 d0                	cmp    %dl,%al
>   12:	7c 08                	jl     0x1c
>   14:	84 d2                	test   %dl,%dl
>   16:	0f 85 b7 00 00 00    	jne    0xd3
>   1c:	8b 15 f4 f7 7c 04    	mov    0x47cf7f4(%rip),%edx        # 0x47cf816
>   22:	85 d2                	test   %edx,%edx
>   24:	0f 84 f2 fd ff ff    	je     0xfffffffffffffe1c
>   2a:*	0f 0b                	ud2    		<-- trapping instruction
>   2c:	e9 eb fd ff ff       	jmpq   0xfffffffffffffe1c
>   31:	48 c7 c7 c0 eb 89 85 	mov    $0xffffffff8589ebc0,%rdi
>   38:	e8 41 fd ff ff       	callq  0xfffffffffffffd7e
>   3d:	49                   	rex.WB
>   3e:	8d                   	.byte 0x8d
>   3f:	bf                   	.byte 0xbf
> 
> Code starting with the faulting instruction
> ===========================================
>    0:	0f 0b                	ud2    
>    2:	e9 eb fd ff ff       	jmpq   0xfffffffffffffdf2
>    7:	48 c7 c7 c0 eb 89 85 	mov    $0xffffffff8589ebc0,%rdi
>    e:	e8 41 fd ff ff       	callq  0xfffffffffffffd54
>   13:	49                   	rex.WB
>   14:	8d                   	.byte 0x8d
>   15:	bf                   	.byte 0xbf
> [    0.055225][    T0] RSP: 0000:ffffc900001f7cd0 EFLAGS: 00010002
> [    0.055225][    T0] RAX: 0000000000000003 RBX: ffff888817ca9020 RCX: 1ffffffff0b13da3
> [    0.055225][    T0] RDX: 0000000000000001 RSI: 0000000000000085 RDI: ffffc900001f7d68
> [    0.055225][    T0] RBP: ffffc900001f7d08 R08: 0000000000000000 R09: fffffbfff0962288
> [    0.055225][    T0] R10: ffffffff84b11443 R11: 0000000000000001 R12: 0000000000000085
> [    0.055225][    T0] R13: 0000000000000000 R14: 0000000000000000 R15: ffffffff8683dee0
> [    0.055225][    T0] FS:  0000000000000000(0000) GS:ffff888817c80000(0000) knlGS:0000000000000000
> [    0.055225][    T0] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [    0.055225][    T0] CR2: 0000000000000000 CR3: 000000089c85a001 CR4: 00000000003706b0
> [    0.055225][    T0] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [    0.055225][    T0] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [    0.055225][    T0] Call Trace:
> [    0.055225][    T0]  <TASK>
> [ 0.055225][ T0] ? __warn (kernel/panic.c:694) 
> [ 0.055225][ T0] ? do_clear_cpu_cap (arch/x86/kernel/cpu/cpuid-deps.c:118 (discriminator 1)) 
> [ 0.055225][ T0] ? report_bug (lib/bug.c:180 lib/bug.c:219) 
> [ 0.055225][ T0] ? handle_bug (arch/x86/kernel/traps.c:239 (discriminator 1)) 
> [ 0.055225][ T0] ? exc_invalid_op (arch/x86/kernel/traps.c:260 (discriminator 1)) 
> [ 0.055225][ T0] ? asm_exc_invalid_op (arch/x86/include/asm/idtentry.h:621) 
> [ 0.055225][ T0] ? do_clear_cpu_cap (arch/x86/kernel/cpu/cpuid-deps.c:118 (discriminator 1)) 
> [ 0.055225][ T0] ? __pfx_do_clear_cpu_cap (arch/x86/kernel/cpu/cpuid-deps.c:109) 
> [ 0.055225][ T0] init_ia32_feat_ctl (arch/x86/kernel/cpu/feat_ctl.c:181)

Yap, works as designed:

..
                goto update_sgx;

        if ( (tboot && !(msr & FEAT_CTL_VMX_ENABLED_INSIDE_SMX)) ||
            (!tboot && !(msr & FEAT_CTL_VMX_ENABLED_OUTSIDE_SMX))) {
                if (IS_ENABLED(CONFIG_KVM_INTEL))
                        pr_err_once("VMX (%s TXT) disabled by BIOS\n",
                                    tboot ? "inside" : "outside");
                clear_cpu_cap(c, X86_FEATURE_VMX);			<--- here
        } else {

Clearing feature flags after alternatives have been applied means that
code which does

	alternative(, ... X86_FEATURE_VMX, ...)

won't work as expected because the patching has already happened.

And I'm not sure even the dynamic testing *cpu_has() does will always
work as we do this dance in get_cpu_cap() with forced flags.

So, I'm thinking init_ia32_feat_ctl() should run in early_init_intel()
which is before alternatives.

And looking at init_ia32_feat_ctl(), all it does is set and clear
a bunch of bits so I think it should be ok.

Sean?

> [ 0.055225][ T0] init_intel (arch/x86/include/asm/msr.h:146 arch/x86/include/asm/msr.h:300 arch/x86/kernel/cpu/intel.c:583 arch/x86/kernel/cpu/intel.c:687) 
> [ 0.055225][ T0] identify_cpu (arch/x86/kernel/cpu/common.c:1824) 
> [ 0.055225][ T0] identify_secondary_cpu (arch/x86/kernel/cpu/common.c:1949) 
> [ 0.055225][ T0] smp_store_cpu_info (arch/x86/kernel/smpboot.c:333) 
> [ 0.055225][ T0] start_secondary (arch/x86/kernel/smpboot.c:197 arch/x86/kernel/smpboot.c:281) 
> [ 0.055225][ T0] ? __pfx_start_secondary (arch/x86/kernel/smpboot.c:231) 
> [ 0.055225][ T0] common_startup_64 (arch/x86/kernel/head_64.S:421) 
> [    0.055225][    T0]  </TASK>
> [    0.055225][    T0] ---[ end trace 0000000000000000 ]---
> 
> 
> The kernel config and materials to reproduce are available at:
> https://download.01.org/0day-ci/archive/20240430/202404302233.f27f91b2-oliver.sang@intel.com
> 
> 
> 
> -- 
> 0-DAY CI Kernel Test Service
> https://github.com/intel/lkp-tests/wiki
> 

-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette