Message-ID: <55A56709.6020201@linux.intel.com>
Date: Tue, 14 Jul 2015 12:46:17 -0700
From: Dave Hansen <dave.hansen@linux.intel.com>
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:31.0) Gecko/20100101 Thunderbird/31.7.0
MIME-Version: 1.0
To: Ingo Molnar <mingo@kernel.org>, linux-kernel@vger.kernel.org
CC: Andy Lutomirski <luto@amacapital.net>, Borislav Petkov <bp@alien8.de>,
        Fenghua Yu <fenghua.yu@intel.com>, "H. Peter Anvin" <hpa@zytor.com>,
        Linus Torvalds <torvalds@linux-foundation.org>,
        Oleg Nesterov <oleg@redhat.com>, Thomas Gleixner <tglx@linutronix.de>,
        Ross Zwisler <ross.zwisler@linux.intel.com>
Subject: 4.2-rc2: early boot memory corruption from FPU rework
References: <1430848300-27877-1-git-send-email-mingo@kernel.org> <1430848300-27877-19-git-send-email-mingo@kernel.org>
In-Reply-To: <1430848300-27877-19-git-send-email-mingo@kernel.org>
Content-Type: text/plain; charset=windows-1252
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Content-Length: 2187
Lines: 53

On 05/05/2015 10:49 AM, Ingo Molnar wrote:
> @@ -574,12 +573,10 @@ static void setup_init_fpu_buf(void)
>  	on_boot_cpu = 0;
>  
>  	/*
> -	 * Setup init_xstate_buf to represent the init state of
> +	 * Setup init_xstate_ctx to represent the init state of
>  	 * all the features managed by the xsave
>  	 */
> -	init_xstate_buf = alloc_bootmem_align(xstate_size,
> -					      __alignof__(struct xsave_struct));
> -	fx_finit(&init_xstate_buf->i387);
> +	fx_finit(&init_xstate_ctx.i387);

This is causing memory corruption in 4.2-rc2.

We do not know the size of the 'init_xstate_buf' before we boot.  It's
completely enumerated in CPUID leaves but it is not static by any means.
 This commit when applied (3e5e126774) tries to replace the dynamic
allocation with a static one.  When we do the first 'xrstor' (in
copy_xregs_to_kernel_booting()) it overruns init_fpstate and corrupts
the next chunk of memory (which is xfeatures_mask in my case).

I'm seeing this on a system with states not represented in
XSTATE_RESERVE (XSTATE_ZMM_Hi256 / XSTATE_OPMASK / XSTATE_Hi16_ZMM).
The systems affected are not widely available, but this is something
that we absolutely do not want to see regress.

This bug could also occur if a future CPU decided to change the amount
of storage allocated for a given xstate feature (which would be
architecturally OK).

According to the commit:

>     This removes the last bootmem allocation from the FPU init path, allowing
>     it to be called earlier in the boot sequence.

so we can't easily just revert this, although I'm not 100% that this is
before bootmem is availalble.

This patch works around the problem, btw:

	https://www.sr71.net/~dave/intel/bloat-xsave-gunk-2.patch

One curiosity here is that the bisect for this actually turned up the
patch that disables 'XSAVES' support.  When we used 'XSAVES' and the
"compacted" format, we managed to fit in to the buffer and things worked
(accidentally).
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/