Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Sat, 3 Jun 2023 13:54:19 -0700
From:   Ashok Raj <ashok.raj@intel.com>
To:     Thomas Gleixner <tglx@linutronix.de>
CC:     LKML <linux-kernel@vger.kernel.org>, <x86@kernel.org>,
        Ashok Raj <ashok.raj@linux.intel.com>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        "Tony Luck" <tony.luck@intel.com>,
        Arjan van de Veen <arjan@linux.intel.com>,
        Peter Zijlstra <peterz@infradead.org>,
        Eric Biederman <ebiederm@xmission.com>,
        Ashok Raj <ashok.raj@intel.com>
Subject: Re: [patch 4/6] x86/smp: Cure kexec() vs. mwait_play_dead() breakage
Message-ID: <ZHuoewLHjeyjW9vL@a4bf019067fa.jf.intel.com>
References: <20230603193439.502645149@linutronix.de>
 <20230603200459.832650526@linutronix.de>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20230603200459.832650526@linutronix.de>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?Ezg4SwG8VbU19fcjcuUQvSKExb4HWPQCcY7bb/9BvDM7z4YrtgoRaJCpyrd8?=
 =?us-ascii?Q?JJi/FnplNGbZmMzKdvIgr3mZPQj2XOjRg8q545ikhC0v2rTaDcXfePeDJtgZ?=
 =?us-ascii?Q?JI14t1XnAKeKGEZnGHIpaKSpQbAYJBQaEdzZpINwegvve7M8tmo1y4uMbvV4?=
 =?us-ascii?Q?PnZ51vYHk8X41rTBzmqVkRM+ATj9/n/07RgE+qmk6Sz2lj9DltlAH6pbQkHJ?=
 =?us-ascii?Q?SRI1145Q27c8k5/p0HG4nPgMIm2kGxYlXRYlRSnRv2pQgbyeirpg9PnAWnL0?=
 =?us-ascii?Q?UcfcU7iCW/JyvnO08b1lmc0SBTpYpKVmU2DaGYPE0BLKZueLUBso9FMUrUVu?=
 =?us-ascii?Q?/wkFqHDG9meM2uXZms/izmpUnfombf3k5PwZW6byHGUDuYsukZQnyHW0dPec?=
 =?us-ascii?Q?ga6/pBFeAE/foL+akeUUzDG6WygBMv+HtKGbSzUQcEsmMBZ/qglmp6c8YJBk?=
 =?us-ascii?Q?ldfhEiEufb+i8PV6nqtSdHpNeDolghFYlIRnhwCBrJK07HRgcqaW7+dwa3TF?=
 =?us-ascii?Q?cVsjdNdDYbeg4YMaASHnDruuQiAtdFvAXLQu7q5DSh89kJ9gC3CyHzQoZT/e?=
 =?us-ascii?Q?OkwC0r4h/rz3HPV5sslg7+goNorkXzTHrpeV1aSqJOWQoEVsiWVb8wLr4uvM?=
 =?us-ascii?Q?I8Jvu+DEkDK3uW5ZsUBEapgOqLKZkgCkkjUnwgRUXYvhtzP+I/sfVeErjsLj?=
 =?us-ascii?Q?UVAgx84e9zv3LW+Q7CQry6sXbM5ur2TYBXMSM7AVbqBp1UFYg2uIWEiSWwPm?=
 =?us-ascii?Q?fps/g9/CpyuMx4okLvyTeTS9ypXIdUR+bz+uk8yAg1oLJFBeR374+BUDCPK0?=
 =?us-ascii?Q?fbE3y3uJ4OfEFNy4+VrmymosiSe2HiatrHmxLdYXj6EHOamINYZ6j2cp4eCc?=
 =?us-ascii?Q?l32uYXH0h+OlixwwLyfE/IvJbZR56K0yrvohe0IPOSzrNH1yv2S4HplLOFnW?=
 =?us-ascii?Q?ZohbjcOD+Bv+a7T8Ii1k/QoDbGyeSWl7yOQJbRf5OmGmjPN/j7bDm33iP1HB?=
 =?us-ascii?Q?5+hPu2kjm2HmIycX5mJokKdsP5CEcdWKiwP/FjZGpnZTmZo5/vuIUFdPr9hW?=
 =?us-ascii?Q?p5gIGVc5PQm68iEldepb75H51dJdYF+O78MfPQBfwSNByKXF27SAvmWwArsL?=
 =?us-ascii?Q?7xpJ5Mg4XPCCaGEIB+w5iwxJhzsZffYFsLukqneYK5en2HYf/3Ns3jD7RZGX?=
 =?us-ascii?Q?Ng1k2oY44HJzHcqCpNcuasFcZW1E0PHFuZn+tgpst+dNAoZGDXYmyc6F91GE?=
 =?us-ascii?Q?Y0YbohnxBQTu4sfg3w26NztPnnNbUl6XsQs1VYQVO71OExNzmhkqyFVpfhZF?=
 =?us-ascii?Q?m7lGmPU3Zj+MjpDDdVQ0R47QDYUMaL4Zm7WFrxbI9iH71lzXgBhO0jg+PK7c?=
 =?us-ascii?Q?GXbho2XmuqDVmutn10TuRbMxyrxoS72Wg3EUSR/ABvFUGnBKtUUJyezSpBmD?=
 =?us-ascii?Q?3FfL1j8PkTvkEgaBD2Rqu4wxar895cn2qPYKAa64/MytIVNVzbBW8SmEUChC?=
 =?us-ascii?Q?PB/2n8H+czqwr1+QUoyGac9HJPaDxzA90bimRCnemeZsYiScveZI1UhytITV?=
 =?us-ascii?Q?XFr7G9Z2bm4CS/37fNV+4RnluD86AxIhWh7vyEfT?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 51c34bb1-9e35-4766-23cb-08db6474b57e
X-MS-Exchange-CrossTenant-AuthSource: SJ1PR11MB6201.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 03 Jun 2023 20:54:22.8785
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 3/+Leo/Kbo71nDXZRq1G+ekCGesOjBoVhg0VkSRvJN+sl86yanM+GvBPH/YQnCT6+h8+oqF1fl10iQmNt+u80Q==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ2PR11MB8404
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE,
        T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no
        version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Sat, Jun 03, 2023 at 10:07:01PM +0200, Thomas Gleixner wrote:
> TLDR: It's a mess.
> 
> When kexec() is executed on a system with "offline" CPUs, which are parked
> in mwait_play_dead() it can end up in a triple fault during the bootup of
> the kexec kernel or cause hard to diagnose data corruption.
> 
> The reason is that kexec() eventually overwrites the previous kernels text,
> page tables, data and stack, If it writes to the cache line which is
> monitored by an previously offlined CPU, MWAIT resumes execution and ends
> up executing the wrong text, dereferencing overwritten page tables or
> corrupting the kexec kernels data.
> 
> Cure this by bringing the offline CPUs out of MWAIT into HLT.
> 
> Write to the monitored cache line of each offline CPU, which makes MWAIT
> resume execution. The written control word tells the offline CPUs to issue
> HLT, which does not have the MWAIT problem.
> 
> That does not help, if a stray NMI, MCE or SMI hits the offline CPUs as
> those make it come out of HLT.
> 
> A follow up change will put them into INIT, which protects at least against
> NMI and SMI.
> 
> Fixes: ea53069231f9 ("x86, hotplug: Use mwait to offline a processor, fix the legacy case")
> Reported-by: Ashok Raj <ashok.raj@intel.com>
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>

Does this need a stable tag?

Reviewed-by: Ashok Raj <ashok.raj@intel.com>
Tested-by: Ashok Raj <ashok.raj@intel.com>