Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Thu, 10 Aug 2023 14:49:39 -0700
From:   Ashok Raj <ashok.raj@intel.com>
To:     Peter Zijlstra <peterz@infradead.org>
CC:     Thomas Gleixner <tglx@linutronix.de>,
        LKML <linux-kernel@vger.kernel.org>, <x86@kernel.org>,
        Borislav Petkov <bp@alien8.de>,
        Arjan van de Ven <arjan@linux.intel.com>,
        Ashok Raj <ashok.raj@intel.com>
Subject: Re: [patch 28/30] x86/microcode: Handle "offline" CPUs correctly
Message-ID: <ZNVbc7qy8k49Dwhi@araj-mobl.amr.corp.intel.com>
References: <20230810153317.850017756@linutronix.de>
 <20230810160806.562016788@linutronix.de>
 <20230810204605.GF212435@hirez.programming.kicks-ass.net>
 <ZNVNibrpZ9bJLok7@araj-mobl.amr.corp.intel.com>
 <20230810210511.GH212435@hirez.programming.kicks-ass.net>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <20230810210511.GH212435@hirez.programming.kicks-ass.net>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?sNP1SvZUX3GGSjc3xf3Bxakktr0qiq6gZfLwiNj3P0B7jyRPxoGW2kf4iNfY?=
 =?us-ascii?Q?n3YX5MnMpIOmG9ZIxj1dvPtnAihVN2yCS12x/bYQ3e5uvRRGyoEclPUfmEVy?=
 =?us-ascii?Q?T/sCSlNTOpVb28rNRrhrJeECVCmNkDV6HKiutRXguJ3KYVKlIXlBR1X6FHfX?=
 =?us-ascii?Q?63JukJKRRhnmNKpp9RBb8DY6VGGfI0JOdPcC1QMplgogznj1gUZ9hLt0yBKL?=
 =?us-ascii?Q?O+Ha1MRr0XpTONa2Y1hsFXsSlSG7olFzVf0KeQFlaVGvljWHMcYe36NV3pIy?=
 =?us-ascii?Q?C/tygAIt/PPsZ3VObvVB726XHAiVNqTFtKExJxsIi5JfC5A6huDU29Zfzxxu?=
 =?us-ascii?Q?rU4tcztF8zsGOVC9O+h4Fla1kRXDEY8JtJZ85xTB/LAC6+Spkyy17Z/DmVTW?=
 =?us-ascii?Q?/nTojQKQd3LjumwS/u878KKNAGn1o540X4dIb2T6pedI1iMUVkJEHZcL/Rgx?=
 =?us-ascii?Q?dSdMQsqxElqZ77i1IH45x/i23rwZgSPvzPSxZLY1ANlv90IC1k/BFnvfy9XX?=
 =?us-ascii?Q?SSTCv+07x2i+4bYS/ZXPAa8wmko0w6gbfp2YIhABMH2Villvk5nILKlQbAcx?=
 =?us-ascii?Q?V10NYKlmFOEKue0V9tSiJo6vMpvMcnzWUpiyoU+odWhd3UkdWlohkzxLnq2H?=
 =?us-ascii?Q?/za+w/k1mQKMQI9/oy/Gb8iKz5qlmIv5tJT2Ae7UYlnXzWXH+3xWmEd/Bvxn?=
 =?us-ascii?Q?bEWCxAfIp/cVrKLlsCExvVSHZJTFROnxW5Qglr3Y1XH7dU2PodhvuxkkjeyO?=
 =?us-ascii?Q?0mFAn8Zvco+npyIfCtYzstZJZvGZ0Crh7VIzYe8wyMn2fEAXjTDDfLf1A8pE?=
 =?us-ascii?Q?pD1TRFzSndV/lBAjwRxDyk+GbfC9J961sBf8BAli6H+Ebwe0pIpBIcMZdVOd?=
 =?us-ascii?Q?2YcMhBb8kdfbtwXNtdQGHbxR6vgCEUlGWDrmsmL4rekhHMp4L2v0PMlttN5l?=
 =?us-ascii?Q?8kbuIW+bPpzhUqMhoFZuzmlyu87KIt3lyhXM+9+SQ7BG1BDBnBVnT7qNdYvU?=
 =?us-ascii?Q?pycVAf3+794IBbnrxxlHixIBxP+6ahFtwQ9IlgzQJC6JFz52u0anZwUVJGYC?=
 =?us-ascii?Q?Gw/QOjR1R6T3m1M0X5wDqriN4+6gz397Zo9nDtvUrrskBuhXdmJPJt1um+X9?=
 =?us-ascii?Q?HdiySeHE+bh5H6K0o1kzydREuuL9/wAQ5JX1aJhIIgy3jSLHHkYZAh7uNBfZ?=
 =?us-ascii?Q?p6NjvwT1uuaHqQhT+HnxRD2uNAYTs7FqgdEzcKTnELwTgfiC1kgR2b90JZcJ?=
 =?us-ascii?Q?oTtDuqttBSEW11pfVN99Z1Qb+RTZvCSG3TRhg5vE6GuU7xqp8mCRwUcdWC/U?=
 =?us-ascii?Q?+3RwoBahmmIqKFfFqQYEdMyb3+/W60PjGHrq+/9WdlT0vRy+ZIJa0pnrjXm1?=
 =?us-ascii?Q?mej7Pe65saG3pGp5f4Uqq8fJL+pqri/KFVierqqLOrJiOTVrsd+Al6P1PoHQ?=
 =?us-ascii?Q?5wdMZNQ0z/Wpb+xQ4h6Cj8P9G6pgPQuxqmm/X9w1U41vwYpZ52iluS1CwfVo?=
 =?us-ascii?Q?AX205DQs4nM8SRKz4Mz0ObLua4/XzqchlVXDbiSTWUYq330lj4O/xEAoSuko?=
 =?us-ascii?Q?W1zP8M9w6rqHiMKW8d55THXfkz6ZrZ39dnNpD1YG?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 530f146d-7c01-40a0-9132-08db99ebb44c
X-MS-Exchange-CrossTenant-AuthSource: SJ1PR11MB6201.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 10 Aug 2023 21:49:42.5549
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: zCSGq98xWJIwLN5+iZNJihDxobNV+EMMQHYYWwciZNivAC2C9fEb/y4TLeeN9/+fPJR8bjejrQyydoccK9k4+A==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SN7PR11MB6677
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,
        RCVD_IN_DNSWL_BLOCKED,SPF_HELO_NONE,SPF_NONE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Thu, Aug 10, 2023 at 11:05:11PM +0200, Peter Zijlstra wrote:
> On Thu, Aug 10, 2023 at 01:50:17PM -0700, Ashok Raj wrote:
> > On Thu, Aug 10, 2023 at 10:46:05PM +0200, Peter Zijlstra wrote:
> > > On Thu, Aug 10, 2023 at 08:38:07PM +0200, Thomas Gleixner wrote:
> > > 
> > > >  	for_each_cpu_and(cpu, cpu_present_mask, &cpus_booted_once_mask) {
> > > > +		/*
> > > > +		 * Offline CPUs sit in one of the play_dead() functions
> > > > +		 * with interrupts disabled, but they still react on NMIs
> > > > +		 * and execute arbitrary code. Also MWAIT being updated
> > > > +		 * while the offline CPU sits there is not necessarily safe
> > > > +		 * on all CPU variants.
> > > > +		 *
> > > > +		 * Mark them in the offline_cpus mask which will be handled
> > > > +		 * by CPU0 later in the update process.
> > > > +		 *
> > > > +		 * Ensure that the primary thread is online so that it is
> > > > +		 * guaranteed that all cores are updated.
> > > > +		 */
> > > >  		if (!cpu_online(cpu)) {
> > > > +			if (topology_is_primary_thread(cpu) || !allow_smt_offline) {
> > > > +				pr_err("CPU %u not online, loading aborted\n", cpu);
> > > 
> > > We could make the NMI handler do the ucode load, no? Also, you just need
> > > any thread online, don't particularly care about primary thread or not
> > > afaict.
> > 
> > Patch 25 does that load in NMI. You are right, we just need "a" CPU in each
> > core online. 
> 
> Patch 25 does it for online CPUs, offline CPUs are having a separate
> code path:
> 
>   microcode_nmi_handler()
> 
> vs
> 
>   microcode_offline_nmi_handler()

Since the code enforces all primary CPUs to be ONLINE, the secondaries are the
other thread of the same core. So they automatically get the update when
primary does it. 

The secondaries are parked in NMI just to avoid the risk of executing code
that might be patched by primary.

Or maybe you had something else in mind.