Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Sun, 14 Aug 2022 02:54:29 +0000
From:   Ashok Raj <ashok.raj@intel.com>
To:     Andy Lutomirski <luto@kernel.org>
CC:     Borislav Petkov <bp@alien8.de>,
        Thomas Gleixner <tglx@linutronix.de>,
        "Tony Luck" <tony.luck@intel.com>,
        Dave Hansen <dave.hansen@intel.com>,
        "Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
        the arch/x86 maintainers <x86@kernel.org>,
        "luto@amacapital.net" <luto@amacapital.net>,
        Tom Lendacky <thomas.lendacky@amd.com>,
        Andrew Cooper <andrew.cooper3@citrix.com>,
        "Ashok Raj" <ashok.raj@intel.com>
Subject: Re: [PATCH 5/5] x86/microcode: Handle NMI's during microcode update.
Message-ID: <Yvhj5YaLdJnF60uR@araj-dh-work>
References: <20220813223825.3164861-1-ashok.raj@intel.com>
 <20220813223825.3164861-6-ashok.raj@intel.com>
 <0d6afdb6-e546-4ef0-80c2-84827c11f2ce@www.fastmail.com>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <0d6afdb6-e546-4ef0-80c2-84827c11f2ce@www.fastmail.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?LWsgmgQZLMiu8zxj84YIGmtaqr7sSaeHCKQIgcLcWnR4+NDT5RAH+1XXX83m?=
 =?us-ascii?Q?KFq2Az42AvTMy4+4nQGt/bRpXV6NrOoOxKaRi1Rs68Lg21Zb9vlOaLgB5NGU?=
 =?us-ascii?Q?KZRc65FmhcO9B5NIultU3r2OoVOf7deA+xo12nvHRG/ZLdvv44+/XjOmzzFp?=
 =?us-ascii?Q?umHvO9JCNFTX0g4MaGrTkUXhcVitacICe4RiOBQvla3eODJ090Yi43fl2bnY?=
 =?us-ascii?Q?b4jgdHUrQ0UEIVFWKEN4LVASafYGViXYlL23eXZR92WgY1xLfmvG028RNCy+?=
 =?us-ascii?Q?quKedTJt85kD3GJIYezCMU0mfhp5G1DiIYAHv5TxWIV9qLJY/NHyOEwMWMtq?=
 =?us-ascii?Q?BdmPOnoQp0VZS23TZ6B5CoTwZM0+nxxfcE4WifF/doAaJc5f4pdo/UFwQ62e?=
 =?us-ascii?Q?nsLJgjpO06seUEHci1qpyxUQmyBo9dOC6i7GsouD2cAvkMFd3zVgGVIttq6V?=
 =?us-ascii?Q?/DhDvymK/gxqnZauREPur6O8EEVSKOdmgFw84A6++/KfF2T6XWFOePwoVQVG?=
 =?us-ascii?Q?bpHq2IL+eoYrdxy2RRZVMidS1PUsMv9FDzMYAa69i47Jz6yQLLXkJP+oBElx?=
 =?us-ascii?Q?7xlg96+ifygNAd0RjNHhN/vsT74n1rpYn2Kw+5zFP9k84WGcRZTGRIPoTuS0?=
 =?us-ascii?Q?/MzPBWO5+0WnRxqTZEsPoBOiwbypLpDKCNJCI4d3+MiBQORcNgQRgENMo+tS?=
 =?us-ascii?Q?fLrQTPcWfPhgyvxqevPQkd74dm7pCHtICqacX48TgOpqNLivZAKD2+O8RmZp?=
 =?us-ascii?Q?3IVO/GNOjic8/i7eM/j1GAACqjrAsyycYu8dDHnko5LWlv4mtYMupsEqHBA0?=
 =?us-ascii?Q?M4BR0tnlkdQje604ppQn1hujcmDbFrU2+iFRiGwkQ866jqnGVGMJySULIOCL?=
 =?us-ascii?Q?CxD3iPyECFCfsXCudZjMuh9+bsi91AKiq4innaFgTAv3fJG5huRVz1CQRCAZ?=
 =?us-ascii?Q?NwjYS8qs3rtpsktkEh036EV9hAYVchtcloJnxO6pL9cQeDuLSVy0jlqsFRME?=
 =?us-ascii?Q?Rt2bUv0JyRhTBRIW0WwM2XTzA/1srLUf2cdMKhbpSvDEJRxvxoOfl1Q/VQMj?=
 =?us-ascii?Q?E7D86u6si3zydmWJLiLTReo/7RyV9DLL2pNeAZUQI13d8pvdZrz4qCNAfMVY?=
 =?us-ascii?Q?r09D+UKfajKp904ejNW0wpYKdw0iTONIFC4DT8BJGNm5Tfy299dTis/a0SDH?=
 =?us-ascii?Q?WujiVlyKX4/m9l4Kb2MMU+pVV/8UVJjEG9cBJ4DNyatYqAHpQU3WPZ3KX3zT?=
 =?us-ascii?Q?S+UAtcp7UbBRzEzloL5hCT4Sycx6x1Fp/Uf92pI+0wYpzlEoC/W6/Id65rA/?=
 =?us-ascii?Q?aP1SyX2/ddG8MqKghfC/GLSBOoaEErdUa0bdhQXCk2XMfCKjbSA+cbqtn9gD?=
 =?us-ascii?Q?byqfusQWmzVWrEer4fF+OUHSlgSl/rx9pwL7xaygMzi4f99wrexmjeG/QtAw?=
 =?us-ascii?Q?m2d6HEgungfjfMiZ34eI9q+owqspLst2rALUOzca2G8BIM0b/XUAdDuEw/2x?=
 =?us-ascii?Q?zcasFkLJ61j94+91UMxrF/YB3Qhkr0u6dloAMPqXe+3+f1bECcl9T6gjo6Ky?=
 =?us-ascii?Q?DrI2G9Hl0/WxjULXf3Jwz6zYZft5lfJWpjxhiQ4FrLv8GxRaPP1xTJGFmEqx?=
 =?us-ascii?Q?Gw=3D=3D?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 528a3c84-1746-442c-7b2c-08da7da058e6
X-MS-Exchange-CrossTenant-AuthSource: SJ1PR11MB6201.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Aug 2022 02:54:46.7179
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 8l72py8CwS/J9qNmrX0jHMCi3iSVN3xDHLuqBbnmbkwS7PJS5rlnPWmGVTBzz2eTguKXd0k4Yic9pzR4/OHguQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: MW4PR11MB6713
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-7.7 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_HI,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Sat, Aug 13, 2022 at 05:13:13PM -0700, Andy Lutomirski wrote:
> 
> 
> On Sat, Aug 13, 2022, at 3:38 PM, Ashok Raj wrote:
> > Microcode updates need a guarantee that the thread sibling that is waiting
> > for the update to finish on the primary core will not execute any
> > instructions until the update is complete. This is required to guarantee
> > any MSR or instruction that's being patched will be executed before the
> > update is complete.
> >
> > After the stop_machine() rendezvous, an NMI handler is registered. If an
> > NMI were to happen while the microcode update is not complete, the
> > secondary thread will spin until the ucode update state is cleared.
> >
> > Couple of choices discussed are:
> >
> > 1. Rendezvous inside the NMI handler, and also perform the update from
> >    within the handler. This seemed too risky and might cause instability
> >    with the races that we would need to solve. This would be a difficult
> >    choice.
> 
> I prefer choice 1.  As I understand it, Xen has done this for a while to good effect.
> 
> If I were implementing this, I would rendezvous via stop_machine as usual.  Then I would set a flag or install a handler indicating that we are doing a microcode update, send NMI-to-self, and rendezvous in the NMI handler and do the update.

Well, that is exactly what I did for the first attempt. The code looked so
beautiful in the eyes of the creator :-) but somehow I couldn't get it to
not lock up. But the new implementation seemed to be more efficient. We do
nothing until the secondary drops in the NMI handler, and then hold them
hostage right there.

I thought this was slightly improved, in the sense we don't take extra hit
for sending and receiving interrupts. 

In my first attempt I didn't only rendezvous between threads of the core.
Everything looked great but just didn't work for me in time.

Cheers,
Ashok