Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Date:   Wed, 14 Jun 2023 13:47:11 -0700
From:   Ashok Raj <ashok.raj@intel.com>
To:     Thomas Gleixner <tglx@linutronix.de>
CC:     LKML <linux-kernel@vger.kernel.org>, <x86@kernel.org>,
        Mario Limonciello <mario.limonciello@amd.com>,
        Tom Lendacky <thomas.lendacky@amd.com>,
        "Tony Battersby" <tonyb@cybernetics.com>,
        Ashok Raj <ashok.raj@linux.intel.com>,
        Tony Luck <tony.luck@intel.com>,
        Arjan van de Veen <arjan@linux.intel.com>,
        Eric Biederman <ebiederm@xmission.com>,
        Ashok Raj <ashok.raj@intel.com>
Subject: Re: [patch V2 1/8] x86/smp: Make stop_other_cpus() more robust
Message-ID: <ZIonT7+sxAx8IWEE@a4bf019067fa.jf.intel.com>
References: <20230613115353.599087484@linutronix.de>
 <20230613121615.639116359@linutronix.de>
 <ZIoYMakfbAU9EOjc@a4bf019067fa.jf.intel.com>
 <87ilbp95xq.ffs@tglx>
Content-Type: text/plain; charset="us-ascii"
Content-Disposition: inline
In-Reply-To: <87ilbp95xq.ffs@tglx>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?W0CIvZDYIAc2JpmYc1qriactgyA+gxcr7zYowJh/f7U+gAoZFO/uBZVmObfY?=
 =?us-ascii?Q?0iLebM0sx740xpW/L7nNUt1mHmGB7M1DqQq8VjAz0mXdPivlKs4yyhHtStjF?=
 =?us-ascii?Q?1p1MInfDq915/iRaC6NAxpP5P3TmWHRIi1jwycsMsbHvUkns6VrhrWk0eWCD?=
 =?us-ascii?Q?Wz9mEdlbzvwToklUC9y6nVusQnqfiA0OegJ0QOgFgJZ9xNx5Qgohvm5zKGsS?=
 =?us-ascii?Q?vXVsf9RD1QOENXwHdH3Uvzc4X0w0wpkNg5cE/pkYWg+VvV5lc4nzPda8bIVe?=
 =?us-ascii?Q?iz/N6Fi/kw98R+svC9p/pzX3sgC3Wq2b4xQBcOR6blumqZy/bzmAqay04a7K?=
 =?us-ascii?Q?uPmgrEYCgkvvKsQgyOn/lPVEJfNDnLgkPzDsknQX7FEUJM9d9asKsgSJYlTG?=
 =?us-ascii?Q?/6cRqNbh7EHHXyErY9hlpGh4F9CRg+uZm41g8e0vnxreqhrDBd0K7shDN6/U?=
 =?us-ascii?Q?lxVZqujtaWB7JhhTKXIi3kqyIIwOiMblpvQBMrkArVY2sV7zk1AN0mm/WvQl?=
 =?us-ascii?Q?r8tF7DeAcMe582J1rTJ/rFRLnv/UV+5CWp0mW92WNUiF1PXV1sk9Wh8H0xuS?=
 =?us-ascii?Q?LzXe6pRcEJYfetPJMqQKQyuX7hdzZDcGYNnC4ixlgXW4HpKaepGKn4q3GflB?=
 =?us-ascii?Q?8Ds2wJK/LsTV2ACmX52RGk7yNyFjeiB+7aj9M8gdIuRa29SF27/CeE0pqg4L?=
 =?us-ascii?Q?xYhoiNrrl8ZRYs66+FLbCI/i5WW/LSmgxop7NxWZoS1hTvravDIRCnR9CqPS?=
 =?us-ascii?Q?nhvW9aKd3iXwLvOHi6zCSbjwyxgZutWcpXRPcE/PZycpm6Cms2mu+BxvgBNh?=
 =?us-ascii?Q?0xHZd5mP0tiIYOok7mzSu3BlF5wJR67maQ9wnwLQl7JD0EnA7yYFFzjIt2a2?=
 =?us-ascii?Q?VC0NEm9I7Sl8ZCCyGpvv8jKJRvVoNWX7t56xiEhVQB0SR5aw/1rCTKRnuQ5G?=
 =?us-ascii?Q?p/39J+K1oDR+Z+vMMCuStwAgatui37NBweSSekwk4RobYi2QwPGrF1e3RCSQ?=
 =?us-ascii?Q?IbVVVaDlqbbo4l30PEMdObImnRVZIcRGi4ACLGK5fdrTHCfctgR8r36urCV6?=
 =?us-ascii?Q?bbWzBgRXFRfb3486Sqvmf93O6+17T2wngEnWeoOhgIbzvPOf764kbY73kFdF?=
 =?us-ascii?Q?vN+SkJucKmcgROmKXP3mUKqg+7ef7Hyw39X2c/AUSsovy01ElJWLNIwQHEeB?=
 =?us-ascii?Q?0LUFbLF2IjguRpEb5Zj+IRu+wNuY1GOba+RGoikvuWVRs2EWM+7E7OTPUzZe?=
 =?us-ascii?Q?mlkl04bDDQz8VY5xU3SoGIThWmUEwTVL8SKRr+/j1cN5G2VLevIyr0C+0z5u?=
 =?us-ascii?Q?n4jE7OIdUPLmOtWp8KqUuFSDPPITrDf8whlcuDrWyxBA7BzPa4r8UlinG0/r?=
 =?us-ascii?Q?jft7wOVXMhZjPfRtIogY4NMHWQTPF+ThSnVSr82PUfk+tJ9H2vCP5cgwDE6j?=
 =?us-ascii?Q?5Y6qi8F5kiAlNSbhlgW4y5JmeIeAF99Ou+TzMAb+Dk3ydTMDwDs8HGGvNRWm?=
 =?us-ascii?Q?FSJrjoOsdXIBVld27byV+NmMXbu6T50r53vwiS3ouzGVZOIVynK68ECtl4hD?=
 =?us-ascii?Q?Gh1PtW046gKEycevJFWiFHdzwQLhXR/qss6/PAEu?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 61480522-0e65-4a21-5b31-08db6d188881
X-MS-Exchange-CrossTenant-AuthSource: SJ1PR11MB6201.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 14 Jun 2023 20:47:14.1700
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: /mCRTl9adZvaR8/KSJfjGTUdO/gy4TARBcGR3euKEuP2VH1OmXkcPu6GpbTX9mYbQcIupq7YoRMmxgPfo+YoTg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CYYPR11MB8357
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-4.4 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,
        SPF_HELO_NONE,SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED
        autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Wed, Jun 14, 2023 at 09:53:21PM +0200, Thomas Gleixner wrote:
> > If we go down the INIT path, life is less complicated.. 
> >
> > After REBOOT_VECTOR IPI, if stop_cpus_count > 0, we send NMI to all CPUs.
> > Won't this completely update the atomic_dec() since CPUs in hlt() will also
> > take the NMI correct? I'm not sure if this is problematic.
> >
> > Or should we reinitialize stop_cpus_count before the NMI hurrah
> 
> Bah. Didn't think about HLT. Let me go back to the drawing board. Good catch!
> 
> >> +	/*
> >> +	 * Ensure that the cache line is invalidated on the other CPUs. See
> >> +	 * comment vs. SME in stop_this_cpu().
> >> +	 */
> >> +	atomic_set(&stop_cpus_count, INT_MAX);
> >
> > Didn't understand why INT_MAX here?
> 
> Any random number will do. The only purpose is to ensure that there is
> no dirty cache line on the other (stopped) CPUs.
> 
> Now let me look into this NMI cruft.
> 

Maybe if each CPU going down can set their mask, we can simply hit NMI to
only the problematic ones?

The simple count doesn't capture the CPUs in trouble.