From:   "Tian, Kevin" <kevin.tian@intel.com>
To:     Thomas Gleixner <tglx@linutronix.de>,
        Paolo Bonzini <pbonzini@redhat.com>
CC:     "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "kvm@vger.kernel.org" <kvm@vger.kernel.org>,
        "Yamahata, Isaku" <isaku.yamahata@intel.com>,
        "Huang, Kai" <kai.huang@intel.com>,
        "Nakajima, Jun" <jun.nakajima@intel.com>,
        "Hansen, Dave" <dave.hansen@intel.com>,
        "Gao, Chao" <chao.gao@intel.com>
Subject: RE: Q. about KVM and CPU hotplug
Thread-Topic: Q. about KVM and CPU hotplug
Thread-Index: Adflvg/SIgoQKcU9QlaunmfcAiKUJgADptsAACzuVmAAB4iHgACJE/aQ
Date:   Sat, 4 Dec 2021 03:57:06 +0000
Message-ID: <BN9PR11MB543302C3841AED15AF52B2A28C6B9@BN9PR11MB5433.namprd11.prod.outlook.com>
References: <BL1PR11MB54295ADE4D7A81523EA50B2D8C679@BL1PR11MB5429.namprd11.prod.outlook.com>
 <3d3296f0-9245-40f9-1b5a-efffdb082de9@redhat.com>
 <BN9PR11MB54333C976289C4AA42D7B1AA8C689@BN9PR11MB5433.namprd11.prod.outlook.com>
 <87bl20aa72.ffs@tglx>
In-Reply-To: <87bl20aa72.ffs@tglx>
Accept-Language: en-US
Content-Language: en-US
x-ms-exchange-antispam-messagedata-chunkcount: 1
x-ms-exchange-antispam-messagedata-0: =?us-ascii?Q?hJ0+QN837cuh22TyQJUIpnzmzGIy/97jyWCYXKlPklgt+q4FTJW3umG1olyk?=
 =?us-ascii?Q?En+sYhd3i3KnZwzy0B7rxVK8vNtiNyZq+7723X1X4ngghYYbwqBc671Hvj46?=
 =?us-ascii?Q?POq708qRJYS+pXca8IAfnfbe/Jj0Qwz5lUt6DHNcH2o8SS3uM53HdJME51j9?=
 =?us-ascii?Q?qvYV6B1965Pehi0avFM3HhegZxjwEa+2pNKLNn3mUSQ1lCAC8vExQUpZ4KpX?=
 =?us-ascii?Q?44bgjPsRfqBlMFROGKjSG6jzkl/OUiAmg6DQeAuheCSoS2bR31ZwvCPfZGG0?=
 =?us-ascii?Q?TyVrpAibIp9mVTchvr1i6tByAorBzM6spk4Gj50ce4re5JdQnUS93lq8aAnq?=
 =?us-ascii?Q?+ASG1+KvCOGBePKDK9I2sbKxTPGnOe7nG5yKN6Op7nIcG1DN+vxe8mfALt5I?=
 =?us-ascii?Q?OFLuZxYB3kYSug1LRDErHaMNwzDg8wwD4P5/RHqINNYRPnljWlnNOBo+Anws?=
 =?us-ascii?Q?2c/R8h1QUXGPB+ixnNx1o4ylrF1yz/9fxwb1DYm6TCSqRSNUbTqtqD+vgqRI?=
 =?us-ascii?Q?3zJLPXHNIo6+adbR75DKgCarCzahCIKXm7CMihpJnJamaIK6/E9nPZTdQcOt?=
 =?us-ascii?Q?J6oFWuzNJ9aBrs5lmtVjuUEmWz0qUX8jhg4F4BNwHgc90sPnQGtp3Ij2hvUx?=
 =?us-ascii?Q?aI5PDguyF51HPv4U9GUkOkYr0YJplZB3b/b8JXTLdTCNRGdh5vWyrdfLn+wu?=
 =?us-ascii?Q?QNU79nQgQZZKgKdrFuUEnX5qSF03Y0JRpOaybwy7bHpu3QMtmmy/n5qUjuFZ?=
 =?us-ascii?Q?IGNMcleFHgRO60SpQTih8LEqLA0zW94j45sO1X14aEy+fy0o/RmdU0PwI+rL?=
 =?us-ascii?Q?cnnBn7xoqygxG9Hy7f6+YFyFLDM3S4gxP3zkjRl6JRQgM62Wcn/7OCSyH3kD?=
 =?us-ascii?Q?oq4OriazBGHdAfCf7YPyijqqMAYBvYTUp6N9nOo11BT2GERASiv+xDOHNqGG?=
 =?us-ascii?Q?idvzA8ZhskmgpZ+f3sDb8cti/iCmDxpCiUAbTBQREdfJohWdaHm/rFamkvk8?=
 =?us-ascii?Q?stZnKnONOiZ1xha2VeVLyzQM//oa+I0vTlVMdJQ/VMHDw72YZ4kcWkPw85ju?=
 =?us-ascii?Q?enmjCD8XvaAUtuM2JXsXhVIxj/i3u0BJneWUKMvjWpPlFdV3xDb9ANTxcrna?=
 =?us-ascii?Q?x0TYGunn2dZxxk6VqGaxfsg/bxi/mgztUIQWlFpZuVkF+2+tJns9XMiuvaqN?=
 =?us-ascii?Q?ZtRioJlXfTJO4twe9S+znU6gfafbJDrx/rNVfQeBkRm2NVe0MMkKwkqHy8pP?=
 =?us-ascii?Q?AErBEYsNT//5FM74KjBMAKYQFOYilcqFLt95NInFfjLIxGKONdzE0zKH0hz1?=
 =?us-ascii?Q?+xQydmyAAN415XovAy7Njg7HTcSWbAajhvbfKjkXPTpcpbuEAPPXzh8GWHuN?=
 =?us-ascii?Q?g1XDR/pXxymQaCJRUJ+t3lW5BQjzYQKpcLBiGq217nk/7rIWEevJelXXePxv?=
 =?us-ascii?Q?YQK/WnBndQfYm1tHpYhGBNvGuhne+Zn2t/WoP5Nxep7qFK9vHVbzHQ0Rl5QP?=
 =?us-ascii?Q?UH7tOd1GcRFB6taa3sROqH6gB6Tor+WB7upXEZeSmupgP40hxDjWL6SMOJwr?=
 =?us-ascii?Q?1//V3Tzz+mQIIjppbSKvSODLPoob1fPNV1RZJ21cpS05HhWHZnLOdP4OZsQL?=
 =?us-ascii?Q?5g=3D=3D?=
Content-Type: text/plain; charset="us-ascii"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-AuthSource: BN9PR11MB5433.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4bb7f881-74b4-405b-6778-08d9b6da2373
X-MS-Exchange-CrossTenant-originalarrivaltime: 04 Dec 2021 03:57:06.2621
 (UTC)
X-MS-Exchange-CrossTenant-fromentityheader: Hosted
X-MS-Exchange-CrossTenant-id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-mailboxtype: HOSTED
X-MS-Exchange-CrossTenant-userprincipalname: +iozyZyC2PY5QGs/D2s3gQizDV0olukf+8Zh00oOhZDe1g2Ktn/kozWHsNmJ6zSEgmDlMi4ar8JfegIoEzPqyw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: BN6PR11MB1538
X-OriginatorOrg: intel.com
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

> From: Thomas Gleixner <tglx@linutronix.de>
> Sent: Wednesday, December 1, 2021 6:31 PM
>=20
> On Wed, Dec 01 2021 at 06:59, Kevin Tian wrote:
> >> From: Paolo Bonzini <paolo.bonzini@gmail.com>
> >> It should fail the first vmptrld instruction.  It will result in a few
> >> WARN_ONCE and pr_warn_ratelimited (see vmx_insn_failed).  For VMX
> this
> >> should be a pretty bad firmware bug, and it has never been reported.
> >> KVM did find some undocumented errata but not this one!
> >>
> >
> > or it may be caused by incompatible CPU capabilities, which is currentl=
y
> > missing a check in kvm_starting_cpu(). So far the compatibility check i=
s
> > done only once before registering cpu hotplug state machine:
> >
> >         for_each_online_cpu(cpu) {
> >                 smp_call_function_single(cpu, check_processor_compat, &=
c, 1);
> >                 if (r < 0)
> >                         goto out_free_2;
> >         }
> >
> >         r =3D cpuhp_setup_state_nocalls(CPUHP_AP_KVM_STARTING,
> "kvm/cpu:starting",
> >                                       kvm_starting_cpu, kvm_dying_cpu);
>=20
> Duh. This is silly _and_ broken.
>=20
> Using for_each_inline_cpu() without holding cpus_read_lock() is racy
> against concurrent hotplug. But even if the locking is added then
> nothing prevents a CPU from being plugged _after_ the lock is dropped.
>=20
> The right solution is to move the hotplug state into the threaded
> section as I pointed out and do:
>=20
>          r =3D cpuhp_setup_state(CPUHP_AP_KVM_STARTING, "kvm/cpu:starting=
",
>                                kvm_starting_cpu, kvm_dying_cpu);
>=20
> which will do the right thing automatically. Checking for compatibility
> would just be part of the kvm_starting_cpu() callback.
>=20

Yes, this sounds the right thing to do. We'll work on a fix.

And as said in another reply to Paolo, future TDX compatibility check
will also be added to this callback.

Thanks
Kevin