Received: by 2002:a05:6a10:1a4d:0:0:0:0 with SMTP id nk13csp1795569pxb; Wed, 9 Feb 2022 04:48:57 -0800 (PST) X-Google-Smtp-Source: ABdhPJxXMD/sU8Ui/xh7WCAf8az8jX37lIv0v8z7iG3vuGuglxMey0ZmPdQeVxEVCDsPMm0OizYU X-Received: by 2002:a17:903:124f:: with SMTP id u15mr2268943plh.15.1644410937214; Wed, 09 Feb 2022 04:48:57 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1644410937; cv=none; d=google.com; s=arc-20160816; b=MMvO1pEWZAkTetULajld34oe+9TTLrFa2lhuCmyRYcypRC7+Yi9TOudKQhvRT5wPFM 7Wa01ZLHTdSgBEWgWpJqJvWnte3fJJC94mGENLIfmqZgCg19xs0keUg0fwUu7sdKCAHK RN+HAzmtpH1FJu/UL8zCB/dK1vnyD++igRteUBXvGTgXirDqXQ7PpWxdxG0efPpNmvjk vK7/kCpzE5TFOWE8xybGSHnAyMAthWaY5I2YdjH3MmmZsy9+syd1xwcvgtGbCUQjyg4J Ho2I16mP03awlzw8g0mbWyhEeA+YZ2MB5JzN1WEFtQejWl5X9LBt2G7eFPwcvjQhOT+a jYug== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :message-id:date:subject:cc:to:from:dkim-signature; bh=vteorb5NL26GNg8C0c6veqgpmA+N3f2PfOBmx7wN1wg=; b=Y/fk5aerOm6whZG9bICUD9lKN3ON6/4IwL9J+zNYG879FM5kTM92b7ISgoZk+1uX/b pXZ5Dt8lCAX2Il4XPvd2d9wnM5I5xiDCMBj/21UA+v5MMdzH3DVPbXALbwIzxudFzpOU 45d1cA+ru42WRMeov7Luc1JWpff9KWKVvTwMqzJFcrWKFK71WarGapWBLI2mfPSLAoyJ m3JLfsYbb8YKyx51UgUbqRVu4tapMqJ3xeu9uYFK4A2Y1woYB5VEKcolwBEYBqw+BiNo gnIyKsuK/0N6vVDl6AmF5x5+TC2SzlToAZc+uycrDkO2eqwCHF/KCfIHsaElYR6+7Ehf mJ2Q== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Q1G4XrTJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from lindbergh.monkeyblade.net (lindbergh.monkeyblade.net. [2620:137:e000::1:18]) by mx.google.com with ESMTPS id np14si932273pjb.84.2022.02.09.04.48.56 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Wed, 09 Feb 2022 04:48:57 -0800 (PST) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) client-ip=2620:137:e000::1:18; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Q1G4XrTJ; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by lindbergh.monkeyblade.net (Postfix) with ESMTP id 3B1B2E15892B; Wed, 9 Feb 2022 02:30:00 -0800 (PST) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238875AbiBIHle (ORCPT + 99 others); Wed, 9 Feb 2022 02:41:34 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39606 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232283AbiBIHlb (ORCPT ); Wed, 9 Feb 2022 02:41:31 -0500 Received: from mga04.intel.com (mga04.intel.com [192.55.52.120]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 40D20C0613CB; Tue, 8 Feb 2022 23:41:35 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1644392495; x=1675928495; h=from:to:cc:subject:date:message-id:mime-version: content-transfer-encoding; bh=ptt8R/RvwFaVk8njj6dZCgZ7G353iPS7PsxtHDnt7q4=; b=Q1G4XrTJmgjUvRYOmGMMd5++rTE9xHuMR0IggqiL2/R0i9GlqR8pmTnN ZOFZF8MC4mlJ92HZUd31PBouep2cmZjNYNCG8gox2HRuywErqHsy90aVh o2NfMU2ugg3l5PGBzLxoGf3rNabjHAJUaSKJN7x/o2Mug5H12lTmNz0vn 3GQpk7B07A1INrBJm9SlmWSJkJkgNZkNWfSP7JCSeiYC6RY2BpkD4KGSY UMHrshfBrkNgy765kxfqRirgH5BhYFE6DGOnxJtbtf08VmdXWy32MCggz QuAjuZH0GDQM3IBfub0lIjuAm46lW/oemsJylS2o1OQaMbxsOFiwqYv5v A==; X-IronPort-AV: E=McAfee;i="6200,9189,10252"; a="247980712" X-IronPort-AV: E=Sophos;i="5.88,355,1635231600"; d="scan'208";a="247980712" Received: from orsmga008.jf.intel.com ([10.7.209.65]) by fmsmga104.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2022 23:41:34 -0800 X-IronPort-AV: E=Sophos;i="5.88,355,1635231600"; d="scan'208";a="540984524" Received: from hyperv-sh4.sh.intel.com ([10.239.48.22]) by orsmga008-auth.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 08 Feb 2022 23:41:21 -0800 From: Chao Gao To: kvm@vger.kernel.org, seanjc@google.com, pbonzini@redhat.com, kevin.tian@intel.com, tglx@linutronix.de Cc: Chao Gao , Albert Ou , Aleksandar Markovic , Alexander Gordeev , Alexandru Elisei , Anup Patel , Atish Patra , Benjamin Herrenschmidt , Bharata B Rao , Borislav Petkov , Catalin Marinas , Christian Borntraeger , Claudio Imbrenda , Daniel Lezcano , "Darrick J. Wong" , Dave Hansen , David Hildenbrand , Fabiano Rosas , Heiko Carstens , "H. Peter Anvin" , Huacai Chen , Ingo Molnar , James Morse , Janosch Frank , Jim Mattson , Joerg Roedel , John Garry , kvmarm@lists.cs.columbia.edu, kvm-riscv@lists.infradead.org, linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, linux-mips@vger.kernel.org, linuxppc-dev@lists.ozlabs.org, linux-riscv@lists.infradead.org, linux-s390@vger.kernel.org, "Maciej S. Szmigiero" , Marc Zyngier , Mel Gorman , Michael Ellerman , Nicholas Piggin , Nick Desaulniers , Palmer Dabbelt , Paul Mackerras , Paul Walmsley , Ravi Bangoria , Shaokun Zhang , Sumanth Korikkar , Suzuki K Poulose , Thomas Bogendoerfer , Thomas Richter , Tom Zanussi , Vasily Gorbik , Vitaly Kuznetsov , Wanpeng Li , Will Deacon , x86@kernel.org Subject: [PATCH v3 0/5] Improve KVM's interaction with CPU hotplug Date: Wed, 9 Feb 2022 15:41:01 +0800 Message-Id: <20220209074109.453116-1-chao.gao@intel.com> X-Mailer: git-send-email 2.25.1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE autolearn=no autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Changes from v2->v3: - rebased to the latest kvm/next branch. - patch 1: rename {svm,vmx}_check_processor_compat to follow the name convention - patch 3: newly added to provide more information when hardware enabling fails - patch 4: reset hardware_enable_failed if hardware enabling fails. And remove redundent kernel log. - patch 5: add a pr_err() for setup_vmcs_config() path. Changes from v1->v2: (all comments/suggestions on v1 are from Sean, thanks) - Merged v1's patch 2 into patch 1, and v1's patch 5 into patch 6. - Use static_call for check_processor_compatibility(). - Generate patch 2 with "git revert" and do manual changes based on that. - Loosen the WARN_ON() in kvm_arch_check_processor_compat() instead of removing it. - KVM always prevent incompatible CPUs from being brought up regardless of running VMs. - Use pr_warn instead of pr_info to emit logs when KVM finds offending CPUs. KVM registers its CPU hotplug callback to CPU starting section. And in the callback, KVM enables hardware virtualization on hotplugged CPUs if any VM is running on existing CPUs. There are two problems in the process: 1. KVM doesn't do compatibility checks before enabling hardware virtualization on hotplugged CPUs. This may cause #GP if VMX isn't supported or vmentry failure if some in-use VMX features are missing on hotplugged CPUs. Both break running VMs. 2. Callbacks in CPU STARTING section cannot fail. So, even if KVM finds some incompatible CPUs, its callback cannot block CPU hotplug. This series improves KVM's interaction with CPU hotplug to avoid incompatible CPUs breaking running VMs. Following changes are made: 1. move KVM's CPU hotplug callback to ONLINE section (suggested by Thomas) 2. do compatibility checks on hotplugged CPUs. 3. abort onlining incompatible CPUs This series is a follow-up to the discussion about KVM and CPU hotplug https://lore.kernel.org/lkml/3d3296f0-9245-40f9-1b5a-efffdb082de9@redhat.com/T/ Note: this series is tested only on Intel systems. Chao Gao (4): KVM: x86: Move check_processor_compatibility from init ops to runtime ops Partially revert "KVM: Pass kvm_init()'s opaque param to additional arch funcs" KVM: Rename and move CPUHP_AP_KVM_STARTING to ONLINE section KVM: Do compatibility checks on hotplugged CPUs Sean Christopherson (1): KVM: Provide more information in kernel log if hardware enabling fails arch/arm64/kvm/arm.c | 2 +- arch/mips/kvm/mips.c | 2 +- arch/powerpc/kvm/powerpc.c | 2 +- arch/riscv/kvm/main.c | 2 +- arch/s390/kvm/kvm-s390.c | 2 +- arch/x86/include/asm/kvm-x86-ops.h | 1 + arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/kvm/svm/svm.c | 4 +- arch/x86/kvm/vmx/evmcs.c | 2 +- arch/x86/kvm/vmx/evmcs.h | 2 +- arch/x86/kvm/vmx/vmx.c | 22 +++++---- arch/x86/kvm/x86.c | 16 +++++-- include/linux/cpuhotplug.h | 2 +- include/linux/kvm_host.h | 2 +- virt/kvm/kvm_main.c | 73 +++++++++++++++++++----------- 15 files changed, 83 insertions(+), 53 deletions(-) -- 2.25.1