Received: by 2002:a25:ab43:0:0:0:0:0 with SMTP id u61csp4405649ybi; Mon, 27 May 2019 17:54:41 -0700 (PDT) X-Google-Smtp-Source: APXvYqwnsHnFsKn+je5hBu0rsbf0g+0L0GdZ0iCRZwQc+3/A97iWClv5diZqjmkQHsZ65b4BBnmv X-Received: by 2002:a17:902:7895:: with SMTP id q21mr22355034pll.73.1559004881205; Mon, 27 May 2019 17:54:41 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1559004881; cv=none; d=google.com; s=arc-20160816; b=Arml3yqTm1Irt9FCyiG5KI5fZWp6/SjmluV1S1Fnggs+X2CKdzv6txLHDzEidQLhOB GgjsiZctQBVkvQv26dKrDYfqS32XoaHP3uUp1EB7KB00cf13jrggstO8FAMaPYRY1/bm XK5wPYQvXa/7zHC9zL/n+ybJ3xQgxZjBYNCrS37o/34/0fNWtKkcXPZAZjRUDoEHi9IO MFj4K8iYlBgxGw9/K2ifYFg05SRgN5hNX95HuG98R24iurjEKvoSCNVZ1TTgVVlCLTER LqgAlzSR9MN0uMYZJQd63w5c2M7qeYLELnUarrhTYZNWNsAFRkLz0wn+t6rqMn9ZBfv3 rTag== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=vBM+WK9qA4pZcm6B2byvLBaRkxAGft4URtktgvLLdCc=; b=hp4L4Y1UZWipfdVqNq8R9RALuzxJy6fSR5I9MkK3U2SwcU5veBCTw467r/3Gn7i71d PVNQlwcPI9mwEKrp++AWJ2xs6Cn8hS7/xrm1b+GjDRiSqWN+lr/mknwXwrniXC/VnlgI qXvEjieJT3X6NZlzq3iZWOAhbunUat6/QYmPMhkFMOAzZ5HDpCX5zf7oQfscHTyZ7E2D RJ2CvhhV5feF9+dMKgS6f3JcfWFEEuse2WWfYCCMnsX9zrnzBdbCsNSvQJwoJlwesXaF 6YIiUsHb73V3nSYnFWM/z+dgCQBsPYZUY6H3225Q/VdUGPqhAV3saQ0ttMvfAhVSu2Cv Zxbw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=OnPktCen; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id j8si19225094plt.134.2019.05.27.17.54.26; Mon, 27 May 2019 17:54:41 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=OnPktCen; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1727660AbfE1AxZ (ORCPT + 99 others); Mon, 27 May 2019 20:53:25 -0400 Received: from mail-pg1-f194.google.com ([209.85.215.194]:38674 "EHLO mail-pg1-f194.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1727641AbfE1AxX (ORCPT ); Mon, 27 May 2019 20:53:23 -0400 Received: by mail-pg1-f194.google.com with SMTP id v11so9870541pgl.5; Mon, 27 May 2019 17:53:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=vBM+WK9qA4pZcm6B2byvLBaRkxAGft4URtktgvLLdCc=; b=OnPktCen3c/b4BacE9HNm79x8hm0I4fjeUT40aTvy/XSMpSzH7A6hDmZZ5urbJ9sLD PK4i/dCD+H/9iRh6E5RXzxsgIrRpBc272MbHErF9ONhWkOJ34ZiMoWVpgGBrmfhFhpWM eCOJHtESFJWJnFryn8irE7Kv55d/BsGF23j1qIbEq7SixjjegSMT5GQ220mcjSzUioET DW88TdhpsKC8H648xo8OJEi7Ia9OjqPeOdQ3N7AORFPxfxHiJQJtJWAbTdtJ2CLQTgK3 xFSFza6WEsCP/+3/IPVdTpUzNXcb4ovTyfXb82yXXe5zNRV3jVg+9zBv0MEh/sWdJlgE uZwA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=vBM+WK9qA4pZcm6B2byvLBaRkxAGft4URtktgvLLdCc=; b=kBKYy1qzFYLeJq2c6yyI2AIpt75rkBtZsEiOeeIKwe7aWBYOhPVQgJf3TFpX76VsZi PTpajNMIBqrHThympxXA+XBkjmhfPqcx8GjwsCA5ODCRambi/rZwccvngF93wDPr/sdj Y9/sJ4jLsB4tuKEjMI4uuuWgmjluTWG0v6ZcZCF2OAZs6P1KvYlRbDN/WgWSNu3c0aZG 2TRXJAKM+87BMzUACPnLMK4cxc8uIJnufvGPsGvcv5D3KxNoSczfJkidhmsgWK7spAdx vxnPjo+ytl9JbdJHQ+TZRxHMzu6LuA9cOf/Ggvzz9f8GYk+uFHKQlg43e4tRsFNAEBs9 EJvw== X-Gm-Message-State: APjAAAWak42S9UW+BCQICTS9N1F7QQ7Y9paNCKfcY1lDnZiYPoBcV98p MYSRyMujjqsRzlLyYrH+ykrwNEt8 X-Received: by 2002:a63:c750:: with SMTP id v16mr127910115pgg.409.1559004802685; Mon, 27 May 2019 17:53:22 -0700 (PDT) Received: from localhost.localdomain ([203.205.141.123]) by smtp.googlemail.com with ESMTPSA id f16sm622085pja.18.2019.05.27.17.53.21 (version=TLS1_2 cipher=ECDHE-RSA-AES128-SHA bits=128/128); Mon, 27 May 2019 17:53:22 -0700 (PDT) From: Wanpeng Li X-Google-Original-From: Wanpeng Li To: linux-kernel@vger.kernel.org, kvm@vger.kernel.org Cc: Paolo Bonzini , =?UTF-8?q?Radim=20Kr=C4=8Dm=C3=A1=C5=99?= Subject: [PATCH v2 1/3] KVM: X86: Implement PV sched yield in linux guest Date: Tue, 28 May 2019 08:53:13 +0800 Message-Id: <1559004795-19927-2-git-send-email-wanpengli@tencent.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1559004795-19927-1-git-send-email-wanpengli@tencent.com> References: <1559004795-19927-1-git-send-email-wanpengli@tencent.com> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Wanpeng Li When sending a call-function IPI-many to vCPUs, yield if any of the IPI target vCPUs was preempted, we just select the first preempted target vCPU which we found since the state of target vCPUs can change underneath and to avoid race conditions. Cc: Paolo Bonzini Cc: Radim Krčmář Signed-off-by: Wanpeng Li --- Documentation/virtual/kvm/hypercalls.txt | 11 +++++++++++ arch/x86/include/uapi/asm/kvm_para.h | 1 + arch/x86/kernel/kvm.c | 21 +++++++++++++++++++++ include/uapi/linux/kvm_para.h | 1 + 4 files changed, 34 insertions(+) diff --git a/Documentation/virtual/kvm/hypercalls.txt b/Documentation/virtual/kvm/hypercalls.txt index da24c13..da21065 100644 --- a/Documentation/virtual/kvm/hypercalls.txt +++ b/Documentation/virtual/kvm/hypercalls.txt @@ -141,3 +141,14 @@ a0 corresponds to the APIC ID in the third argument (a2), bit 1 corresponds to the APIC ID a2+1, and so on. Returns the number of CPUs to which the IPIs were delivered successfully. + +7. KVM_HC_SCHED_YIELD +------------------------ +Architecture: x86 +Status: active +Purpose: Hypercall used to yield if the IPI target vCPU is preempted + +a0: destination APIC ID + +Usage example: When sending a call-function IPI-many to vCPUs, yield if +any of the IPI target vCPUs was preempted. diff --git a/arch/x86/include/uapi/asm/kvm_para.h b/arch/x86/include/uapi/asm/kvm_para.h index 19980ec..d0bf77c 100644 --- a/arch/x86/include/uapi/asm/kvm_para.h +++ b/arch/x86/include/uapi/asm/kvm_para.h @@ -29,6 +29,7 @@ #define KVM_FEATURE_PV_TLB_FLUSH 9 #define KVM_FEATURE_ASYNC_PF_VMEXIT 10 #define KVM_FEATURE_PV_SEND_IPI 11 +#define KVM_FEATURE_PV_SCHED_YIELD 12 #define KVM_HINTS_REALTIME 0 diff --git a/arch/x86/kernel/kvm.c b/arch/x86/kernel/kvm.c index 3f0cc82..54400c2 100644 --- a/arch/x86/kernel/kvm.c +++ b/arch/x86/kernel/kvm.c @@ -540,6 +540,21 @@ static void kvm_setup_pv_ipi(void) pr_info("KVM setup pv IPIs\n"); } +static void kvm_smp_send_call_func_ipi(const struct cpumask *mask) +{ + int cpu; + + native_send_call_func_ipi(mask); + + /* Make sure other vCPUs get a chance to run if they need to. */ + for_each_cpu(cpu, mask) { + if (vcpu_is_preempted(cpu)) { + kvm_hypercall1(KVM_HC_SCHED_YIELD, per_cpu(x86_cpu_to_apicid, cpu)); + break; + } + } +} + static void __init kvm_smp_prepare_cpus(unsigned int max_cpus) { native_smp_prepare_cpus(max_cpus); @@ -651,6 +666,12 @@ static void __init kvm_guest_init(void) #ifdef CONFIG_SMP smp_ops.smp_prepare_cpus = kvm_smp_prepare_cpus; smp_ops.smp_prepare_boot_cpu = kvm_smp_prepare_boot_cpu; + if (kvm_para_has_feature(KVM_FEATURE_PV_SCHED_YIELD) && + !kvm_para_has_hint(KVM_HINTS_REALTIME) && + kvm_para_has_feature(KVM_FEATURE_STEAL_TIME)) { + smp_ops.send_call_func_ipi = kvm_smp_send_call_func_ipi; + pr_info("KVM setup pv sched yield\n"); + } if (cpuhp_setup_state_nocalls(CPUHP_AP_ONLINE_DYN, "x86/kvm:online", kvm_cpu_online, kvm_cpu_down_prepare) < 0) pr_err("kvm_guest: Failed to install cpu hotplug callbacks\n"); diff --git a/include/uapi/linux/kvm_para.h b/include/uapi/linux/kvm_para.h index 6c0ce49..8b86609 100644 --- a/include/uapi/linux/kvm_para.h +++ b/include/uapi/linux/kvm_para.h @@ -28,6 +28,7 @@ #define KVM_HC_MIPS_CONSOLE_OUTPUT 8 #define KVM_HC_CLOCK_PAIRING 9 #define KVM_HC_SEND_IPI 10 +#define KVM_HC_SCHED_YIELD 11 /* * hypercalls use architecture specific -- 2.7.4