Received: by 2002:ac0:a581:0:0:0:0:0 with SMTP id m1-v6csp6169343imm; Wed, 27 Jun 2018 03:28:51 -0700 (PDT) X-Google-Smtp-Source: ADUXVKILIWXlzRRgiInY1SzZLRyrRLVIBIhsXkIpBgjIJGpdWUqUXtDiqxaBrV7r7mnlvJV+/cc9 X-Received: by 2002:a65:6657:: with SMTP id z23-v6mr4718286pgv.257.1530095331576; Wed, 27 Jun 2018 03:28:51 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1530095331; cv=none; d=google.com; s=arc-20160816; b=p9v7pzzfQ34Sgq47iOZJ8uYyKytP3ceGujHH89UVyh2gYw5qdsjrwZwSCe1xC00rJU FKJjtPpYB1fRUFTImem4aexSj4U0SKYujR9pT3bO4cIJOGeRLRDgakWrqh/Z0O6gJxxW rUlbrgvo/DQ+vRriGs+ruDV6ix8Bal1w9A84NnXD/kmELqvlsyNiK+dWlH1DWEpSUvFB tdzc23oCJF7hU7PZY7H0H/zupoHoE6r0NgbGuy1W7lILVmqIpl9ZAziDXZhiSeHNjdWI lcfpOZN82HlY/2I8ORdL/iKuD6daRS5WvuyS7NoDpa/4DTLPgLddMJdA+7dT68MzdmeR vKiA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature :arc-authentication-results; bh=62RYM+ZcHX9H3T2yH1Wop7i6AAEkAvVxwkADLNV79UE=; b=y9YQie6Xi/ZCc+Ea984C5kaH5hTJMmV2E/wezrB4eJSGsNQAIROKs+H/GuCuAnweRm IXd8cPolyVWA5psnLI/OkjoawrKQlWoRKVyJO6EGnv6Z7yr5TqI3wQTl4hP0RJ2tASB3 uBMLQlE0cTzNwVJ3v/I9I6IqDegftoTmpmCtG1rwE2AKiN/4094+e3V4vr5jpPsvLd1S YhQya5Rv96wwxzPBEp5Sd+m+H16FbIgWU+Amo+ffwBs5hPCBNX8pjOwcCleFnIzWb/vK VQE8Tsq0+ngAH24rCvHeYvZL7NVqV1I8VFDmaX6CCiLdc3Ts+mBUwtBGN/RVn3dUdBbn bIxA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=JtRYMeHS; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id z13-v6si1887948pge.255.2018.06.27.03.28.34; Wed, 27 Jun 2018 03:28:51 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20161025 header.b=JtRYMeHS; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753433AbeF0K1U (ORCPT + 99 others); Wed, 27 Jun 2018 06:27:20 -0400 Received: from mail-ot0-f196.google.com ([74.125.82.196]:46759 "EHLO mail-ot0-f196.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752035AbeF0K1S (ORCPT ); Wed, 27 Jun 2018 06:27:18 -0400 Received: by mail-ot0-f196.google.com with SMTP id v24-v6so1607471otk.13; Wed, 27 Jun 2018 03:27:18 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=62RYM+ZcHX9H3T2yH1Wop7i6AAEkAvVxwkADLNV79UE=; b=JtRYMeHSzbPUyJTWyJXuaecGkn4NXjGUEQKQtP1YPFSfo4AwV8v6jcHru5ulosmgh/ IUe6ttaeSdHuWew79aOp5PawkLQOaUOgkOld8+M5ASeQyLqSe7oH9eQ14f/JCM0jBGbY XIEXbJISRvWr0STcaX7I8IjkQ/nLlfjN8UUwWAsOXk2NGt8gHIbUBLKYt7jV2ftUfWn7 0bbGP7A7Q5pDS9U1EcpSgsIm5vHIT36+VnA3xnnIiQTCG/zBcj/TfFmBqjknIgSI/RX2 oVkUgvyGeChBiQSub15GwWqPKlCqlj04OzgYMb0f0dSPGwAjN40e84xsr6/yRXNQs0tr yBfA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=62RYM+ZcHX9H3T2yH1Wop7i6AAEkAvVxwkADLNV79UE=; b=bBC1Eung96fu+tZ4xq8buKDkxyLMB/nD49KZxNwXXh2BoTTzfoGyQbuLSuY+6NcD2p BcQVvcXRUP5OdWRPtWqOSktsDrCoEdxBKjrlic1YQCovb2+241b2brbYFopTpYRe9rCB GXUTiDZBAGDaLdi6ZpYIb3StYcPD639xVwRm8BT3HZ9pWkvrEJyilKEhum3SUdxMFNxW PhiRQhbt25FHLpDjn3KQ3tTFdmDK9X5CR39Nkr/X71mSI8kEcRsE0JcTyhylNoH4pZbG kPAYWm9uwDIamlOqLelEEaNuRWxkaVfM7nyd9Pu+DJHgOKZuKHtGEp3VTmqxd/e+T1kK nRhA== X-Gm-Message-State: APt69E03J9EQETdB+9dc/DlmadjWNCq2uMg/DH+GvOdszBYgcvq6k92B oTEorq+LrImfoRxQldxB1N/G+XeSC3aH1FK0SjE= X-Received: by 2002:a9d:184:: with SMTP id e4-v6mr3076556ote.249.1530095237856; Wed, 27 Jun 2018 03:27:17 -0700 (PDT) MIME-Version: 1.0 References: <20180622170625.30688-1-vkuznets@redhat.com> <877emkh98h.fsf@vitty.brq.redhat.com> In-Reply-To: <877emkh98h.fsf@vitty.brq.redhat.com> From: Wanpeng Li Date: Wed, 27 Jun 2018 18:27:08 +0800 Message-ID: Subject: Re: [PATCH 0/4] x86/hyper-v: optimize PV IPIs To: Vitaly Kuznetsov Cc: "the arch/x86 maintainers" , devel@linuxdriverproject.org, LKML , "K. Y. Srinivasan" , Haiyang Zhang , Stephen Hemminger , Thomas Gleixner , Ingo Molnar , "H. Peter Anvin" , Tianyu.Lan@microsoft.com, "Michael Kelley (EOSG)" , kvm , Paolo Bonzini , Radim Krcmar Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Wed, 27 Jun 2018 at 17:25, Vitaly Kuznetsov wrote: > > Wanpeng Li writes: > > > Hi Vitaly, (fix my reply mess this time) > > On Sat, 23 Jun 2018 at 01:09, Vitaly Kuznetsov wrote: > >> > >> When reviewing my "x86/hyper-v: use cheaper HVCALL_FLUSH_VIRTUAL_ADDRESS_ > >> {LIST,SPACE} hypercalls when possible" patch Michael suggested to apply the > >> same idea to PV IPIs. Here we go! > >> > >> Despite what Hyper-V TLFS says about HVCALL_SEND_IPI hypercall, it can > >> actually be 'fast' (passing parameters through registers). Use that too. > >> > >> This series can collide with my "KVM: x86: hyperv: PV IPI support for > >> Windows guests" series as I rename ipi_arg_non_ex/ipi_arg_ex structures > >> there. Depending on which one gets in first we may need to do tiny > >> adjustments. > > > > As hyperv PV TLB flush has already been merged, is there any other > > obvious multicast IPIs scenarios? qemu supports interrupt remapping > > since two years ago, I think windows guest can switch to cluster mode > > after entering x2APIC, so sending IPI per cluster. > >I got confused, which of my patch series are you actually looking at? >:-) Yeah, actually originally I want to reply the thread which you sent out to kvm ml "KVM: x86: hyperv: PV IPI support for Windows guests" and miss to reply this one since the subject is similar. > When we manifest ourselves as Hyper-V Windows 'forgets' about x2apic > mode: Hyper-V has a concept of 'Synthetic interrupt controller' - an > xapic extension which we also support in KVM. I don't really know any > obvious scenarios for mass IPIs in Windows besides TLB flush but I'm > worried they may exist. Without PV IPIs any such attempt will likely > lead to a crash. > > In general, I do care more about completeness and correctness of our > Hyper-V emulation at this point: Windows is only being tested on 'real' > Hyper-Vs so when we emulate a subset of enlightenments we're on our own > when something is not working. It is also very helpfult for > Linux-on-Hyper-V depelopment as we can see how Windows-on-Hyper-v > behaves :-) > > > In addition, you > > can also post the benchmark result for this PV IPI optimization, > > although it also fixes the bug which you mentioned above. > > I'd love to get to know how to trigger mass IPIs in Windows so a > benchmark can be performed... I also not sure about windows. I use https://lkml.org/lkml/2017/12/19/141 as a linux kernel module to evaluate broadcast IPI performance in the linux guest laster year. :) > > > I can post one variant for Linux guest PV IPI if it also makes > > sense. :) > > With x2apic support I'm actually not sure. Maybe configurations with > a very large number of vCPUs and IPIs going to > 256 vCPUs can benefit > from a 'single hypercall' solution. Each cluster of x2apic cluster mode can just support 16 unique logical IDs, so I think linux guest can also get benefit as long as VM has > 16 vCPUs. I will cook patches to evaluate it. :) Regards, Wanpeng Li