Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <20180622170625.30688-1-vkuznets@redhat.com> <CANRm+CwNfOkosECeSGaXXDaqTDvh1u228ztU8C5Xe95FGgNsFQ@mail.gmail.com>
 <877emkh98h.fsf@vitty.brq.redhat.com>
In-Reply-To: <877emkh98h.fsf@vitty.brq.redhat.com>
From:   Wanpeng Li <kernellwp@gmail.com>
Date:   Wed, 27 Jun 2018 18:27:08 +0800
Message-ID: <CANRm+CxwQyzW0ud2p=7ZubW0cTAen-4+Xw_T_gkjj4pz9fpofA@mail.gmail.com>
Subject: Re: [PATCH 0/4] x86/hyper-v: optimize PV IPIs
To:     Vitaly Kuznetsov <vkuznets@redhat.com>
Cc:     "the arch/x86 maintainers" <x86@kernel.org>,
        devel@linuxdriverproject.org, LKML <linux-kernel@vger.kernel.org>,
        "K. Y. Srinivasan" <kys@microsoft.com>,
        Haiyang Zhang <haiyangz@microsoft.com>,
        Stephen Hemminger <sthemmin@microsoft.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>,
        "H. Peter Anvin" <hpa@zytor.com>, Tianyu.Lan@microsoft.com,
        "Michael Kelley (EOSG)" <Michael.H.Kelley@microsoft.com>,
        kvm <kvm@vger.kernel.org>, Paolo Bonzini <pbonzini@redhat.com>,
        Radim Krcmar <rkrcmar@redhat.com>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, 27 Jun 2018 at 17:25, Vitaly Kuznetsov <vkuznets@redhat.com> wrote:
>
> Wanpeng Li <kernellwp@gmail.com> writes:
>
> > Hi Vitaly, (fix my reply mess this time)
> > On Sat, 23 Jun 2018 at 01:09, Vitaly Kuznetsov <vkuznets@redhat.com> wrote:
> >>
> >> When reviewing my "x86/hyper-v: use cheaper HVCALL_FLUSH_VIRTUAL_ADDRESS_
> >> {LIST,SPACE} hypercalls when possible" patch Michael suggested to apply the
> >> same idea to PV IPIs. Here we go!
> >>
> >> Despite what Hyper-V TLFS says about HVCALL_SEND_IPI hypercall, it can
> >> actually be 'fast' (passing parameters through registers). Use that too.
> >>
> >> This series can collide with my "KVM: x86: hyperv: PV IPI support for
> >> Windows guests" series as I rename ipi_arg_non_ex/ipi_arg_ex structures
> >> there. Depending on which one gets in first we may need to do tiny
> >> adjustments.
> >
> > As hyperv PV TLB flush has already been merged, is there any other
> > obvious multicast IPIs scenarios? qemu supports interrupt remapping
> > since two years ago, I think windows guest can switch to cluster mode
> > after entering x2APIC, so sending IPI per cluster.
>
>I got confused, which of my patch series are you actually looking at?
>:-)

Yeah, actually originally I want to reply the thread which you sent
out to kvm ml "KVM: x86: hyperv: PV IPI support for Windows guests"
and miss to reply this one since the subject is similar.

> When we manifest ourselves as Hyper-V Windows 'forgets' about x2apic
> mode: Hyper-V has a concept of 'Synthetic interrupt controller' - an
> xapic extension which we also support in KVM. I don't really know any
> obvious scenarios for mass IPIs in Windows besides TLB flush but I'm
> worried they may exist. Without PV IPIs any such attempt will likely
> lead to a crash.
>
> In general, I do care more about completeness and correctness of our
> Hyper-V emulation at this point: Windows is only being tested on 'real'
> Hyper-Vs so when we emulate a subset of enlightenments we're on our own
> when something is not working. It is also very helpfult for
> Linux-on-Hyper-V depelopment as we can see how Windows-on-Hyper-v
> behaves :-)
>
> > In addition, you
> > can also post the benchmark result for this PV IPI optimization,
> > although it also fixes the bug which you mentioned above.
>
> I'd love to get to know how to trigger mass IPIs in Windows so a
> benchmark can be performed...

I also not sure about windows. I use
https://lkml.org/lkml/2017/12/19/141 as a linux kernel module to
evaluate broadcast IPI performance in the linux guest laster year. :)

>
> > I can post one variant for Linux guest PV IPI if it also makes
> > sense. :)
>
> With x2apic support I'm actually not sure. Maybe configurations with
> a very large number of vCPUs and IPIs going to > 256 vCPUs can benefit
> from a 'single hypercall' solution.

Each cluster of x2apic cluster mode can just support 16 unique logical
IDs, so I think linux guest can also get benefit as long as VM has >
16 vCPUs. I will cook patches to evaluate it. :)

Regards,
Wanpeng Li