Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
References: <1530598891-21370-1-git-send-email-wanpengli@tencent.com>
 <1530598891-21370-3-git-send-email-wanpengli@tencent.com> <CALzav=fUmJZSvDgTvsB-7nyv1KiAMHgVcq-N+k0WCn3OsrVpNw@mail.gmail.com>
In-Reply-To: <CALzav=fUmJZSvDgTvsB-7nyv1KiAMHgVcq-N+k0WCn3OsrVpNw@mail.gmail.com>
From:   Wanpeng Li <kernellwp@gmail.com>
Date:   Fri, 20 Jul 2018 11:45:40 +0800
Message-ID: <CANRm+Cx8qy7Ttf42zNuEPDGBc-6J1jMpe8di7UGUL2E6bxmMDg@mail.gmail.com>
Subject: Re: [PATCH v3 2/6] KVM: X86: Implement PV IPIs in linux guest
To:     David Matlack <dmatlack@google.com>
Cc:     LKML <linux-kernel@vger.kernel.org>, kvm <kvm@vger.kernel.org>,
        Paolo Bonzini <pbonzini@redhat.com>,
        Radim Krcmar <rkrcmar@redhat.com>,
        Vitaly Kuznetsov <vkuznets@redhat.com>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Fri, 20 Jul 2018 at 07:05, David Matlack <dmatlack@google.com> wrote:
>
> On Mon, Jul 2, 2018 at 11:23 PM Wanpeng Li <kernellwp@gmail.com> wrote:
> >
> > From: Wanpeng Li <wanpengli@tencent.com>
> >
> > Implement paravirtual apic hooks to enable PV IPIs.
>
> Very cool. Thanks for working on this!

Thanks David!

>
> >
> > apic->send_IPI_mask
> > apic->send_IPI_mask_allbutself
> > apic->send_IPI_allbutself
> > apic->send_IPI_all
> >
> > The PV IPIs supports maximal 128 vCPUs VM, it is big enough for cloud
> > environment currently,
>
> From the Cloud perspective, 128 vCPUs is already obsolete. GCE's
> n1-utlramem-160 VMs have 160 vCPUs where the maximum APIC ID is 231.
> I'd definitely prefer an approach that scales to higher APIC IDs, like
> Paolo's offset idea.

Ok, I will try the offset method in next version.

>
> To Radim's point of real world performance testing, do you know what
> is the primary source of multi-target IPIs? If it's TLB shootdowns we
> might get a bigger bang for our buck with a PV TLB Shootdown.

The "Function Call interrupts", there is a lot of callers for
smp_call_function_many() except TLB Shootdowns in linux kernel which
try to run a function on a set of other CPUs. TLB Shootdown still can
get benefit from PV IPIs even if PV TLB Shootdown is enabled since
IPIs should be sent to the vCPUs which are active and will incur
vmexits. PV IPIs will benefit both vCPUs overcommit and
non-overcommit(which PV TLB Shootdown can't help) scenarios. Btw,
hyperv also implements PV IPIs even if PV TLB Shootdown is present.
https://lkml.org/lkml/2018/7/3/537

Regards,
Wanpeng Li