Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
MIME-Version: 1.0
In-Reply-To: <87d14vrcou.fsf@vitty.brq.redhat.com>
References: <20170802160921.21791-1-vkuznets@redhat.com> <CANRm+Cz7SgUJ3o2O=uDn+un4JYiqQ-aANhf4tNNArfbv6a_uOQ@mail.gmail.com>
 <87d14vrcou.fsf@vitty.brq.redhat.com>
From:   Wanpeng Li <kernellwp@gmail.com>
Date:   Mon, 6 Nov 2017 17:57:04 +0800
Message-ID: <CANRm+Cz+0kAFjpGudnoXsPnWzuM2368KDBxivV4Xi-nPwt=dXw@mail.gmail.com>
Subject: Re: [PATCH v10 0/9] Hyper-V: paravirtualized remote TLB flushing and
 hypercall improvements
To:     Vitaly Kuznetsov <vkuznets@redhat.com>
Cc:     "the arch/x86 maintainers" <x86@kernel.org>,
        devel@linuxdriverproject.org,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        "K. Y. Srinivasan" <kys@microsoft.com>,
        Haiyang Zhang <haiyangz@microsoft.com>,
        Stephen Hemminger <sthemmin@microsoft.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>,
        "H. Peter Anvin" <hpa@zytor.com>,
        Steven Rostedt <rostedt@goodmis.org>,
        Jork Loeser <Jork.Loeser@microsoft.com>,
        Simon Xiao <sixiao@microsoft.com>,
        Andy Lutomirski <luto@kernel.org>,
        Andy Shevchenko <andy.shevchenko@gmail.com>,
        Paolo Bonzini <pbonzini@redhat.com>, kvm <kvm@vger.kernel.org>
Content-Type: text/plain; charset="UTF-8"
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

2017-11-06 17:14 GMT+08:00 Vitaly Kuznetsov <vkuznets@redhat.com>:
> Wanpeng Li <kernellwp@gmail.com> writes:
>
>> 2017-08-03 0:09 GMT+08:00 Vitaly Kuznetsov <vkuznets@redhat.com>:
>>> Changes since v9:
>>> - Rebase to 4.13-rc3.
>>> - Drop PATCH1 as it was already taken by Greg to char-misc tree. There're no
>>>   functional dependencies on this patch so the series can go through a different tree
>>>   (and it actually belongs to x86 if I got Ingo's comment right).
>>> - Add in missing void return type in PATCH1 [Colin King, Ingo Molnar, Greg KH]
>>> - A few minor fixes in what is now PATCH7: add pr_fmt, tiny style fix in
>>>   hyperv_flush_tlb_others() [Andy Shevchenko]
>>> - Fix "error: implicit declaration of function 'virt_to_phys'" in PATCH2
>>>   reported by kbuild test robot (#include <asm/io.h>)
>>> - Add Steven's 'Reviewed-by:' to PATCH9.
>>>
>>> Original description:
>>>
>>> Hyper-V supports hypercalls for doing local and remote TLB flushing and
>>> gives its guests hints when using hypercall is preferred. While doing
>>> hypercalls for local TLB flushes is probably not practical (and is not
>>> being suggested by modern Hyper-V versions) remote TLB flush with a
>>> hypercall brings significant improvement.
>>>
>>> To test the series I wrote a special 'TLB trasher': on a 16 vCPU guest I
>>> was creating 32 threads which were doing 100000 mmap/munmaps each on some
>>> big file. Here are the results:
>>>
>>> Before:
>>> # time ./pthread_mmap ./randfile
>>> real    3m33.118s
>>> user    0m3.698s
>>> sys     3m16.624s
>>>
>>> After:
>>> # time ./pthread_mmap ./randfile
>>> real    2m19.920s
>>> user    0m2.662s
>>> sys     2m9.948s
>>>
>>> This series brings a number of small improvements along the way: fast
>>> hypercall implementation and using it for event signaling, rep hypercalls
>>> implementation, hyperv tracing subsystem (which only traces the newly added
>>> remote TLB flush for now).
>>>
>>
>> Hi Vitaly,
>>
>> Could you attach your benchmark? I'm interested in to try the
>> implementation in paravirt kvm.
>>
>
> Oh, this would be cool) I briefly discussed the idea with Radim (one of
> KVM maintainers) during the last KVM Forum and he wasn't opposed to the
> idea. Need to talk to Paolo too. Good thing is that we have everything

I talk with Paolo today and he points this feature to me, so I believe
he likes it. :) In addition,
https://docs.microsoft.com/en-us/virtualization/hyper-v-on-windows/reference/tlfs
I search Hypervisor Top Level Functional Specification v5.0b.pdf
document but didn't find a section introduce the Hyper-V:
paravirtualized remote TLB flushing and hypercall stuff, could you
point out?

Regards,
Wanpeng Li

> in place for guests now (HAVE_RCU_TABLE_FREE is enabled globaly on x86).
>
> Please see the microbenchmark attached. Adjust defines in the beginning
> to match your needs. It is not anything smart, basically just a TLB
> trasher.
>
> In theory, the best result is achived when we're overcommiting the host
> by running multiple vCPUs on each pCPU. In this case PV tlb flush avoids
> touching vCPUs which are not scheduled and avoid the wait on the main
> CPU.
>
> --
>   Vitaly
>

From 1583307904382142267@xxx Mon Nov 06 09:21:23 +0000 2017
X-GM-THRID: 1574636603958149385
X-Gmail-Labels: Inbox,Category Forums,HistoricalUnread