Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: [PATCH RFC v3 1/6] x86/paravirt: Add pv_idle_ops to paravirt ops
To:     Juergen Gross <jgross@suse.com>, Quan Xu <quan.xu03@gmail.com>,
        kvm@vger.kernel.org, linux-doc@vger.kernel.org,
        linux-fsdevel@vger.kernel.org, linux-kernel@vger.kernel.org,
        virtualization@lists.linux-foundation.org, x86@kernel.org,
        xen-devel@lists.xenproject.org
Cc:     Yang Zhang <yang.zhang.wz@gmail.com>,
        Alok Kataria <akataria@vmware.com>,
        Rusty Russell <rusty@rustcorp.com.au>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>,
        "H. Peter Anvin" <hpa@zytor.com>
References: <1510567565-5118-1-git-send-email-quan.xu0@gmail.com>
 <1510567565-5118-2-git-send-email-quan.xu0@gmail.com>
 <07fac696-e3d4-8f35-8f3d-764d7ab41204@suse.com>
 <902da704-1e4f-583b-91c3-1a62ccd6e73d@gmail.com>
 <fe6eeed1-4eee-eaaa-df3b-8979af8a3891@suse.com>
From:   Quan Xu <quan.xu0@gmail.com>
Message-ID: <79dad15c-2d26-bcf3-7283-293e42a161ea@gmail.com>
Date:   Tue, 14 Nov 2017 17:38:57 +0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.4.0
MIME-Version: 1.0
In-Reply-To: <fe6eeed1-4eee-eaaa-df3b-8979af8a3891@suse.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 8bit
Content-Language: en-US
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk


On 2017/11/14 15:30, Juergen Gross wrote:
> On 14/11/17 08:02, Quan Xu wrote:
>>
>> On 2017/11/13 18:53, Juergen Gross wrote:
>>> On 13/11/17 11:06, Quan Xu wrote:
>>>> From: Quan Xu <quan.xu0@gmail.com>
>>>>
>>>> So far, pv_idle_ops.poll is the only ops for pv_idle. .poll is called
>>>> in idle path which will poll for a while before we enter the real idle
>>>> state.
>>>>
>>>> In virtualization, idle path includes several heavy operations
>>>> includes timer access(LAPIC timer or TSC deadline timer) which will
>>>> hurt performance especially for latency intensive workload like message
>>>> passing task. The cost is mainly from the vmexit which is a hardware
>>>> context switch between virtual machine and hypervisor. Our solution is
>>>> to poll for a while and do not enter real idle path if we can get the
>>>> schedule event during polling.
>>>>
>>>> Poll may cause the CPU waste so we adopt a smart polling mechanism to
>>>> reduce the useless poll.
>>>>
>>>> Signed-off-by: Yang Zhang <yang.zhang.wz@gmail.com>
>>>> Signed-off-by: Quan Xu <quan.xu0@gmail.com>
>>>> Cc: Juergen Gross <jgross@suse.com>
>>>> Cc: Alok Kataria <akataria@vmware.com>
>>>> Cc: Rusty Russell <rusty@rustcorp.com.au>
>>>> Cc: Thomas Gleixner <tglx@linutronix.de>
>>>> Cc: Ingo Molnar <mingo@redhat.com>
>>>> Cc: "H. Peter Anvin" <hpa@zytor.com>
>>>> Cc: x86@kernel.org
>>>> Cc: virtualization@lists.linux-foundation.org
>>>> Cc: linux-kernel@vger.kernel.org
>>>> Cc: xen-devel@lists.xenproject.org
>>> Hmm, is the idle entry path really so critical to performance that a new
>>> pvops function is necessary?
>> Juergen, Here is the data we get when running benchmark netperf:
>>   1. w/o patch and disable kvm dynamic poll (halt_poll_ns=0):
>>      29031.6 bit/s -- 76.1 %CPU
>>
>>   2. w/ patch and disable kvm dynamic poll (halt_poll_ns=0):
>>      35787.7 bit/s -- 129.4 %CPU
>>
>>   3. w/ kvm dynamic poll:
>>      35735.6 bit/s -- 200.0 %CPU
>>
>>   4. w/patch and w/ kvm dynamic poll:
>>      42225.3 bit/s -- 198.7 %CPU
>>
>>   5. idle=poll
>>      37081.7 bit/s -- 998.1 %CPU
>>
>>
>>
>>   w/ this patch, we will improve performance by 23%.. even we could improve
>>   performance by 45.4%, if we use w/patch and w/ kvm dynamic poll. also the
>>   cost of CPU is much lower than 'idle=poll' case..
> I don't question the general idea. I just think pvops isn't the best way
> to implement it.
>
>>> Wouldn't a function pointer, maybe guarded
>>> by a static key, be enough? A further advantage would be that this would
>>> work on other architectures, too.
>> I assume this feature will be ported to other archs.. a new pvops makes

       sorry, a typo.. /other archs/other hypervisors/
       it refers hypervisor like Xen, HyperV and VMware)..

>> code
>> clean and easy to maintain. also I tried to add it into existed pvops,
>> but it
>> doesn't match.
> You are aware that pvops is x86 only?

yes, I'm aware..

> I really don't see the big difference in maintainability compared to the
> static key / function pointer variant:
>
> void (*guest_idle_poll_func)(void);
> struct static_key guest_idle_poll_key __read_mostly;
>
> static inline void guest_idle_poll(void)
> {
> 	if (static_key_false(&guest_idle_poll_key))
> 		guest_idle_poll_func();
> }


thank you for your sample code :)
I agree there is no big difference.. I think we are discussion for two 
things:
  1) x86 VM on different hypervisors
  2) different archs VM on kvm hypervisor

What I want to do is x86 VM on different hypervisors, such as kvm / xen 
/ hyperv ..

> And KVM would just need to set guest_idle_poll_func and enable the
> static key. Works on non-x86 architectures, too.
>

.. referred to 'pv_mmu_ops', HyperV and Xen can implement their own 
functions for 'pv_mmu_ops'.
I think it is the same to pv_idle_ops.

with above explaination, do you still think I need to define the static
key/function pointer variant?

btw, any interest to port it to Xen HVM guest? :)

Quan
Alibaba Cloud

From 1584030927342122693@xxx Tue Nov 14 08:53:31 +0000 2017
X-GM-THRID: 1583947963447418004
X-Gmail-Labels: Inbox,Category Forums,HistoricalUnread