Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754326Ab0DQTDN (ORCPT ); Sat, 17 Apr 2010 15:03:13 -0400 Received: from mx1.redhat.com ([209.132.183.28]:2569 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753708Ab0DQTDM (ORCPT ); Sat, 17 Apr 2010 15:03:12 -0400 Message-ID: <4BCA05D6.4010709@redhat.com> Date: Sat, 17 Apr 2010 22:02:46 +0300 From: Avi Kivity User-Agent: Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.9) Gecko/20100330 Fedora/3.0.4-1.fc12 Thunderbird/3.0.4 MIME-Version: 1.0 To: "Zhang, Xiantao" CC: "kvm@vger.kernel.org" , Marcelo Tosatti , "Yang, Xiaowei" , "Dong, Eddie" , "Li, Xin" , Ingo Molnar , Peter Zijlstra , Mike Galbraith , Linux Kernel Mailing List Subject: Re: VM performance issue in KVM guests. References: <4BC0D125.9050108@redhat.com> <4BC2C07B.4040607@redhat.com> <4BC41338.80206@redhat.com> <4BC57949.2090003@redhat.com> In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 1878 Lines: 44 On 04/16/2010 05:27 AM, Zhang, Xiantao wrote: > > >> When vcpus are pinned to pcpus, there is a 50% chance that a guest's >> vcpus will be co-scheduled and spinlocks will perform will. >> >> When vcpus are not pinned, but affine wakeups are disabled, there is a >> 33% chance that vcpus will be co-scheduled. >> >> When vcpus are not pinned and affine wakeups are enabled there is a 0% >> chance that vcpus will be co-scheduled. >> >> Keeping both vcpus on the same core actually makes sense since they >> can communicate through the local cache faster than across cores. >> What we need is to make sure that they don't spin. >> >> Windows 2008 can report spinlock spinning through a hypercall. Can >> you hook to that interface and see if it happens regularly? >> Altenatively use a PLE capable host and trace the kvm_vcpu_on_spin() >> function. >> > We only tried windows 2003 for the experiments, and have no data related to windows 2008. > But maybe we can have a try later. Anyway, the key point is we have to enhance the scheduler to let it > Know which threads are vcpu threads to avoid perf loss in this case. > I have two worries about this approach: 1. Affine wakeups were introduced for a reason; if we disable them (even just for vcpus), we lost something. Maybe we can tune the mechanism not to fail, instead of disabling it. 2. Affine wakeups are a scheduler internal detail. How do we explain what it does? the scheduler may not have affine wakeups in a few years, yet we'll have an ABI to disable them. -- Do not meddle in the internals of kernels, for they are subtle and quick to panic. -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/