Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1753026AbbH2Wb1 (ORCPT ); Sat, 29 Aug 2015 18:31:27 -0400 Received: from mail.pfak.org ([198.50.219.4]:41634 "EHLO mail.pfak.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752422AbbH2WbZ (ORCPT ); Sat, 29 Aug 2015 18:31:25 -0400 X-Greylist: delayed 311 seconds by postgrey-1.27 at vger.kernel.org; Sat, 29 Aug 2015 18:31:25 EDT DomainKey-Signature: a=rsa-sha1; c=nofws; d=kieser.ca; h=subject:to :references:cc:from:message-id:date:mime-version:in-reply-to :content-type; q=dns; s=mail; b=HPxXHWsRDQl1+XOwIWvkquLrhwU/aZh3 XtFcceBfE2WqSsJCURLmMPrwAv/ISHn7BA/nqF1v0YHd0oT/v78g/JWJeqwYQ9Sq dcKlTTGYDoU80d5gzGzVGHDZxoc4HAypoecNtC4U9zoYD2fI0CxRfeH1vT0InbYz m9k3nJ7Drss= Subject: Re: [PATCH v4 0/3] KVM: Dynamic Halt-Polling To: Wanpeng Li References: <55E221B2.3000702@kieser.ca> Cc: Paolo Bonzini , David Matlack , kvm , "linux-kernel@vger.kernel.org" From: Peter Kieser Message-ID: <55E23183.3050806@kieser.ca> Date: Sat, 29 Aug 2015 15:26:11 -0700 User-Agent: Mozilla/5.0 (Windows NT 10.0; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-256; boundary="------------ms060805030702080605010604" Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 12048 Lines: 244 This is a cryptographically signed message in MIME format. --------------ms060805030702080605010604 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable Thanks, Wanpeng. Applied this to Linux 3.18 and seeing much higher CPU=20 usage (200%) for qemu 2.4.0 process on a Windows 10 x64 guest. qemu=20 parameters: qemu-system-x86_64 -enable-kvm -name arwan-20150704 -S -machine=20 pc-q35-2.2,accel=3Dkvm,usb=3Doff -cpu=20 Haswell,hv_time,hv_relaxed,hv_vapic,hv_spinlocks=3D0x1000 -m 8192=20 -realtime mlock=3Doff -smp 4,sockets=3D4,cores=3D1,threads=3D1 -uuid=20 7c2fc02d-2798-4fc9-ad04-db5f1af92723 -no-user-config -nodefaults=20 -chardev=20 socket,id=3Dcharmonitor,path=3D/var/lib/libvirt/qemu/arwan-20150704.monit= or,server,nowait=20 -mon chardev=3Dcharmonitor,id=3Dmonitor,mode=3Dcontrol -rtc base=3Dlocalt= ime=20 -no-shutdown -boot strict=3Don -device=20 i82801b11-bridge,id=3Dpci.1,bus=3Dpcie.0,addr=3D0x1e -device=20 pci-bridge,chassis_nr=3D2,id=3Dpci.2,bus=3Dpci.1,addr=3D0x1 -device=20 nec-usb-xhci,id=3Dusb1,bus=3Dpci.2,addr=3D0x4 -device=20 virtio-serial-pci,id=3Dvirtio-serial0,bus=3Dpci.2,addr=3D0x5 -drive=20 file=3D/dev/mapper/crypt-arwan-20150704,if=3Dnone,id=3Ddrive-virtio-disk0= ,format=3Draw,cache=3Dnone,discard=3Dunmap,aio=3Dnative=20 -device=20 virtio-blk-pci,scsi=3Doff,bus=3Dpci.2,addr=3D0x3,drive=3Ddrive-virtio-dis= k0,id=3Dvirtio-disk0,bootindex=3D2=20 -drive=20 file=3D/usr/share/virtio-win/virtio-win.iso,if=3Dnone,media=3Dcdrom,id=3D= drive-sata0-0-2,readonly=3Don,format=3Draw=20 -device ide-cd,bus=3Dide.2,drive=3Ddrive-sata0-0-2,id=3Dsata0-0-2,bootind= ex=3D1=20 -netdev tap,fds=3D31:32:33:34,id=3Dhostnet0,vhost=3Don,vhostfds=3D35:36:3= 7:38=20 -device=20 virtio-net-pci,guest_csum=3Doff,guest_tso4=3Doff,guest_tso6=3Doff,mq=3Don= ,vectors=3D10,netdev=3Dhostnet0,id=3Dnet0,mac=3D52:54:00:f3:6b:c4,bus=3Dp= ci.2,addr=3D0x2=20 -chardev=20 socket,id=3Dcharchannel0,path=3D/var/lib/libvirt/qemu/channel/target/arwa= n-20150704.org.qemu.guest_agent.0,server,nowait=20 -device=20 virtserialport,bus=3Dvirtio-serial0.0,nr=3D1,chardev=3Dcharchannel0,id=3D= channel0,name=3Dorg.qemu.guest_agent.0=20 -chardev spicevmc,id=3Dcharchannel1,name=3Dvdagent -device=20 virtserialport,bus=3Dvirtio-serial0.0,nr=3D2,chardev=3Dcharchannel1,id=3D= channel1,name=3Dcom.redhat.spice.0=20 -vnc 127.0.0.1:4 -device=20 qxl-vga,id=3Dvideo0,ram_size=3D67108864,vram_size=3D67108864,vgamem_mb=3D= 16,bus=3Dpcie.0,addr=3D0x1=20 -device virtio-balloon-pci,id=3Dballoon0,bus=3Dpci.2,addr=3D0x1 -msg time= stamp=3Don I revert patch, qemu shows 17% CPU usage on host. Thoughts? -Peter On 2015-08-29 3:21 PM, Wanpeng Li wrote: > Hi Peter, > On 8/30/15 5:18 AM, Peter Kieser wrote: >> Hi Wanpeng, >> >> Do I need to set any module parameters to use your patch, or should=20 >> halt_poll_ns automatically tune with just your patch series applied? >> > > You don't need any module parameters. > > Regards, > Wanpeng Li > >> Thanks. >> >> On 2015-08-27 2:47 AM, Wanpeng Li wrote: >>> v3 -> v4: >>> * bring back grow vcpu->halt_poll_ns when interrupt arrives and=20 >>> shrinks >>> when idle VCPU is detected >>> >>> v2 -> v3: >>> * grow/shrink vcpu->halt_poll_ns by *halt_poll_ns_grow or=20 >>> /halt_poll_ns_shrink >>> * drop the macros and hard coding the numbers in the param=20 >>> definitions >>> * update the comments "5-7 us" >>> * remove halt_poll_ns_max and use halt_poll_ns as the max=20 >>> halt_poll_ns time, >>> vcpu->halt_poll_ns start at zero >>> * drop the wrappers >>> * move the grow/shrink logic before "out:" w/ "if (waited)" >>> >>> v1 -> v2: >>> * change kvm_vcpu_block to read halt_poll_ns from the vcpu instead = of >>> the module parameter >>> * use the shrink/grow matrix which is suggested by David >>> * set halt_poll_ns_max to 2ms >>> >>> There is a downside of halt_poll_ns since poll is still happen for id= le >>> VCPU which can waste cpu usage. This patchset add the ability to adju= st >>> halt_poll_ns dynamically, grows halt_poll_ns if an interrupt arrives = >>> and >>> shrinks halt_poll_ns when idle VCPU is detected. >>> >>> There are two new kernel parameters for changing the halt_poll_ns: >>> halt_poll_ns_grow and halt_poll_ns_shrink. >>> >>> >>> Test w/ high cpu overcommit ratio, pin vCPUs, and the halt_poll_ns of= >>> halt-poll is the default 500000ns, the max halt_poll_ns of dynamic >>> halt-poll is 2ms. Then watch the %C0 in the dump of Powertop tool. >>> The test method is almost from David. >>> >>> +-----------------+----------------+-------------------+ >>> | | | | >>> | w/o halt-poll | w/ halt-poll | dynamic halt-poll | >>> +-----------------+----------------+-------------------+ >>> | | | | >>> | ~0.9% | ~1.8% | ~1.2% | >>> +-----------------+----------------+-------------------+ >>> The always halt-poll=20 >>> will increase ~0.9% cpu usage for idle vCPUs and the >>> dynamic halt-poll drop it to ~0.3% which means that reduce the 67%=20 >>> overhead >>> introduced by always halt-poll. >>> >>> Wanpeng Li (3): >>> KVM: make halt_poll_ns per-VCPU >>> KVM: dynamic halt_poll_ns adjustment >>> KVM: trace kvm_halt_poll_ns grow/shrink >>> >>> include/linux/kvm_host.h | 1 + >>> include/trace/events/kvm.h | 30 ++++++++++++++++++++++++++++ >>> virt/kvm/kvm_main.c | 50=20 >>> +++++++++++++++++++++++++++++++++++++++++++--- >>> 3 files changed, 78 insertions(+), 3 deletions(-) >> > --=20 Peter Kieser 604.338.9294 / peter@kieser.ca --------------ms060805030702080605010604 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgEFADCABgkqhkiG9w0BBwEAAKCC DKswggY0MIIEHKADAgECAgEgMA0GCSqGSIb3DQEBBQUAMH0xCzAJBgNVBAYTAklMMRYwFAYD VQQKEw1TdGFydENvbSBMdGQuMSswKQYDVQQLEyJTZWN1cmUgRGlnaXRhbCBDZXJ0aWZpY2F0 ZSBTaWduaW5nMSkwJwYDVQQDEyBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTAe Fw0wNzEwMjQyMTAyNTVaFw0xNzEwMjQyMTAyNTVaMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UE ChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMiU2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUg U2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xhc3MgMiBQcmltYXJ5IEludGVybWVkaWF0 ZSBDbGllbnQgQ0EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQDLKIVFnAEs+xny q6UzjCqgDcvQVe1dIoFnRsQPCFO+y92k8RK0Pn3MbQ2Gd+mehh9GBZ+36uUQA7Xj9AGM6wgP hEE34vKtfpAN5tJ8LcFxveDObCKrL7O5UT9WsnAZHv7OYPYSR68mdmnEnJ83M4wQgKO19b+R t8sPDAz9ptkQsntCn4GeJzg3q2SVc4QJTg/WHo7wF2ah5LMOeh8xJVSKGEmd6uPkSbj113yK Mm8vmNptRPmM1+YgmVwcdOYJOjCgFtb2sOP79jji8uhWR91xx7TpM1K3hv/wrBZwffrmmEpU euXHRs07JqCCvFh9coKF4UQZvfEg+x3/69xRCzb1AgMBAAGjggGtMIIBqTAPBgNVHRMBAf8E BTADAQH/MA4GA1UdDwEB/wQEAwIBBjAdBgNVHQ4EFgQUrlWDb+wxyrn3HfqvazHzyB3jrLsw HwYDVR0jBBgwFoAUTgvvGqRAW6UXaYcwyjRoQ9BBrvIwZgYIKwYBBQUHAQEEWjBYMCcGCCsG AQUFBzABhhtodHRwOi8vb2NzcC5zdGFydHNzbC5jb20vY2EwLQYIKwYBBQUHMAKGIWh0dHA6 Ly93d3cuc3RhcnRzc2wuY29tL3Nmc2NhLmNydDBbBgNVHR8EVDBSMCegJaAjhiFodHRwOi8v d3d3LnN0YXJ0c3NsLmNvbS9zZnNjYS5jcmwwJ6AloCOGIWh0dHA6Ly9jcmwuc3RhcnRzc2wu Y29tL3Nmc2NhLmNybDCBgAYDVR0gBHkwdzB1BgsrBgEEAYG1NwECATBmMC4GCCsGAQUFBwIB FiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMDQGCCsGAQUFBwIBFihodHRw Oi8vd3d3LnN0YXJ0c3NsLmNvbS9pbnRlcm1lZGlhdGUucGRmMA0GCSqGSIb3DQEBBQUAA4IC AQA6qScNyNO0FpHvaZTQacVMXH33O51KyEKSRw3IvdQxRu31YR0ZDGdSfgSoOVDVMSBSdmfQ fdDInHPzV3LO5DwUXZ+lxjv7z3PO2OkfnFkvTXPfn6dxJ5rJveDsTsCPcJ/Kp6/+qN5g+J6D /SaYcFD018B6L42r0Z4VEBy36P4tjRtF14Ex10tl5tJFVKM16qWKQHbpjIgf73s49UB0CQ5l HT2DHKfq3oPfdNc5Mk93w1v4ryVb+qVrZIej8NsrWU+5r4O2IV91edDb/OtHFddZqHFFXKgS 79IHE/hwQ2LW7r3sTX7cDUCg+dfdwO8zeLxuwk2JF8crUoyrl66RGrRIhT8VoG/OJ1Y9uUlO av69V4cG8upi4ZG2l7JZFbcBFk91Wp+Payo5SuF61CmGFrZ386umkmpObtFacXda2O/bVoQ9 xHQrzoTc/0KZTWvlZCLK3Ke/vGYT9ZdW9lOjGsSFbXrlTA919L84iMK+48WGnvRWY28ZaVHp ql43AtEGhXze6iNCbEDACy+4hkQYOytAqDgcxAnQ937mYpeZFPyz/XK9QSt9VNFMuudWxZwD DDJKoQAoSG59Hou9lZ26UrK60nRdAQBmEPL8h2nuWgoPh++XVQld9yuhbsWa39Pck8/lcfz5 HUVGJF5mc/zk38iV7FDlF68puiryNq2KXHEpOTCCBm8wggVXoAMCAQICAlHLMA0GCSqGSIb3 DQEBBQUAMIGMMQswCQYDVQQGEwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UE CxMiU2VjdXJlIERpZ2l0YWwgQ2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRD b20gQ2xhc3MgMiBQcmltYXJ5IEludGVybWVkaWF0ZSBDbGllbnQgQ0EwHhcNMTQwODE3MTAz OTA5WhcNMTYwODE3MTM1NTMxWjCBjjEZMBcGA1UEDRMQSkhyOElnZDY4ZW1FMHlRejELMAkG A1UEBhMCQ0ExGTAXBgNVBAgTEEJyaXRpc2ggQ29sdW1iaWExEjAQBgNVBAcTCVZhbmNvdXZl cjEVMBMGA1UEAxMMUGV0ZXIgS2llc2VyMR4wHAYJKoZIhvcNAQkBFg9wZXRlckBraWVzZXIu Y2EwggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC/zZi0sMysoRWvMFWODp+A9mrV vozuJKsxo7d9hiE8RIcHnkdgHkBYZgV08MABxpwBFlPpUEl6CKFfR3YvBoU3hc9wOSz0y5f8 GunZr/FQn2GkRL6EsYxDeJ7ArXVPD+jS1BOJqgXu2Yj/ZQ5KloJndHwxGI4jpyVN3HaC4ebw xzsrvvZpjw1RARCxYG23ULMoiDSkDeX8BJUP+YcIqf+ZRQq0/EmGFUu7NicBD7bz5qZficjs ESWpIVUb+FnfC8r/63ISuK4sHzMh4bisG2ykcbNx80h47bIxmdrUtVMwxGpS+2UbBwuBL2Zo ARoORNJdQc+3AuCXDyMvvBteDo2ZAgMBAAGjggLVMIIC0TAJBgNVHRMEAjAAMAsGA1UdDwQE AwIEsDAdBgNVHSUEFjAUBggrBgEFBQcDAgYIKwYBBQUHAwQwHQYDVR0OBBYEFLw3DlkKqBhh Cvig3OqKZ1dYrqSJMB8GA1UdIwQYMBaAFK5Vg2/sMcq59x36r2sx88gd46y7MBoGA1UdEQQT MBGBD3BldGVyQGtpZXNlci5jYTCCAUwGA1UdIASCAUMwggE/MIIBOwYLKwYBBAGBtTcBAgMw ggEqMC4GCCsGAQUFBwIBFiJodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS9wb2xpY3kucGRmMIH3 BggrBgEFBQcCAjCB6jAnFiBTdGFydENvbSBDZXJ0aWZpY2F0aW9uIEF1dGhvcml0eTADAgEB GoG+VGhpcyBjZXJ0aWZpY2F0ZSB3YXMgaXNzdWVkIGFjY29yZGluZyB0byB0aGUgQ2xhc3Mg MiBWYWxpZGF0aW9uIHJlcXVpcmVtZW50cyBvZiB0aGUgU3RhcnRDb20gQ0EgcG9saWN5LCBy ZWxpYW5jZSBvbmx5IGZvciB0aGUgaW50ZW5kZWQgcHVycG9zZSBpbiBjb21wbGlhbmNlIG9m IHRoZSByZWx5aW5nIHBhcnR5IG9ibGlnYXRpb25zLjA2BgNVHR8ELzAtMCugKaAnhiVodHRw Oi8vY3JsLnN0YXJ0c3NsLmNvbS9jcnR1Mi1jcmwuY3JsMIGOBggrBgEFBQcBAQSBgTB/MDkG CCsGAQUFBzABhi1odHRwOi8vb2NzcC5zdGFydHNzbC5jb20vc3ViL2NsYXNzMi9jbGllbnQv Y2EwQgYIKwYBBQUHMAKGNmh0dHA6Ly9haWEuc3RhcnRzc2wuY29tL2NlcnRzL3N1Yi5jbGFz czIuY2xpZW50LmNhLmNydDAjBgNVHRIEHDAahhhodHRwOi8vd3d3LnN0YXJ0c3NsLmNvbS8w DQYJKoZIhvcNAQEFBQADggEBAMnGxCRup78n279l+J6YKtXHlPstamJl3MwgTbzUSdsr1V3P JY40LCkQ86a/CYUwoOEjGYdwtwCDgRsa2ArMmsUS90rH4f+j7YNxMSH1TY+gqGwdaYk67rDk WlQ2EoxLWWUUeImLfZg0aP5mUYYeEETGp2WMAlzxuJq4LqkODS2FXYE7B2IAk0cpjG8q0NPn iBnQDv7M3Iq0aLwA8sa6DHJcPk05Qe50K6LyLE1bXhqntK3pykdG83VrHsUBJ7SqQcSS8vx0 0iB3kB1Oa+4cI5P79WqCpYprmim3RkyZddAWJPPqW1PCRhJdbJtXDZ+DCmMIZ1hUUwRDxIlz +lcjXD0xggPqMIID5gIBATCBkzCBjDELMAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29t IEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBEaWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxODA2 BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDIgUHJpbWFyeSBJbnRlcm1lZGlhdGUgQ2xpZW50IENB AgJRyzANBglghkgBZQMEAgEFAKCCAicwGAYJKoZIhvcNAQkDMQsGCSqGSIb3DQEHATAcBgkq hkiG9w0BCQUxDxcNMTUwODI5MjIyNjExWjAvBgkqhkiG9w0BCQQxIgQgtz48RphCh/UM2c1i joinGeV6z3jNTbawtJwyQ5IM6nQwbAYJKoZIhvcNAQkPMV8wXTALBglghkgBZQMEASowCwYJ YIZIAWUDBAECMAoGCCqGSIb3DQMHMA4GCCqGSIb3DQMCAgIAgDANBggqhkiG9w0DAgIBQDAH BgUrDgMCBzANBggqhkiG9w0DAgIBKDCBpAYJKwYBBAGCNxAEMYGWMIGTMIGMMQswCQYDVQQG EwJJTDEWMBQGA1UEChMNU3RhcnRDb20gTHRkLjErMCkGA1UECxMiU2VjdXJlIERpZ2l0YWwg Q2VydGlmaWNhdGUgU2lnbmluZzE4MDYGA1UEAxMvU3RhcnRDb20gQ2xhc3MgMiBQcmltYXJ5 IEludGVybWVkaWF0ZSBDbGllbnQgQ0ECAlHLMIGmBgsqhkiG9w0BCRACCzGBlqCBkzCBjDEL MAkGA1UEBhMCSUwxFjAUBgNVBAoTDVN0YXJ0Q29tIEx0ZC4xKzApBgNVBAsTIlNlY3VyZSBE aWdpdGFsIENlcnRpZmljYXRlIFNpZ25pbmcxODA2BgNVBAMTL1N0YXJ0Q29tIENsYXNzIDIg UHJpbWFyeSBJbnRlcm1lZGlhdGUgQ2xpZW50IENBAgJRyzANBgkqhkiG9w0BAQEFAASCAQAD PP5WiD1CovGYLQRrMDjLDkRXGZagRmKGo5sd8mmzICdi4OMr8xWATG3IfGWRVCrPeYIjFJyw tP8K2x/GM2a6hS3MqTlbcjf9/vOBsc+B5oA9YGqObaRb3A4EE+DXnpBs3dKfk+tDsjtDIq6w +U6r/afJnQplOEA1h9CpkQc3icXCEZSG2EWCSha0afk8WL6iuU7rMVnOlwKCHvXVA1mHzFFN zWZyStxJ3NeT+G128RAuP+ydRIIM/NodxcVQ7Kb1pU877q1SjpgYGlqozvNtaoQhxTsXjfIT n36MEMR/E136z/QLngHNp3TfPFeDi9HeUcLB6qQMHokeBkzzv+gTAAAAAAAA --------------ms060805030702080605010604-- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/