Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755014AbbHXPrK (ORCPT ); Mon, 24 Aug 2015 11:47:10 -0400 Received: from mail-io0-f182.google.com ([209.85.223.182]:34736 "EHLO mail-io0-f182.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754989AbbHXPrJ (ORCPT ); Mon, 24 Aug 2015 11:47:09 -0400 Subject: Re: [PATCH 3/3] sched: Implement interface for cgroup unified hierarchy To: Tejun Heo , Paul Turner References: <1438641689-14655-1-git-send-email-tj@kernel.org> <1438641689-14655-4-git-send-email-tj@kernel.org> <20150804090711.GL25159@twins.programming.kicks-ass.net> <20150804151017.GD17598@mtj.duckdns.org> <20150805091036.GT25159@twins.programming.kicks-ass.net> <20150805143132.GK17598@mtj.duckdns.org> <20150818203117.GC15739@mtj.duckdns.org> <20150822182916.GE20768@mtj.duckdns.org> Cc: Peter Zijlstra , Ingo Molnar , Johannes Weiner , lizefan@huawei.com, cgroups , LKML , kernel-team , Linus Torvalds , Andrew Morton From: Austin S Hemmelgarn Message-ID: <55DB3C76.5010009@gmail.com> Date: Mon, 24 Aug 2015 11:47:02 -0400 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.2.0 MIME-Version: 1.0 In-Reply-To: <20150822182916.GE20768@mtj.duckdns.org> Content-Type: multipart/signed; protocol="application/pkcs7-signature"; micalg=sha-512; boundary="------------ms020402090409010007050801" X-Antivirus: avast! (VPS 150824-0, 2015-08-24), Outbound message X-Antivirus-Status: Clean Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6861 Lines: 116 This is a cryptographically signed message in MIME format. --------------ms020402090409010007050801 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: quoted-printable On 2015-08-22 14:29, Tejun Heo wrote: > Hello, Paul. > > On Fri, Aug 21, 2015 at 12:26:30PM -0700, Paul Turner wrote: > ... >> A very concrete example of the above is a virtual machine in which you= >> want to guarantee scheduling for the vCPU threads which must schedule >> beside many hypervisor support threads. A hierarchy is the only way >> to fix the ratio at which these compete. > > Just to learn more, what sort of hypervisor support threads are we > talking about? They would have to consume considerable amount of cpu > cycles for problems like this to be relevant and be dynamic in numbers > in a way which letting them competing against vcpus makes sense. Do > IO helpers meet these criteria? > Depending on the configuration, yes they can. VirtualBox has some=20 rather CPU intensive threads that aren't vCPU threads (their emulated=20 APIC thread immediately comes to mind), and so does QEMU depending on=20 the emulated hardware configuration (it gets more noticeable when the=20 disk images are stored on a SAN and served through iSCSI, NBD, FCoE, or=20 ATAoE, which is pretty typical usage for large virtualization=20 deployments). I've seen cases first hand where the vCPU's can make no=20 reasonable progress because they are constantly getting crowded out by=20 other threads. The use of the term 'hypervisor support threads' for this is probably=20 not the best way of describing the contention, as it's almost always a=20 full system virtualization issue, and the contending threads are usually = storage back-end access threads. I would argue that there are better ways to deal properly with this=20 (Isolate the non vCPU threads on separate physical CPU's from the=20 hardware emulation threads), but such methods require large systems to=20 be practical at any scale, and many people don't have the budget for=20 such large systems, and this way of doing things is much more flexible=20 for small scale use cases (for example, someone running one or two VM's=20 on a laptop under QEMU or VirtualBox). --------------ms020402090409010007050801 Content-Type: application/pkcs7-signature; name="smime.p7s" Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename="smime.p7s" Content-Description: S/MIME Cryptographic Signature MIAGCSqGSIb3DQEHAqCAMIACAQExDzANBglghkgBZQMEAgMFADCABgkqhkiG9w0BBwEAAKCC Brgwgga0MIIEnKADAgECAgMQblUwDQYJKoZIhvcNAQENBQAweTEQMA4GA1UEChMHUm9vdCBD QTEeMBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0IFNp Z25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2VydC5vcmcwHhcN MTUwMzI1MTkzNDM4WhcNMTUwOTIxMTkzNDM4WjBjMRgwFgYDVQQDEw9DQWNlcnQgV29UIFVz ZXIxIzAhBgkqhkiG9w0BCQEWFGFoZmVycm9pbjdAZ21haWwuY29tMSIwIAYJKoZIhvcNAQkB FhNhaGVtbWVsZ0BvaGlvZ3QuY29tMIICIjANBgkqhkiG9w0BAQEFAAOCAg8AMIICCgKCAgEA nQ/81tq0QBQi5w316VsVNfjg6kVVIMx760TuwA1MUaNQgQ3NyUl+UyFtjhpkNwwChjgAqfGd LIMTHAdObcwGfzO5uI2o1a8MHVQna8FRsU3QGouysIOGQlX8jFYXMKPEdnlt0GoQcd+BtESr pivbGWUEkPs1CwM6WOrs+09bAJP3qzKIr0VxervFrzrC5Dg9Rf18r9WXHElBuWHg4GYHNJ2V Ab8iKc10h44FnqxZK8RDN8ts/xX93i9bIBmHnFfyNRfiOUtNVeynJbf6kVtdHP+CRBkXCNRZ qyQT7gbTGD24P92PS2UTmDfplSBcWcTn65o3xWfesbf02jF6PL3BCrVnDRI4RgYxG3zFBJuG qvMoEODLhHKSXPAyQhwZINigZNdw5G1NqjXqUw+lIqdQvoPijK9J3eijiakh9u2bjWOMaleI SMRR6XsdM2O5qun1dqOrCgRkM0XSNtBQ2JjY7CycIx+qifJWsRaYWZz0aQU4ZrtAI7gVhO9h pyNaAGjvm7PdjEBiXq57e4QcgpwzvNlv8pG1c/hnt0msfDWNJtl3b6elhQ2Pz4w/QnWifZ8E BrFEmjeeJa2dqjE3giPVWrsH+lOvQQONsYJOuVb8b0zao4vrWeGmW2q2e3pdv0Axzm/60cJQ haZUv8+JdX9ZzqxOm5w5eUQSclt84u+D+hsCAwEAAaOCAVkwggFVMAwGA1UdEwEB/wQCMAAw VgYJYIZIAYb4QgENBEkWR1RvIGdldCB5b3VyIG93biBjZXJ0aWZpY2F0ZSBmb3IgRlJFRSBo ZWFkIG92ZXIgdG8gaHR0cDovL3d3dy5DQWNlcnQub3JnMA4GA1UdDwEB/wQEAwIDqDBABgNV HSUEOTA3BggrBgEFBQcDBAYIKwYBBQUHAwIGCisGAQQBgjcKAwQGCisGAQQBgjcKAwMGCWCG SAGG+EIEATAyBggrBgEFBQcBAQQmMCQwIgYIKwYBBQUHMAGGFmh0dHA6Ly9vY3NwLmNhY2Vy dC5vcmcwMQYDVR0fBCowKDAmoCSgIoYgaHR0cDovL2NybC5jYWNlcnQub3JnL3Jldm9rZS5j cmwwNAYDVR0RBC0wK4EUYWhmZXJyb2luN0BnbWFpbC5jb22BE2FoZW1tZWxnQG9oaW9ndC5j b20wDQYJKoZIhvcNAQENBQADggIBABr5e8W+NiTER+Q/7wiA2LxWN3UdhT3eZJjqqSlP370P KL5iWqeTfxQ67Ai/mHbJcT2PgAJ+/D2Ji+aRR03UWnU/vtOwzyDLUMstqnfl0Zs+sz/CJe7x nBA5jlpjC2DKuMVfbPze7eySaen7XSGFHKE1QoVIIpQ2kVjC4nbbJQnUbAVX1Iz29WxeVGt9 XYigz3tDPf3tglN+q23E7YjQl4abTIoM7i98yV1H9gfY8lFfKZ6jREB9+n6ie2EwS3Kat2mG tl2wBx4MfRnoSQSKsLKQ5oTwhWf0JqlFwpLfl374p0Njcykej9/jnWG8Ks1V/AXTHqI4eyIP Mf5yMZkPv7n7LS9WWKdG4Nd38iv4T2EiAaWsmgu+r81qL5CJu9AyA0SBS4ttKf6k3e63w2Mv N9R45vpQ3QhAhfWyFxFhZN95APe3YECDG3+XIRJpRYPEtHuIsOyzI70ajF93gg/BidvqKsmV MM2ccktDMfqwZXea6zey7F8Geu9R7BqjXmG2HlNuXu7e/xnHOgXf5D3wPmnRLlBhXL1Ch97a w2KjaupjpAHfFjv5kGnZXN87UvvlwzIZiKXwa3vTDwK+rrKn/sHPkfDZPSiyt/ZBIK6lX83P 34H/CzGg+Kx57rHYOIHGumIvpDa5vfWp8O0sGgawb1C2Aae4sTUVIWmIjVuGI062MYIE0TCC BM0CAQEwgYAweTEQMA4GA1UEChMHUm9vdCBDQTEeMBwGA1UECxMVaHR0cDovL3d3dy5jYWNl cnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0IFNpZ25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcN AQkBFhJzdXBwb3J0QGNhY2VydC5vcmcCAxBuVTANBglghkgBZQMEAgMFAKCCAiEwGAYJKoZI hvcNAQkDMQsGCSqGSIb3DQEHATAcBgkqhkiG9w0BCQUxDxcNMTUwODI0MTU0NzAyWjBPBgkq hkiG9w0BCQQxQgRAB8ppBwNqD55wrvs+F1aV23SQsVtz2beZ0iD5dT55zrMzixSdqjmFDl4h G58M2rMKrVpjSC2xZiHrWA4KffRiBjBsBgkqhkiG9w0BCQ8xXzBdMAsGCWCGSAFlAwQBKjAL BglghkgBZQMEAQIwCgYIKoZIhvcNAwcwDgYIKoZIhvcNAwICAgCAMA0GCCqGSIb3DQMCAgFA MAcGBSsOAwIHMA0GCCqGSIb3DQMCAgEoMIGRBgkrBgEEAYI3EAQxgYMwgYAweTEQMA4GA1UE ChMHUm9vdCBDQTEeMBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlD QSBDZXJ0IFNpZ25pbmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2Vy dC5vcmcCAxBuVTCBkwYLKoZIhvcNAQkQAgsxgYOggYAweTEQMA4GA1UEChMHUm9vdCBDQTEe MBwGA1UECxMVaHR0cDovL3d3dy5jYWNlcnQub3JnMSIwIAYDVQQDExlDQSBDZXJ0IFNpZ25p bmcgQXV0aG9yaXR5MSEwHwYJKoZIhvcNAQkBFhJzdXBwb3J0QGNhY2VydC5vcmcCAxBuVTAN BgkqhkiG9w0BAQEFAASCAgBSzT2svp/Sm0G21sUU7Ioezq/y4k3hGZxRnLspiLgeFKo01pLv cZTDGzljkpsMovdmsachHdMGsFhXhSE30X9x/vbbNYLiSyG015Cg8PdC5Ze3qr1sM1PgySDC KKl88z2PjHHr2s6ccTfSewEpsxbNu8GyeXLrUzT5LlS2Q25ZHMHHnUFGC5wwiyDsJP6zfpDj iW0Jza69UlI92gF+7nAWNBrhkqqbnrttSyU2Ucu5gOL9oXmeZATFKmMiE5OnzBjyHZ9lWCBK YUJfKp6U7EHpE3ir9SJy1nFFZojyvA8XPZVZbETGBs3KnmtU+8xGJpRfo6NRRIWuEWQOl3Ke UH5qq2el06oHLpdWa4gEqr0hrLTXuMYziz+zwalbgDE+s8OPAmi2gfGqN48OWFMtQNT+tXfa 9/QnyxTfZVz75jd9uhHsHWCk6dne4/3g6q9d0A8GiLUqWAlr4hC/0DA7EDTmgOvIfM+4UI9a Y5ZbM/uqVqhEFrC+SM7iSomnEM7ZGo7hE9k3jDCfGQW4uHICJmcHbZk1xS3aKe6J5NY7zjW/ cAs5Yxw+LKNT5s92dwFL5J+mYdkryJkzytAUuJT0wtdfTeXJKVTuyL+v34Ry1okL8rpmFmvp XKaTdPzzxdbQK42neqWRA9l1qEgtPVh19g+R5utyIKqYXaO9BsA6MOxA7gAAAAAAAA== --------------ms020402090409010007050801-- -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/