Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1755363AbbHXREe (ORCPT ); Mon, 24 Aug 2015 13:04:34 -0400 Received: from mail-qk0-f180.google.com ([209.85.220.180]:36433 "EHLO mail-qk0-f180.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752983AbbHXREb (ORCPT ); Mon, 24 Aug 2015 13:04:31 -0400 Date: Mon, 24 Aug 2015 13:04:27 -0400 From: Tejun Heo To: Austin S Hemmelgarn Cc: Paul Turner , Peter Zijlstra , Ingo Molnar , Johannes Weiner , lizefan@huawei.com, cgroups , LKML , kernel-team , Linus Torvalds , Andrew Morton Subject: Re: [PATCH 3/3] sched: Implement interface for cgroup unified hierarchy Message-ID: <20150824170427.GA27262@mtj.duckdns.org> References: <20150804090711.GL25159@twins.programming.kicks-ass.net> <20150804151017.GD17598@mtj.duckdns.org> <20150805091036.GT25159@twins.programming.kicks-ass.net> <20150805143132.GK17598@mtj.duckdns.org> <20150818203117.GC15739@mtj.duckdns.org> <20150822182916.GE20768@mtj.duckdns.org> <55DB3C76.5010009@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <55DB3C76.5010009@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2377 Lines: 51 Hello, Austin. On Mon, Aug 24, 2015 at 11:47:02AM -0400, Austin S Hemmelgarn wrote: > >Just to learn more, what sort of hypervisor support threads are we > >talking about? They would have to consume considerable amount of cpu > >cycles for problems like this to be relevant and be dynamic in numbers > >in a way which letting them competing against vcpus makes sense. Do > >IO helpers meet these criteria? > > > Depending on the configuration, yes they can. VirtualBox has some rather > CPU intensive threads that aren't vCPU threads (their emulated APIC thread > immediately comes to mind), and so does QEMU depending on the emulated And the number of those threads fluctuate widely and dynamically? > hardware configuration (it gets more noticeable when the disk images are > stored on a SAN and served through iSCSI, NBD, FCoE, or ATAoE, which is > pretty typical usage for large virtualization deployments). I've seen cases > first hand where the vCPU's can make no reasonable progress because they are > constantly getting crowded out by other threads. That alone doesn't require hierarchical resource distribution tho. Setting nice levels reasonably is likely to alleviate most of the problem. > The use of the term 'hypervisor support threads' for this is probably not > the best way of describing the contention, as it's almost always a full > system virtualization issue, and the contending threads are usually storage > back-end access threads. > > I would argue that there are better ways to deal properly with this (Isolate > the non vCPU threads on separate physical CPU's from the hardware emulation > threads), but such methods require large systems to be practical at any > scale, and many people don't have the budget for such large systems, and > this way of doing things is much more flexible for small scale use cases > (for example, someone running one or two VM's on a laptop under QEMU or > VirtualBox). I don't know. "Someone running one or two VM's on a laptop under QEMU" doesn't really sound like the use case which absolutely requires hierarchical cpu cycle distribution. Thanks. -- tejun -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/