Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1754311Ab0BVSMj (ORCPT ); Mon, 22 Feb 2010 13:12:39 -0500 Received: from trinity.develer.com ([83.149.158.210]:37612 "EHLO trinity.develer.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754075Ab0BVSMg (ORCPT ); Mon, 22 Feb 2010 13:12:36 -0500 Date: Mon, 22 Feb 2010 19:12:27 +0100 From: Andrea Righi To: Vivek Goyal Cc: Balbir Singh , KAMEZAWA Hiroyuki , Suleiman Souhlal , Andrew Morton , containers@lists.linux-foundation.org, linux-kernel@vger.kernel.org Subject: Re: [RFC] [PATCH 0/2] memcg: per cgroup dirty limit Message-ID: <20100222181226.GC4052@linux> References: <1266765525-30890-1-git-send-email-arighi@develer.com> <20100222142744.GB13823@redhat.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100222142744.GB13823@redhat.com> User-Agent: Mutt/1.5.20 (2009-06-14) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3625 Lines: 75 On Mon, Feb 22, 2010 at 09:27:45AM -0500, Vivek Goyal wrote: > On Sun, Feb 21, 2010 at 04:18:43PM +0100, Andrea Righi wrote: > > Control the maximum amount of dirty pages a cgroup can have at any given time. > > > > Per cgroup dirty limit is like fixing the max amount of dirty (hard to reclaim) > > page cache used by any cgroup. So, in case of multiple cgroup writers, they > > will not be able to consume more than their designated share of dirty pages and > > will be forced to perform write-out if they cross that limit. > > > > The overall design is the following: > > > > - account dirty pages per cgroup > > - limit the number of dirty pages via memory.dirty_bytes in cgroupfs > > - start to write-out in balance_dirty_pages() when the cgroup or global limit > > is exceeded > > > > This feature is supposed to be strictly connected to any underlying IO > > controller implementation, so we can stop increasing dirty pages in VM layer > > and enforce a write-out before any cgroup will consume the global amount of > > dirty pages defined by the /proc/sys/vm/dirty_ratio|dirty_bytes limit. > > > > Thanks Andrea. I had been thinking about looking into it from IO > controller perspective so that we can control async IO (buffered writes > also). > > Before I dive into patches, two quick things. > > - IIRC, last time you had implemented per memory cgroup "dirty_ratio" and > not "dirty_bytes". Why this change? To begin with either per memcg > configurable dirty ratio also makes sense? By default it can be the > global dirty ratio for each cgroup. I would't like to add many different interfaces to do the same thing. I'd prefer to choose just one interface and always use it. We just have to define which is the best one. IMHO dirty_bytes is more generic. If we want to define the limit as a % we can always do that in userspace. > > - Looks like we will start writeout from memory cgroup once we cross the > dirty ratio, but still there is no gurantee that we be writting pages > belonging to cgroup which crossed the dirty ratio and triggered the > writeout. > > This behavior is not very good at least from IO controller perspective > where if two dd threads are dirtying memory in two cgroups, then if > one crosses it dirty ratio, it should perform writeouts of its own pages > and not other cgroups pages. Otherwise we probably will again introduce > serialization among two writers and will not see service differentation. Right, but I'd prefer to start with a simple solution. Handling this per-page is too costly and not good for entire I/O for now. We can always improve service differentiation and fairness in a second step, I think. > > May be we can modify writeback_inodes_wbc() to check first dirty page > of the inode. And if it does not belong to same memcg as the task who > is performing balance_dirty_pages(), then skip that inode. > > This does not handle the problem of shared files where processes from > two different cgroups are dirtying same file but it will atleast cover > other cases without introducing too much of complexity? Yes, if we want to take care ot that, at least we could start with a per-inode solution. It will probably introduce less overhead and will work the the most part of the cases (except the case when multiple cgroups write to the same file/inode). -Andrea -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/