Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1758236Ab1EMIuE (ORCPT ); Fri, 13 May 2011 04:50:04 -0400 Received: from smtp-out.google.com ([74.125.121.67]:37952 "EHLO smtp-out.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1758132Ab1EMIt7 (ORCPT ); Fri, 13 May 2011 04:49:59 -0400 DomainKey-Signature: a=rsa-sha1; s=beta; d=google.com; c=nofws; q=dns; h=from:to:cc:subject:date:message-id:x-mailer:in-reply-to:references; b=pJ7rywwjL7iNx1Zii3/VsSdjN/DUsCI9rx9sWCIBAcUm+rJrURAhVaKJW/HgNo4s8 SW1vF3ntegrUx26Ck3UWw== From: Greg Thelen To: Andrew Morton Cc: linux-kernel@vger.kernel.org, linux-mm@kvack.org, containers@lists.osdl.org, linux-fsdevel@vger.kernel.org, Andrea Righi , Balbir Singh , KAMEZAWA Hiroyuki , Daisuke Nishimura , Minchan Kim , Johannes Weiner , Ciju Rajan K , David Rientjes , Wu Fengguang , Vivek Goyal , Dave Chinner , Greg Thelen Subject: [RFC][PATCH v7 01/14] memcg: document cgroup dirty memory interfaces Date: Fri, 13 May 2011 01:47:40 -0700 Message-Id: <1305276473-14780-2-git-send-email-gthelen@google.com> X-Mailer: git-send-email 1.7.3.1 In-Reply-To: <1305276473-14780-1-git-send-email-gthelen@google.com> References: <1305276473-14780-1-git-send-email-gthelen@google.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6807 Lines: 135 Document cgroup dirty memory interfaces and statistics. The implementation for these new interfaces routines comes in a series of following patches. Signed-off-by: Andrea Righi Signed-off-by: Greg Thelen Acked-by: KAMEZAWA Hiroyuki Acked-by: Balbir Singh --- Changelog since v6: - Removed 'Inode writeback issue' now that memcg-writeback is implemented in the series. - Trivial reword of section 5.6 "dirty memory". Changelog since v4: - Minor rewording of '5.5 dirty memory' section. - Added '5.5.1 Inode writeback issue' section. Changelog since v3: - Described interactions with memory.use_hierarchy. - Added description of total_dirty, total_writeback, and total_nfs_unstable. Changelog since v1: - Renamed "nfs"/"total_nfs" to "nfs_unstable"/"total_nfs_unstable" in per cgroup memory.stat to match /proc/meminfo. - Allow [kKmMgG] suffixes for newly created dirty limit value cgroupfs files. - Describe a situation where a cgroup can exceed its dirty limit. Documentation/cgroups/memory.txt | 70 ++++++++++++++++++++++++++++++++++++++ 1 files changed, 70 insertions(+), 0 deletions(-) diff --git a/Documentation/cgroups/memory.txt b/Documentation/cgroups/memory.txt index 43b9e46..15019a3 100644 --- a/Documentation/cgroups/memory.txt +++ b/Documentation/cgroups/memory.txt @@ -395,6 +395,10 @@ soft_direct_steal- # of pages reclaimed in global hierarchical reclaim from direct reclaim soft_direct_scan- # of pages scanned in global hierarchical reclaim from direct reclaim +dirty - # of bytes that are waiting to get written back to the disk. +writeback - # of bytes that are actively being written back to the disk. +nfs_unstable - # of bytes sent to the NFS server, but not yet committed to + the actual storage. inactive_anon - # of bytes of anonymous memory and swap cache memory on LRU list. active_anon - # of bytes of anonymous and swap cache memory on active @@ -420,6 +424,9 @@ total_soft_kswapd_steal - sum of all children's "soft_kswapd_steal" total_soft_kswapd_scan - sum of all children's "soft_kswapd_scan" total_soft_direct_steal - sum of all children's "soft_direct_steal" total_soft_direct_scan - sum of all children's "soft_direct_scan" +total_dirty - sum of all children's "dirty" +total_writeback - sum of all children's "writeback" +total_nfs_unstable - sum of all children's "nfs_unstable" total_inactive_anon - sum of all children's "inactive_anon" total_active_anon - sum of all children's "active_anon" total_inactive_file - sum of all children's "inactive_file" @@ -476,6 +483,69 @@ value for efficient access. (Of course, when necessary, it's synchronized.) If you want to know more exact memory usage, you should use RSS+CACHE(+SWAP) value in memory.stat(see 5.2). +5.6 dirty memory + +Control the maximum amount of dirty pages a cgroup can have at any given time. + +Limiting dirty memory is like fixing the max amount of dirty (hard to reclaim) +page cache used by a cgroup. So, in case of multiple cgroup writers, they will +not be able to consume more than their designated share of dirty pages and will +be throttled if they cross that limit. System-wide dirty limits are also +consulted. Dirty memory consumption is checked against both system-wide and +per-cgroup dirty limits. + +The interface is similar to the procfs interface: /proc/sys/vm/dirty_*. It is +possible to configure a limit to trigger throttling of a dirtier or queue +background writeback. The root cgroup memory.dirty_* control files are +read-only and match the contents of the /proc/sys/vm/dirty_* files. + +Per-cgroup dirty limits can be set using the following files in the cgroupfs: + +- memory.dirty_ratio: the amount of dirty memory (expressed as a percentage of + cgroup memory) at which a process generating dirty pages will be throttled. + The default value is the system-wide dirty ratio, /proc/sys/vm/dirty_ratio. + +- memory.dirty_limit_in_bytes: the amount of dirty memory (expressed in bytes) + in the cgroup at which a process generating dirty pages will be throttled. + Suffix (k, K, m, M, g, or G) can be used to indicate that value is kilo, mega + or gigabytes. The default value is the system-wide dirty limit, + /proc/sys/vm/dirty_bytes. + + Note: memory.dirty_limit_in_bytes is the counterpart of memory.dirty_ratio. + Only one may be specified at a time. When one is written it is immediately + taken into account to evaluate the dirty memory limits and the other appears + as 0 when read. + +- memory.dirty_background_ratio: the amount of dirty memory of the cgroup + (expressed as a percentage of cgroup memory) at which background writeback + kernel threads will start writing out dirty data. The default value is the + system-wide background dirty ratio, /proc/sys/vm/dirty_background_ratio. + +- memory.dirty_background_limit_in_bytes: the amount of dirty memory (expressed + in bytes) in the cgroup at which background writeback kernel threads will + start writing out dirty data. Suffix (k, K, m, M, g, or G) can be used to + indicate that value is kilo, mega or gigabytes. The default value is the + system-wide dirty background limit, /proc/sys/vm/dirty_background_bytes. + + Note: memory.dirty_background_limit_in_bytes is the counterpart of + memory.dirty_background_ratio. Only one may be specified at a time. When one + is written it is immediately taken into account to evaluate the dirty memory + limits and the other appears as 0 when read. + +A cgroup may contain more dirty memory than its dirty limit. This is possible +because of the principle that the first cgroup to touch a page is charged for +it. Subsequent page counting events (dirty, writeback, nfs_unstable) are also +counted to the originally charged cgroup. Example: If page is allocated by a +cgroup A task, then the page is charged to cgroup A. If the page is later +dirtied by a task in cgroup B, then the cgroup A dirty count will be +incremented. If cgroup A is over its dirty limit but cgroup B is not, then +dirtying a cgroup A page from a cgroup B task may push cgroup A over its dirty +limit without throttling the dirtying cgroup B task. + +When use_hierarchy=0, each cgroup has independent dirty memory usage and limits. +When use_hierarchy=1 the dirty limits of parent cgroups are also checked to +ensure that no dirty limit is exceeded. + 6. Hierarchy support The memory controller supports a deep hierarchy and hierarchical accounting. -- 1.7.3.1 -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/