Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756108Ab2KNAUY (ORCPT ); Tue, 13 Nov 2012 19:20:24 -0500 Received: from fgwmail5.fujitsu.co.jp ([192.51.44.35]:56708 "EHLO fgwmail5.fujitsu.co.jp" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1756071Ab2KNAUX (ORCPT ); Tue, 13 Nov 2012 19:20:23 -0500 X-SecurityPolicyCheck: OK by SHieldMailChecker v1.8.4 Message-ID: <50A2E3B3.6080007@jp.fujitsu.com> Date: Wed, 14 Nov 2012 09:20:03 +0900 From: Kamezawa Hiroyuki User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:16.0) Gecko/20121026 Thunderbird/16.0.2 MIME-Version: 1.0 To: Michal Hocko CC: linux-mm@kvack.org, linux-kernel@vger.kernel.org, Johannes Weiner , Ying Han , Tejun Heo , Glauber Costa Subject: Re: [RFC 2/5] memcg: rework mem_cgroup_iter to use cgroup iterators References: <1352820639-13521-1-git-send-email-mhocko@suse.cz> <1352820639-13521-3-git-send-email-mhocko@suse.cz> In-Reply-To: <1352820639-13521-3-git-send-email-mhocko@suse.cz> Content-Type: text/plain; charset=ISO-2022-JP Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6056 Lines: 173 (2012/11/14 0:30), Michal Hocko wrote: > mem_cgroup_iter curently relies on css->id when walking down a group > hierarchy tree. This is really awkward because the tree walk depends on > the groups creation ordering. The only guarantee is that a parent node > is visited before its children. > Example > 1) mkdir -p a a/d a/b/c > 2) mkdir -a a/b/c a/d > Will create the same trees but the tree walks will be different: > 1) a, d, b, c > 2) a, b, c, d > > 574bd9f7 (cgroup: implement generic child / descendant walk macros) has > introduced generic cgroup tree walkers which provide either pre-order > or post-order tree walk. This patch converts css->id based iteration > to pre-order tree walk to keep the semantic with the original iterator > where parent is always visited before its subtree. > > cgroup_for_each_descendant_pre suggests using post_create and > pre_destroy for proper synchronization with groups addidition resp. > removal. This implementation doesn't use those because a new memory > cgroup is fully initialized in mem_cgroup_create and css_tryget makes > sure that the group is alive when we encounter it by iterator. > > If the reclaim cookie is used we need to store the last visited group > into the iterator so we have to be careful that it doesn't disappear in > the mean time. Elevated reference count on the memcg guarantees that > the group will not vanish even though it has been already removed from > the tree. In such a case css_tryget will fail and the iteration is > retried (groups are linked with RCU safe lists so the forward progress > is still possible). iter_lock will make sure that only one reclaimer > will see the last_visited group and the reference count game around it. > > Signed-off-by: Michal Hocko > --- > mm/memcontrol.c | 64 ++++++++++++++++++++++++++++++++++++++++++------------- > 1 file changed, 49 insertions(+), 15 deletions(-) > > diff --git a/mm/memcontrol.c b/mm/memcontrol.c > index 0fe5177..5da1e58 100644 > --- a/mm/memcontrol.c > +++ b/mm/memcontrol.c > @@ -142,8 +142,8 @@ struct mem_cgroup_stat_cpu { > }; > > struct mem_cgroup_reclaim_iter { > - /* css_id of the last scanned hierarchy member */ > - int position; > + /* last scanned hierarchy member with elevated ref count */ > + struct mem_cgroup *last_visited; > /* scan generation, increased every round-trip */ > unsigned int generation; > /* lock to protect the position and generation */ > @@ -1063,8 +1063,8 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, > struct mem_cgroup *prev, > struct mem_cgroup_reclaim_cookie *reclaim) > { > - struct mem_cgroup *memcg = NULL; > - int id = 0; > + struct mem_cgroup *memcg = NULL, > + *last_visited = NULL; > > if (mem_cgroup_disabled()) > return NULL; > @@ -1073,7 +1073,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, > root = root_mem_cgroup; > > if (prev && !reclaim) > - id = css_id(&prev->css); > + last_visited = prev; > > if (prev && prev != root) > css_put(&prev->css); > @@ -1086,7 +1086,7 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, > > while (!memcg) { > struct mem_cgroup_reclaim_iter *uninitialized_var(iter); > - struct cgroup_subsys_state *css; > + struct cgroup_subsys_state *css = NULL; > > if (reclaim) { > int nid = zone_to_nid(reclaim->zone); > @@ -1096,30 +1096,64 @@ struct mem_cgroup *mem_cgroup_iter(struct mem_cgroup *root, > mz = mem_cgroup_zoneinfo(root, nid, zid); > iter = &mz->reclaim_iter[reclaim->priority]; > spin_lock(&iter->iter_lock); > + last_visited = iter->last_visited; > if (prev && reclaim->generation != iter->generation) { > + if (last_visited) { > + mem_cgroup_put(last_visited); > + iter->last_visited = NULL; > + } > spin_unlock(&iter->iter_lock); > return NULL; > } > - id = iter->position; > } > > rcu_read_lock(); > - css = css_get_next(&mem_cgroup_subsys, id + 1, &root->css, &id); > - if (css) { > - if (css == &root->css || css_tryget(css)) > - memcg = mem_cgroup_from_css(css); > - } else > - id = 0; > - rcu_read_unlock(); > + /* > + * Root is not visited by cgroup iterators so it needs a special > + * treatment. > + */ > + if (!last_visited) { > + css = &root->css; > + } else { > + struct cgroup *next_cgroup; > + > + next_cgroup = cgroup_next_descendant_pre( > + last_visited->css.cgroup, > + root->css.cgroup); Maybe I miss something but.... last_visited is holded by memcg's refcnt. The cgroup pointed by css.cgroup is by cgroup's refcnt which can be freed before memcg is freed and last_visited->css.cgroup is out of RCU cycle. Is this safe ? Thanks, -Kame > + if (next_cgroup) > + css = cgroup_subsys_state(next_cgroup, > + mem_cgroup_subsys_id); > + } > + > + /* > + * Even if we find a group we have to make sure it is alive. > + * css && !memcg means that the groups should be skipped and > + * we should continue the tree walk. > + */ > + if (css == &root->css || (css && css_tryget(css))) > + memcg = mem_cgroup_from_css(css); > > if (reclaim) { > - iter->position = id; > + struct mem_cgroup *curr = memcg; > + > + if (last_visited) > + mem_cgroup_put(last_visited); > + > + if (css && !memcg) > + curr = mem_cgroup_from_css(css); > + if (curr) > + mem_cgroup_get(curr); > + iter->last_visited = curr; > + > if (!css) > iter->generation++; > else if (!prev && memcg) > reclaim->generation = iter->generation; > spin_unlock(&iter->iter_lock); > + } else if (css && !memcg) { > + last_visited = mem_cgroup_from_css(css); > } > + rcu_read_unlock(); > > if (prev && !css) > return NULL; > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/