Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756535AbdCGWmJ (ORCPT ); Tue, 7 Mar 2017 17:42:09 -0500 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:38491 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1756189AbdCGWmH (ORCPT ); Tue, 7 Mar 2017 17:42:07 -0500 Subject: [PATCH v8 6/6] perf tools: Add cgroup identifier sort order keyword From: Hari Bathini To: ast@fb.com, peterz@infradead.org, lkml , acme@kernel.org, alexander.shishkin@linux.intel.com, mingo@redhat.com Cc: daniel@iogearbox.net, rostedt@goodmis.org, Ananth N Mavinakayanahalli , ebiederm@xmission.com, sargun@sargun.me, Aravinda Prasad , brendan.d.gregg@gmail.com, jolsa@redhat.com Date: Wed, 08 Mar 2017 02:12:13 +0530 In-Reply-To: <148891921533.25309.8328657213311313206.stgit@hbathini.in.ibm.com> References: <148891921533.25309.8328657213311313206.stgit@hbathini.in.ibm.com> User-Agent: StGit/0.17.1-dirty MIME-Version: 1.0 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: 7bit X-TM-AS-MML: disable x-cbid: 17030720-0008-0000-0000-00000539A4A6 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 17030720-0009-0000-0000-0000134420E0 Message-Id: <148891933338.25309.756882900782042645.stgit@hbathini.in.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10432:,, definitions=2017-03-07_16:,, signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 spamscore=0 suspectscore=0 malwarescore=0 phishscore=0 adultscore=0 bulkscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1702020001 definitions=main-1703070162 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 6882 Lines: 208 This patch introduces a cgroup identifier entry field in perf report to identify or distinguish data of different cgroups. It uses the device number and inode number of cgroup namespace, included in perf data with the new PERF_RECORD_NAMESPACES event, as cgroup identifier. With the assumption that each container is created with it's own cgroup namespace, this allows assessment/analysis of multiple containers at once. A simple test for this would be to clone a few processes passing SIGCHILD & CLONE_NEWCROUP flags to each of them, execute shell and run different workloads on each of those contexts, while running perf record command with --namespaces option. Shown below is the output of perf report, sorted with cgroup identifier, on perf.data generated with the above test scenario, clearly indicating one context's considerable use of kernel memory in comparison with others: $ perf report -s cgroup_id,sample --stdio # # Total Lost Samples: 0 # # Samples: 5K of event 'kmem:kmalloc' # Event count (approx.): 5965 # # Overhead cgroup id (dev/inode) Samples # ........ ..................... ............ # 81.27% 3/0xeffffffb 4848 16.24% 3/0xf00000d0 969 1.16% 3/0xf00000ce 69 0.82% 3/0xf00000cf 49 0.50% 0/0x0 30 While this is a start, there is further scope of improving this. For example, instead of cgroup namespace's device and inode numbers, dev and inode numbers of some or all namespaces may be used to distinguish which processes are running in a given container context. Also, scripts to map device and inode info to containers sounds plausible for better tracing of containers. Signed-off-by: Hari Bathini --- tools/perf/Documentation/perf-report.txt | 4 ++- tools/perf/util/hist.c | 7 +++++ tools/perf/util/hist.h | 1 + tools/perf/util/sort.c | 41 ++++++++++++++++++++++++++++++ tools/perf/util/sort.h | 7 +++++ 5 files changed, 59 insertions(+), 1 deletion(-) diff --git a/tools/perf/Documentation/perf-report.txt b/tools/perf/Documentation/perf-report.txt index 33f9190..9eaea2e 100644 --- a/tools/perf/Documentation/perf-report.txt +++ b/tools/perf/Documentation/perf-report.txt @@ -72,7 +72,8 @@ OPTIONS --sort=:: Sort histogram entries by given key(s) - multiple keys can be specified in CSV format. Following sort keys are available: - pid, comm, dso, symbol, parent, cpu, socket, srcline, weight, local_weight. + pid, comm, dso, symbol, parent, cpu, socket, srcline, weight, + local_weight, cgroup_id. Each key has following meaning: @@ -92,6 +93,7 @@ OPTIONS - weight: Event specific weight, e.g. memory latency or transaction abort cost. This is the global weight. - local_weight: Local weight version of the weight above. + - cgroup_id: ID derived from cgroup namespace device and inode numbers. - transaction: Transaction abort flags. - overhead: Overhead percentage of sample - overhead_sys: Overhead percentage of sample running in system mode diff --git a/tools/perf/util/hist.c b/tools/perf/util/hist.c index eaf72a9..e3b38f6 100644 --- a/tools/perf/util/hist.c +++ b/tools/perf/util/hist.c @@ -3,6 +3,7 @@ #include "hist.h" #include "map.h" #include "session.h" +#include "namespaces.h" #include "sort.h" #include "evlist.h" #include "evsel.h" @@ -169,6 +170,7 @@ void hists__calc_col_len(struct hists *hists, struct hist_entry *h) hists__set_unres_dso_col_len(hists, HISTC_MEM_DADDR_DSO); } + hists__new_col_len(hists, HISTC_CGROUP_ID, 20); hists__new_col_len(hists, HISTC_CPU, 3); hists__new_col_len(hists, HISTC_SOCKET, 6); hists__new_col_len(hists, HISTC_MEM_LOCKED, 6); @@ -574,9 +576,14 @@ __hists__add_entry(struct hists *hists, bool sample_self, struct hist_entry_ops *ops) { + struct namespaces *ns = thread__namespaces(al->thread); struct hist_entry entry = { .thread = al->thread, .comm = thread__comm(al->thread), + .cgroup_id = { + .dev = ns ? ns->link_info[CGROUP_NS_INDEX].dev : 0, + .ino = ns ? ns->link_info[CGROUP_NS_INDEX].ino : 0, + }, .ms = { .map = al->map, .sym = al->sym, diff --git a/tools/perf/util/hist.h b/tools/perf/util/hist.h index 2e839bf..ee3670a 100644 --- a/tools/perf/util/hist.h +++ b/tools/perf/util/hist.h @@ -30,6 +30,7 @@ enum hist_column { HISTC_DSO, HISTC_THREAD, HISTC_COMM, + HISTC_CGROUP_ID, HISTC_PARENT, HISTC_CPU, HISTC_SOCKET, diff --git a/tools/perf/util/sort.c b/tools/perf/util/sort.c index f8f16c0..d8c7489 100644 --- a/tools/perf/util/sort.c +++ b/tools/perf/util/sort.c @@ -536,6 +536,46 @@ struct sort_entry sort_cpu = { .se_width_idx = HISTC_CPU, }; +/* --sort cgroup_id */ + +static int64_t _sort__cgroup_dev_cmp(u64 left_dev, u64 right_dev) +{ + return (int64_t)(right_dev - left_dev); +} + +static int64_t _sort__cgroup_inode_cmp(u64 left_ino, u64 right_ino) +{ + return (int64_t)(right_ino - left_ino); +} + +static int64_t +sort__cgroup_id_cmp(struct hist_entry *left, struct hist_entry *right) +{ + int64_t ret; + + ret = _sort__cgroup_dev_cmp(right->cgroup_id.dev, left->cgroup_id.dev); + if (ret != 0) + return ret; + + return _sort__cgroup_inode_cmp(right->cgroup_id.ino, + left->cgroup_id.ino); +} + +static int hist_entry__cgroup_id_snprintf(struct hist_entry *he, + char *bf, size_t size, + unsigned int width __maybe_unused) +{ + return repsep_snprintf(bf, size, "%lu/0x%lx", he->cgroup_id.dev, + he->cgroup_id.ino); +} + +struct sort_entry sort_cgroup_id = { + .se_header = "cgroup id (dev/inode)", + .se_cmp = sort__cgroup_id_cmp, + .se_snprintf = hist_entry__cgroup_id_snprintf, + .se_width_idx = HISTC_CGROUP_ID, +}; + /* --sort socket */ static int64_t @@ -1459,6 +1499,7 @@ static struct sort_dimension common_sort_dimensions[] = { DIM(SORT_TRANSACTION, "transaction", sort_transaction), DIM(SORT_TRACE, "trace", sort_trace), DIM(SORT_SYM_SIZE, "symbol_size", sort_sym_size), + DIM(SORT_CGROUP_ID, "cgroup_id", sort_cgroup_id), }; #undef DIM diff --git a/tools/perf/util/sort.h b/tools/perf/util/sort.h index f583325..baf20a3 100644 --- a/tools/perf/util/sort.h +++ b/tools/perf/util/sort.h @@ -54,6 +54,11 @@ struct he_stat { u32 nr_events; }; +struct namespace_id { + u64 dev; + u64 ino; +}; + struct hist_entry_diff { bool computed; union { @@ -91,6 +96,7 @@ struct hist_entry { struct map_symbol ms; struct thread *thread; struct comm *comm; + struct namespace_id cgroup_id; u64 ip; u64 transaction; s32 socket; @@ -212,6 +218,7 @@ enum sort_type { SORT_TRANSACTION, SORT_TRACE, SORT_SYM_SIZE, + SORT_CGROUP_ID, /* branch stack specific sort keys */ __SORT_BRANCH_STACK,