Received: by 2002:ac0:946b:0:0:0:0:0 with SMTP id j40csp3037047imj; Mon, 11 Feb 2019 12:43:01 -0800 (PST) X-Google-Smtp-Source: AHgI3IaSg3jl2aX/9YGpfNCmSXSB2occq2brh5ev0KcI8NeNd/EJS2OJNZPBdjePBQjY91Rzv5QM X-Received: by 2002:a63:ef04:: with SMTP id u4mr91351pgh.197.1549917781791; Mon, 11 Feb 2019 12:43:01 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1549917781; cv=none; d=google.com; s=arc-20160816; b=jztwguJj21l5QvvsXDa3Ktjlq4K5Av7ZD3EG+aWdEodYJfoogHoeRHc+VTfrdb9oND SF9HnFtwWC/p1yCAoB+6BxbmIs/qFfcg5/JZPPHRhkt2NJUyfAJwyTNiDx3KGeFXip/Z Fl43K+I/3jsLl1gs7JQYvtpRP5Bfy3r5PTC817/y1KC/ti7swaQwqCiA6uC32NyJBcfZ z7bs0a0vrLRLkRkrcWO7dvHfdepdu8pz02Cd4EEHMFsUV/UVVBYt6SqCm0w7aVJoHxSe +YD3jV6Kp01aTSy+t/Or6fvmdMGIDatExQKo8wwDpo2jVOYBY4viZaRWLJprsQTQNl6Z waLg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:cc:to:subject:message-id:date:from :in-reply-to:references:mime-version:dkim-signature; bh=XxJxqy0JVpfH4MMaRLx8EkJwpOL/lVshLgZXsB8rvVQ=; b=XbzZrCvboOU3gK7m0Znvx9je/yqx9SS1dk00ZFgMitQ4IuPalcZ9oZCMzojNVt6weH hpNF9PvyD8fozMgxVy76BowdFm1TIZqHk7lRvHRiE3K65bP49zGV2m3iVi+H1fTLclNZ cbqJOTumbt3Jd7gGop+17WMMyUTTP3jTco+VLK1E9qgIeDwFPq8dWzouiqbm09Qivv/7 u62Dv3VJNnlyUS+0P/4vNMsDMyjtRHv2G7WKFFZEWi1cc/Q2kNAht86qyG9YsHdeIOqa baYx15SV6QaYAj+dO68YMqB4guz+7xCqu9jUGJiljNuAYuf3t25mjCNLaH5x3+YU4sVf M0BA== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=MkSN3Yip; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id c3si11006632pls.73.2019.02.11.12.42.46; Mon, 11 Feb 2019 12:43:01 -0800 (PST) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@google.com header.s=20161025 header.b=MkSN3Yip; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=REJECT sp=REJECT dis=NONE) header.from=google.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2388211AbfBKTaV (ORCPT + 99 others); Mon, 11 Feb 2019 14:30:21 -0500 Received: from mail-ua1-f66.google.com ([209.85.222.66]:34317 "EHLO mail-ua1-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1731515AbfBKTaU (ORCPT ); Mon, 11 Feb 2019 14:30:20 -0500 Received: by mail-ua1-f66.google.com with SMTP id x15so32013uaq.1 for ; Mon, 11 Feb 2019 11:30:19 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=mime-version:references:in-reply-to:from:date:message-id:subject:to :cc; bh=XxJxqy0JVpfH4MMaRLx8EkJwpOL/lVshLgZXsB8rvVQ=; b=MkSN3Yip13MwZeTXwcPcT7re/hNwbn6Q453vFdNi/YC+/m9O+VQ/GRMqf+7rZt5UkL HJaCvV3478Ygw50nHVfkTdOajVmdBfDxQn20UetDMpQ2dBhcpkl6soxIqWL47su5sAQM D6Fo5iPu+K24rxLsTodm8ELzodQ2oCsCJOnBsGlb4xMt4M+jk6U/VmUa87RZ1o1XKg58 vZcLAwUV8DwXP8XUGf5XHFmeQiJSzWxpgG6q6QxSpv1TJEzLIPIT7KMLiP1jZ+a6axR1 FTOLUQXv7ZibW0JbIUQXBZLdwJ3ACNrNCUji7qDMWXgG8WWYfgSx/1tlNQIp/3pB3TPh tpbg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:mime-version:references:in-reply-to:from:date :message-id:subject:to:cc; bh=XxJxqy0JVpfH4MMaRLx8EkJwpOL/lVshLgZXsB8rvVQ=; b=Dz+JjINUc/ufLS4WyAW6KXa9Th3yf3KYa5+gIqESAfNL5U3D9CzWd3BGtpNDdeQArn +1njr4ZGt0bQ826Gtpk0urSiJc1MX5X2CIXDM77VUPCk5fiHf1Zrr0VH1ytFN4o7pK/M U4SyKMNzzjT+6kuqli56kZsgyFbzQXEDjewK8HmH4R4OJf2WTfbzrBd/mO6IkJKnQiQ8 uZ2N5/wI3ASxgyE0ixo+IjdQcJzbb5gqjh2coY8ywj/+sgVlH+VMaePx64Ps89KsBhwi fnbhGhwhM82pIQrmopdmpDRNrKEz90wfqLD03uOCKeVtRgf6rtvzjHcRRHgnd7UwQ7Uz eegg== X-Gm-Message-State: AHQUAuaRiTlmp3CaORJ6FP0Wf963xwQj541d8FXh+dXJ+aM3szqgrH5S vunfRS/zBWpX8kAmII3iHiA3PqwzBNC08+Hobk+9Wg== X-Received: by 2002:ab0:5590:: with SMTP id v16mr14827577uaa.132.1549913419017; Mon, 11 Feb 2019 11:30:19 -0800 (PST) MIME-Version: 1.0 References: <6bf24b7d-2bd3-8091-cf49-363c91e4e864@linux.intel.com> <20190204114144.GC18141@krava> <20190204192721.GI5593@kernel.org> <20190204202818.GC4794@krava> <20190205133727.GF4794@krava> <20190211101957.GB14253@krava> <20190211185510.GF3269@kernel.org> In-Reply-To: <20190211185510.GF3269@kernel.org> From: Stephane Eranian Date: Mon, 11 Feb 2019 11:30:07 -0800 Message-ID: Subject: Re: [RFC/PATCH 00/14] perf record: Add support to store data in directory To: Arnaldo Carvalho de Melo Cc: Jiri Olsa , Alexey Budankov , Jiri Olsa , lkml , Ingo Molnar , Namhyung Kim , Alexander Shishkin , Peter Zijlstra , Adrian Hunter , Andi Kleen , Song Liu Content-Type: text/plain; charset="UTF-8" Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Arnaldo, On Mon, Feb 11, 2019 at 10:55 AM Arnaldo Carvalho de Melo wrote: > > Em Mon, Feb 11, 2019 at 10:34:16AM -0800, Stephane Eranian escreveu: > > Jiri, > > > > On Mon, Feb 11, 2019 at 2:20 AM Jiri Olsa wrote: > > > > > > On Tue, Feb 05, 2019 at 02:37:27PM +0100, Jiri Olsa wrote: > > > > On Mon, Feb 04, 2019 at 02:44:37PM -0800, Stephane Eranian wrote: > > > > > Jiri, > > > > > > > > > > While you're looking at the output format, I think it would be good > > > > > time to simplify the code handling perf.data file. > > > > > Today, perf record can emit in two formats: file mode or pipe mode. > > > > > This adds complexity in the code and > > > > > is error prone as the file mode path is tested more than the pipe mode > > > > > path. We have run into multiple issues with > > > > > the pipe mode in recent years. There is no real reason why we need to > > > > > maintain two formats. If I recall, the pipe format > > > > > was introduced because on pipes you cannot lseek to update the headers > > > > > and therefore some of the information present as tables > > > > > updated on the fly needed to be generated as pseudo records by the > > > > > tool. I believe that the pipe format covers all the needs and could > > > > > supersede the file mode format. That would simplify code in perf > > > > > record and eliminate the risk of errors when new headers > > > > > are introduced. > > > > > > > > yep, I think we have almost all the features covered for pipe mode, > > > > and we have all necessary events to describe events features > > > > > > > > so with some effort we could switch off the superfluos file header > > > > and use only events to describe events ;-) make sense, I'll check > > > > on it > > > > > > so following features are not synthesized: > > > > > > FEAT_OPN(TRACING_DATA, tracing_data, false), > > > FEAT_OPN(BUILD_ID, build_id, false), > > > FEAT_OPN(BRANCH_STACK, branch_stack, false), > > > FEAT_OPN(AUXTRACE, auxtrace, false), > > > FEAT_OPN(STAT, stat, false), > > > FEAT_OPN(CACHE, cache, true), > > > > > What do you need for BRANCH_STACK? > > > > > I think all could be added and worked around with exception > > > of BUILD_ID, which we store at the end (after processing > > > all data) and we need it early in the report phase > > > > > Buildids are injected after the fact via perf inject when in pipe mode. > > > > > maybe it's time to re-think that buildid -> mmap event > > > association again, because it's pain in current implementation > > > as well > > > > > Sure, but what do you propose? > > this keeps resurfacing, the idea is to have the building go together > with the PERF_RECORD_MMAP3 event, i.e. as part of setting up an > executable mapping the loader would get the buildid and ask the kernel > to keep it aroung, then when a PERF_RECORD_MMAP needs to be issued, it > can include the build id, so tooling will not need to get it. > And how would the dynamic loader (ld.so) communicate the buildid to the kernel? How would that work for statically linked binaries. I think you're say the kernel would parse the ELF header looking for that note section and extract the buildid from there. Is that what you are proposing? > Alternatively, we would have a separate thread to process > PERF_RECORD_MMAP events, and as soon as it gets one from the kernel, > augment it straight away with the build-id it reads from the ELF file, > i.e. no need to have the kernel provide it, do it just like we do with > PERF_RECORD_BPF_EVENT, which reminds me Song probably already posted > thise bits... > But that would not work in pipe mode, wouldn't it? Unless that thread intercepts everything pushed to the pipe looking for MMAP records. > > > looks like bpf code is actualy getting build ids and storing > > > it for the callchains in kernel.. we can check if we can do > > > something similar for mmap events > > > > > > jirka > > -- > > - Arnaldo