Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S932145AbbLNJir (ORCPT ); Mon, 14 Dec 2015 04:38:47 -0500 Received: from mail-wm0-f44.google.com ([74.125.82.44]:32991 "EHLO mail-wm0-f44.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S932105AbbLNJip (ORCPT ); Mon, 14 Dec 2015 04:38:45 -0500 Date: Mon, 14 Dec 2015 10:38:41 +0100 From: Ingo Molnar To: Peter Zijlstra Cc: David Ahern , Namhyung Kim , Namhyung Kim , Arnaldo Carvalho de Melo , Jiri Olsa , LKML , Frederic Weisbecker , Andi Kleen , Stephane Eranian , Adrian Hunter Subject: Re: [PATCHSET 00/16] perf top: Add multi-thread support (v1) Message-ID: <20151214093841.GB30347@gmail.com> References: <1449734015-9148-1-git-send-email-namhyung@kernel.org> <20151210080118.GA8664@gmail.com> <5E6F0F13-9696-45F1-A0E8-CA0B95020D10@gmail.com> <20151211081141.GA21600@gmail.com> <566AE54B.1010702@gmail.com> <20151214092613.GL6357@twins.programming.kicks-ass.net> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20151214092613.GL6357@twins.programming.kicks-ass.net> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 2563 Lines: 69 * Peter Zijlstra wrote: > On Fri, Dec 11, 2015 at 08:01:31AM -0700, David Ahern wrote: > > On 12/11/15 1:11 AM, Ingo Molnar wrote: > > > > > >* Namhyung Kim wrote: > > > > > >>IIRC David said that thread per cpu seems too much especially on a large system > > >>(like ~1024 cpu). [...] > > > > > >Too much in what fashion? For recording I think it's the fastest, most natural > > >model - anything else will create cache line bounces. > > > > The intrusiveness of perf on the system under observation. I understand > > there are a lot of factors that go into it. > > So I can see some of that, if every cpu has its own thread then every > cpu will occasionally schedule that thread. Whereas if there were less, > you'd not have that. > > Still, I think it makes sense to implement it, we need the multi-file > option anyway. Once we have that, we can also implement a per-node > option, which should be a fairly simple hybrid of the two approaches. > > The thing is, perf-record is really struggling on big machines. > > And in an unrelated note, I absolutely detest --buildid being the > default, it makes perf-record blow chunks. So I'd absolutely _love_ to split up the singular perf.data into a hierarchy of files in a .perf directory, with a structure like this (4-core system): .perf/cmdline .perf/features .perf/evlist .perf/ring_buffers/cpu0/raw.trace .perf/ring_buffers/cpu1/raw.trace .perf/ring_buffers/cpu2/raw.trace .perf/ring_buffers/cpu3/raw.trace ... I.e. the current single file format of perf.data would be split up into individual files. Each CPU would get its own trace file output - any sorting and ordering would be done afterwards. 'perf record' itself would never by default have to do any of that, it's a pure recording session. 'perf archive' would still create a single file to make transport between machines easy. perf.data.old would be replaced by a .perf.old directory or so. Debugging would be easier too I think, as there's no complex perf data format anymore, it's all in individual (typically text, or binary dump) files in the .perf directory. This would solve all the scalability problems - and would make the format more extensible and generally more accessible as well. What do you think? Thanks, Ingo -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/