Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1757151Ab0BCVVs (ORCPT ); Wed, 3 Feb 2010 16:21:48 -0500 Received: from 0122700014.0.fullrate.dk ([95.166.99.235]:45253 "EHLO kernel.dk" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1755681Ab0BCVVp (ORCPT ); Wed, 3 Feb 2010 16:21:45 -0500 Date: Wed, 3 Feb 2010 22:21:43 +0100 From: Jens Axboe To: Frederic Weisbecker Cc: Ingo Molnar , LKML , Peter Zijlstra , Arnaldo Carvalho de Melo , Steven Rostedt , Paul Mackerras , Hitoshi Mitake , Li Zefan , Lai Jiangshan , Masami Hiramatsu Subject: Re: [RFC GIT PULL] perf/trace/lock optimization/scalability improvements Message-ID: <20100203212143.GW5733@kernel.dk> References: <1265188475-23509-1-git-send-regression-fweisbec@gmail.com> <20100203102540.GQ5733@kernel.dk> <20100203205009.GB5068@nowhere> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20100203205009.GB5068@nowhere> Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3286 Lines: 104 On Wed, Feb 03 2010, Frederic Weisbecker wrote: > On Wed, Feb 03, 2010 at 11:25:41AM +0100, Jens Axboe wrote: > > On Wed, Feb 03 2010, Frederic Weisbecker wrote: > > > Hi, > > > > > > There are many things that happen in this patchset, treating > > > different problems: > > > > > > - remove most of the string copy overhead in fast path > > > - open the way for lock class oriented profiling (as > > > opposite to lock instance profiling. Both can be useful > > > in different ways). > > > - remove the buffers muliplexing (less contention) > > > - event injection support > > > - remove violent lock events recursion (only 2 among 3, the remaining > > > one is detailed below). > > > > > > Some differences, by running: > > > perf lock record perf sched pipe -l 100000 > > > > > > Before the patchset: > > > > > > Total time: 91.015 [sec] > > > > > > 910.157300 usecs/op > > > 1098 ops/sec > > > > > > After this patchset applied: > > > > > > Total time: 43.706 [sec] > > > > > > 437.062080 usecs/op > > > 2288 ops/sec > > > > This does a lot better here, even if it isn't exactly stellar > > performance. It generates a LOT of data: > > > > root@nehalem:/dev/shm # time perf lock rec -fg ls > > perf.data perf.data.old > > [ perf record: Woken up 0 times to write data ] > > [ perf record: Captured and wrote 137.224 MB perf.data (~5995421 > > samples) ] > > > > Doh, 137 MB for a single ls :) Yeah, it's pretty crazy. It varies a lot too, I didn't post the other run which was ~170MB. > That said we don't have yet support for callchains in perf lock, > and callchains can fill the buffer quickly, especially on lock > events. You can drop the -g option for now. OK > > So while this is orders of magnitude better than the previous patchset, > > it's still not anywhere near lean. But I expect you know that, just > > consider this a 'I tested it and this is what happened' report :-) > > > Ok, thanks a lot, the fact you can test on a 64 threads box is critically > helpful. My pleasure, I'd love to have a fast and functional perf lockstat. If my testing helps getting there, consider me signed up :-) > I also wonder what happens after this patch applied: > > diff --git a/kernel/perf_event.c b/kernel/perf_event.c > index 98fd360..254b3d4 100644 > --- a/kernel/perf_event.c > +++ b/kernel/perf_event.c > @@ -3094,7 +3094,8 @@ static u32 perf_event_tid(struct perf_event *event, struct task_struct *p) > if (event->parent) > event = event->parent; > > - return task_pid_nr_ns(p, event->ns); > + return p->pid; > } > > In my box it has increased the speed from 2x this patchset. Cool, I'll give that a spin in the morning, the box is off atm. > I wonder if the tool becomes usable for you with that. > Otherwise, it means we have other things to fix, and > the result of: > > perf record -g -f perf lock record sleep 6 > perf report > > would be very nice to have. I'll package that up for you and put it somewhere. -- Jens Axboe -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majordomo@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/