Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751111AbcCHSCv (ORCPT ); Tue, 8 Mar 2016 13:02:51 -0500 Received: from mail-wm0-f66.google.com ([74.125.82.66]:33946 "EHLO mail-wm0-f66.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750702AbcCHSCo (ORCPT ); Tue, 8 Mar 2016 13:02:44 -0500 Date: Tue, 8 Mar 2016 19:02:39 +0100 From: Ingo Molnar To: Dmitry Vyukov Cc: Peter Zijlstra , Wang Nan , Ingo Molnar , LKML , He Kuang , Alexei Starovoitov , Arnaldo Carvalho de Melo , Brendan Gregg , Jiri Olsa , Masami Hiramatsu , Namhyung Kim , Zefan Li , pi3orama@163.com Subject: Re: [RESEND PATCH 0/5] perf core: Support overwrite ring buffer Message-ID: <20160308180238.GA4220@gmail.com> References: <20160308164438.GA24109@gmail.com> <20160308164859.GA27516@gmail.com> <20160308172425.GA3017@gmail.com> <20160308173709.GC3017@gmail.com> <20160308174856.GA28862@gmail.com> <20160308175753.GA3015@gmail.com> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <20160308175753.GA3015@gmail.com> User-Agent: Mutt/1.5.23 (2014-03-12) Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 3706 Lines: 61 * Ingo Molnar wrote: > > * Ingo Molnar wrote: > > > Things got a lot more lively after that! > > > > But most of the overhead seems to come from systemd trying to dump core or > > something like that: > > > > 85872 mingo 20 0 34712 3016 2656 S 4.6 0.0 0:00.14 systemd-coredum > > 85440 mingo 20 0 34712 3028 2664 S 4.2 0.0 0:00.13 systemd-coredum > > 85751 mingo 20 0 34712 3076 2716 S 4.2 0.0 0:00.13 systemd-coredum > > 85840 mingo 20 0 34712 2988 2624 S 4.2 0.0 0:00.13 systemd-coredum > > 85861 mingo 20 0 34712 3080 2720 S 4.2 0.0 0:00.13 systemd-coredum > > 85954 mingo 20 0 34712 3028 2664 S 4.2 0.0 0:00.13 systemd-coredum > > > > and I have: > > > > fomalhaut:~/go/src/github.com/google/syzkaller> ulimit -c > > 0 > > > > weird ... Has any of you seen such behavior? > > So the workaround for that is to disable systemd trying to log every core dump to > the system journal (!), via: > > echo > /proc/sys/kernel/core_pattern So with that fixed, I finally started fuzzing for real. With nproc set to 120 it seems to be chugging along at about 25% system utilization: Tasks: 1271 total, 1 running, 1270 sleeping, 0 stopped, 0 zombie %Cpu(s): 4.5 us, 35.3 sy, 0.0 ni, 55.2 id, 0.5 wa, 4.5 hi, 0.0 si, 0.0 st KiB Mem : 26401230+total, 25401017+free, 1624640 used, 8377496 buff/cache KiB Swap: 0 total, 0 free, 0 used. 26143329+avail Mem PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 87593 mingo 20 0 7850436 162284 11672 S 2941 0.1 33:01.83 syz-fuzzer 923 root 20 0 84772 44344 43840 S 22.8 0.0 1:23.86 systemd-journal 1369 root 16 -4 114636 3256 2832 S 15.7 0.0 0:29.03 auditd 1379 root 12 -8 80236 1764 1432 S 8.3 0.0 0:15.79 audispd 878 root 20 0 0 0 0 S 6.9 0.0 0:16.55 jbd2/sda1-8 1381 root 16 -4 52216 3232 2892 S 3.8 0.0 0:07.08 sedispatch Even that one is not ideal - obviously there's way too much systemd-journal overhead, but I'm unable to turn the darn thing off ... with nproc=480 it does not seem to be working very well - it quickly generates: 2016/03/08 18:59:38 local-0: lost connection: exit status 2 2016/03/08 18:59:38 local-0: saving crash 'lost connection' to crash-local-0-1457459978295423413 and then after some time starts to ramp up again. It's mostly idling around. Thanks, Ingo