Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
Subject: Re: collecting simple benchmark scripts?
To:     Kees Cook <keescook@google.com>, Shuah Khan <shuah@kernel.org>,
        Fengguang Wu <fengguang.wu@intel.com>,
        Tycho Andersen <tycho@tycho.ws>,
        Steven Rostedt <srostedt@redhat.com>,
        Kevin Hilman <khilman@kernel.org>,
        Thomas Gleixner <tglx@linutronix.de>,
        "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>
Cc:     LKML <linux-kernel@vger.kernel.org>
References: <CAGXu5j+LVvfdeD4rhTKAQ0PP5s5ysAjNtrfkpcp2sLzwKqbVHw@mail.gmail.com>
From:   Laura Abbott <labbott@redhat.com>
Message-ID: <7723ae8d-8333-ba17-6983-a45ec8b11c54@redhat.com>
Date:   Thu, 15 Feb 2018 10:32:21 -0800
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Thunderbird/52.5.2
MIME-Version: 1.0
In-Reply-To: <CAGXu5j+LVvfdeD4rhTKAQ0PP5s5ysAjNtrfkpcp2sLzwKqbVHw@mail.gmail.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On 02/14/2018 02:50 PM, Kees Cook wrote:
> Hi,
> 
> In a separate thread, some folks were looking for some simple
> benchmarks for evaluating various changes to kernel internals (as
> opposed to the much more focused things like xfstests). For me, this
> has been an area of lore and passed-around scripts, and it seems like
> maybe we should have a subdirectory of tools/testing/benchmarks/ or
> something to collect these?
> 
> (Or maybe this already exists and I've totally missed it?)
> 
> I've got at least one micro-benchmark in
> tools/testing/selftests/seccomp/seccomp_benchmark.c, and searches show
> tools/testing/selftests/vm/gup_benchmark.c too, but I was thinking of
> either more generali things more like the famous "kernel build
> benchmark" or a wrapper for running hackbench to get some statistics
> out of it, etc.
> 
> Or, I guess, at least collecting all the micro-benchmarks in some
> single place, as they're a bit scattered.
> 
> I'm sure I'm not remotely the first person to bring this up, but my
> attempts at searches for this have failed.
> 
> Thoughts?
> 
> -Kees
> 

This is the script I've been using for hackbench since it can
be noisy. I expect someone to tell me the math is wrong but
I'd be happy to throw this in a repo if others are interested

#!/bin/sh

CNT=100
mean=0.0
M2=0.0
for i in $(seq 1 $CNT); do
	echo $i
	r=`hackbench -g 20 -l 1000 | grep Time | cut -d ' ' -f 2`
	d_calc="$r-$mean"
         d=`echo $d_calc | bc -l`
	mean_calc="$mean+($d/$i)"
	mean=`echo $mean_calc | bc -l`
	M2_calc="$M2+($d*($r-$mean))"
	M2=`echo  $M2_calc| bc -l`
done

echo "mean $mean"
V_calc="$M2/$(($CNT-1))"
V=`echo $V_calc | bc -l`
DEV_calc="sqrt($V)"
DEV=`echo $DEV_calc | bc -l`
echo "variance $V"
echo "stdev $DEV"