Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
Date:   Mon, 14 Sep 2020 22:43:00 +0530
From:   Gautham R Shenoy <ego@linux.vnet.ibm.com>
To:     Pratik Rajesh Sampat <psampat@linux.ibm.com>
Cc:     rjw@rjwysocki.net, daniel.lezcano@linaro.org,
        srivatsa@csail.mit.edu, shuah@kernel.org, npiggin@gmail.com,
        ego@linux.vnet.ibm.com, svaidy@linux.ibm.com,
        linux-pm@vger.kernel.org, linux-kernel@vger.kernel.org,
        linux-kselftest@vger.kernel.org, pratik.r.sampat@gmail.com
Subject: Re: [RFC v4 0/1] Selftest for cpuidle latency measurement
Message-ID: <20200914171259.GA25628@in.ibm.com>
Reply-To: ego@linux.vnet.ibm.com
References: <20200902114506.45809-1-psampat@linux.ibm.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20200902114506.45809-1-psampat@linux.ibm.com>
User-Agent: Mutt/1.5.23 (2014-03-12)
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

On Wed, Sep 02, 2020 at 05:15:05PM +0530, Pratik Rajesh Sampat wrote:
> Changelog v3-->v4:
> 1. Overhaul in implementation from kernel module to a userspace selftest 
> ---
> 
> The patch series introduces a mechanism to measure wakeup latency for
> IPI and timer based interrupts
> The motivation behind this series is to find significant deviations
> behind advertised latency and residency values
> 
> To achieve this in the userspace, IPI latencies are calculated by
> sending information through pipes and inducing a wakeup, similarly
> alarm events are setup for calculate timer based wakeup latencies.

> 
> To account for delays from kernel-userspace interactions baseline
> observations are taken on a 100% busy CPU and subsequent obervations
> must be considered relative to that.
> 
> In theory, wakeups induced by IPI and Timers should have similar
> wakeup latencies, however in practice there may be deviations which may
> need to be captured.
> 
> One downside of the userspace approach in contrast to the kernel
> implementation is that the run to run variance can turn out to be high
> in the order of ms; which is the scope of the experiments at times.
> 
> Another downside of the userspace approach is that it takes much longer
> to run and hence a command-line option quick and full are added to make
> sure quick 1 CPU tests can be carried out when needed and otherwise it
> can carry out a full system comprehensive test.
> 
> Usage
> ---
> ./cpuidle --mode <full / quick / num_cpus> --output <output location> 
> full: runs on all CPUS
> quick: run on a random CPU
> num_cpus: Limit the number of CPUS to run on
> 
> Sample output snippet
> ---------------------
> --IPI Latency Test---
> SRC_CPU   DEST_CPU IPI_Latency(ns)
> ...
>   0          5       256178
>   0          6       478161
>   0          7       285445
>   0          8       273553
> Expected IPI latency(ns): 100000
> Observed Average IPI latency(ns): 248334

I suppose by run-to-run variance you are referring to the outliers in
the above sequence (like 478161) ? Or is it that each time you run
your test program you observe completely different series of values ?

If it is the former, then perhaps we could discard the outliers for
the purpose of average latency computation and print the max, min and
the corrected-average values above.


> 
> --Timeout Latency Test--
> --Baseline Timeout Latency measurement: CPU Busy--
> Wakeup_src Baseline_delay(ns)
> ...
>  32          972405
>  33         1004287
>  34          986663
>  35          994022
> Expected timeout(ns): 10000000
> Observed Average timeout diff(ns): 991844
>

It would be good to see a complete sample output, perhaps for the
--mode=10 so that it is easy to discern if there are cases when the
observed timeouts/IPI latencies for the busy case are larger than the
idle-case.


> Pratik Rajesh Sampat (1):
>   selftests/cpuidle: Add support for cpuidle latency measurement
> 
>  tools/testing/selftests/Makefile          |   1 +
>  tools/testing/selftests/cpuidle/Makefile  |   7 +
>  tools/testing/selftests/cpuidle/cpuidle.c | 616 ++++++++++++++++++++++
>  tools/testing/selftests/cpuidle/settings  |   1 +
>  4 files changed, 625 insertions(+)
>  create mode 100644 tools/testing/selftests/cpuidle/Makefile
>  create mode 100644 tools/testing/selftests/cpuidle/cpuidle.c
>  create mode 100644 tools/testing/selftests/cpuidle/settings
> 
> -- 
> 2.26.2
> 

--
Thanks and Regards
gautham.