Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::3:1 as permitted sender) client-ip=2620:137:e000::3:1;
Message-ID: <09b5de78a39277400a1198f1120448e95e9855cd.camel@linux.vnet.ibm.com>
Subject: Re: [RFC v3 0/2] CPU-Idle latency selftest framework
From:   Aboorva Devarajan <aboorvad@linux.vnet.ibm.com>
To:     20230911053620.87973-1-aboorvad@linux.vnet.ibm.com,
        mpe@ellerman.id.au, rafael@kernel.org, daniel.lezcano@linaro.org,
        linux-pm@vger.kernel.org
Cc:     sshegde@linux.vnet.ibm.com, srikar@linux.vnet.ibm.com,
        npiggin@gmail.com, rmclure@linux.ibm.com, arnd@arndb.de,
        joel@jms.id.au, shuah@kernel.org, linux-kselftest@vger.kernel.org,
        linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org,
        pratik.r.sampat@gmail.com
In-Reply-To: <2725211932028cf24ab34da9eb9d19190848cceb.camel@linux.vnet.ibm.com>
References: <20230911053620.87973-1-aboorvad@linux.vnet.ibm.com>
         <2725211932028cf24ab34da9eb9d19190848cceb.camel@linux.vnet.ibm.com>
Content-Type: text/plain; charset="UTF-8"
Date:   Thu, 12 Oct 2023 10:18:06 +0530
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
Precedence: bulk

On Mon, 2023-09-25 at 10:36 +0530, Aboorva Devarajan wrote:

Gentle ping to check if there are any feedback or comments on this
patch-set.

Thanks
Aboorva

> On Mon, 2023-09-11 at 11:06 +0530, Aboorva Devarajan wrote:
> 
> CC'ing CPUidle lists and maintainers,
> 
> Patch Summary:
> 
> The patchset introduces a kernel module and userspace driver designed
> for estimating the wakeup latency experienced when waking up from
> various CPU idle states. It primarily measures latencies related to
> two
> types of events: Inter-Processor Interrupts (IPIs) and Timers.
> 
> Background:
> 
> Initially, these patches were introduced as a generic self-test.
> However, it was later discovered that Intel platforms incorporate
> timer-based wakeup optimizations. These optimizations allow CPUs to
> perform a pre-wakeup, which limits the effectiveness of latency
> observation in certain scenarios because it only measures the
> optimized
> wakeup latency [1]. 
> 
> Therefore, in this RFC, the self-test is specifically integrated into
> PowerPC, as it has been tested and used in PowerPC so far.
> 
> Another proposal is to introduce these patches as a generic cpuilde
> IPI
> and timer wake-up test. While this method may not give us an exact
> measurement of latency variations at the hardware level, it can still
> help us assess this metric from a software observability standpoint.
> 
> Looking forward to hearing what you think and any suggestions you may
> have regarding this. Thanks.
> 
> [1] 
> https://lore.kernel.org/linux-pm/20200914174625.GB25628@in.ibm.com/T/#m5c004b9b1a918f669e91b3d0f33e2e3500923234
> 
> > Changelog: v2 -> v3
> > 
> > * Minimal code refactoring
> > * Rebased on v6.6-rc1
> > 
> > RFC v1: 
> > https://lore.kernel.org/all/20210611124154.56427-1-psampat@linux.ibm.com/
> > 
> > RFC v2:
> > https://lore.kernel.org/all/20230828061530.126588-2-aboorvad@linux.vnet.ibm.com/
> > 
> > Other related RFC:
> > https://lore.kernel.org/all/20210430082804.38018-1-psampat@linux.ibm.com/
> > 
> > Userspace selftest:
> > https://lkml.org/lkml/2020/9/2/356
> > 
> > ----
> > 
> > A kernel module + userspace driver to estimate the wakeup latency
> > caused by going into stop states. The motivation behind this
> > program
> > is
> > to find significant deviations behind advertised latency and
> > residency
> > values.
> > 
> > The patchset measures latencies for two kinds of events. IPIs and
> > Timers
> > As this is a software-only mechanism, there will be additional
> > latencies
> > of the kernel-firmware-hardware interactions. To account for that,
> > the
> > program also measures a baseline latency on a 100 percent loaded
> > CPU
> > and the latencies achieved must be in view relative to that.
> > 
> > To achieve this, we introduce a kernel module and expose its
> > control
> > knobs through the debugfs interface that the selftests can engage
> > with.
> > 
> > The kernel module provides the following interfaces within
> > /sys/kernel/debug/powerpc/latency_test/ for,
> > 
> > IPI test:
> >     ipi_cpu_dest = Destination CPU for the IPI
> >     ipi_cpu_src = Origin of the IPI
> >     ipi_latency_ns = Measured latency time in ns
> > Timeout test:
> >     timeout_cpu_src = CPU on which the timer to be queued
> >     timeout_expected_ns = Timer duration
> >     timeout_diff_ns = Difference of actual duration vs expected
> > timer
> > 
> > Sample output is as follows:
> > 
> > # --IPI Latency Test---
> > # Baseline Avg IPI latency(ns): 2720
> > # Observed Avg IPI latency(ns) - State snooze: 2565
> > # Observed Avg IPI latency(ns) - State stop0_lite: 3856
> > # Observed Avg IPI latency(ns) - State stop0: 3670
> > # Observed Avg IPI latency(ns) - State stop1: 3872
> > # Observed Avg IPI latency(ns) - State stop2: 17421
> > # Observed Avg IPI latency(ns) - State stop4: 1003922
> > # Observed Avg IPI latency(ns) - State stop5: 1058870
> > #
> > # --Timeout Latency Test--
> > # Baseline Avg timeout diff(ns): 1435
> > # Observed Avg timeout diff(ns) - State snooze: 1709
> > # Observed Avg timeout diff(ns) - State stop0_lite: 2028
> > # Observed Avg timeout diff(ns) - State stop0: 1954
> > # Observed Avg timeout diff(ns) - State stop1: 1895
> > # Observed Avg timeout diff(ns) - State stop2: 14556
> > # Observed Avg timeout diff(ns) - State stop4: 873988
> > # Observed Avg timeout diff(ns) - State stop5: 959137
> > 
> > Aboorva Devarajan (2):
> >   powerpc/cpuidle: cpuidle wakeup latency based on IPI and timer
> > events
> >   powerpc/selftest: Add support for cpuidle latency measurement
> > 
> >  arch/powerpc/Kconfig.debug                    |  10 +
> >  arch/powerpc/kernel/Makefile                  |   1 +
> >  arch/powerpc/kernel/test_cpuidle_latency.c    | 154 ++++++
> >  tools/testing/selftests/powerpc/Makefile      |   1 +
> >  .../powerpc/cpuidle_latency/.gitignore        |   2 +
> >  .../powerpc/cpuidle_latency/Makefile          |   6 +
> >  .../cpuidle_latency/cpuidle_latency.sh        | 443
> > ++++++++++++++++++
> >  .../powerpc/cpuidle_latency/settings          |   1 +
> >  8 files changed, 618 insertions(+)
> >  create mode 100644 arch/powerpc/kernel/test_cpuidle_latency.c
> >  create mode 100644
> > tools/testing/selftests/powerpc/cpuidle_latency/.gitignore
> >  create mode 100644
> > tools/testing/selftests/powerpc/cpuidle_latency/Makefile
> >  create mode 100755
> > tools/testing/selftests/powerpc/cpuidle_latency/cpuidle_latency.sh
> >  create mode 100644
> > tools/testing/selftests/powerpc/cpuidle_latency/settings
> >