Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67;
From:   Nathan Lynch <nathanl@linux.ibm.com>
To:     "Gautham R. Shenoy" <ego@linux.vnet.ibm.com>
Cc:     linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org,
        Michael Ellerman <mpe@ellerman.id.au>,
        Vaidyanathan Srinivasan <svaidy@linux.vnet.ibm.com>,
        Kamalesh Babulal <kamalesh@linux.vnet.ibm.com>,
        "Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com>,
        Tyrel Datwyler <tyreld@linux.ibm.com>
Subject: Re: [PATCH 1/3] powerpc/pseries: Account for SPURR ticks on idle CPUs
In-Reply-To: <1574856072-30972-2-git-send-email-ego@linux.vnet.ibm.com>
References: <1574856072-30972-1-git-send-email-ego@linux.vnet.ibm.com> <1574856072-30972-2-git-send-email-ego@linux.vnet.ibm.com>
Date:   Wed, 04 Dec 2019 16:24:52 -0600
Message-ID: <87o8wnu3t7.fsf@linux.ibm.com>
MIME-Version: 1.0
Content-Type: text/plain
Sender: linux-kernel-owner@vger.kernel.org
Precedence: bulk

"Gautham R. Shenoy" <ego@linux.vnet.ibm.com> writes:
> diff --git a/arch/powerpc/kernel/idle.c b/arch/powerpc/kernel/idle.c
> index a36fd05..708ec68 100644
> --- a/arch/powerpc/kernel/idle.c
> +++ b/arch/powerpc/kernel/idle.c
> @@ -33,6 +33,8 @@
>  unsigned long cpuidle_disable = IDLE_NO_OVERRIDE;
>  EXPORT_SYMBOL(cpuidle_disable);
>  
> +DEFINE_PER_CPU(u64, idle_spurr_cycles);
> +

Does idle_spurr_cycles need any special treatment for CPU
online/offline?

>  static int __init powersave_off(char *arg)
>  {
>  	ppc_md.power_save = NULL;
> diff --git a/drivers/cpuidle/cpuidle-pseries.c b/drivers/cpuidle/cpuidle-pseries.c
> index 74c2479..45e2be4 100644
> --- a/drivers/cpuidle/cpuidle-pseries.c
> +++ b/drivers/cpuidle/cpuidle-pseries.c
> @@ -30,11 +30,14 @@ struct cpuidle_driver pseries_idle_driver = {
>  static struct cpuidle_state *cpuidle_state_table __read_mostly;
>  static u64 snooze_timeout __read_mostly;
>  static bool snooze_timeout_en __read_mostly;
> +DECLARE_PER_CPU(u64, idle_spurr_cycles);

This belongs in a header... 


> -static inline void idle_loop_prolog(unsigned long *in_purr)
> +static inline void idle_loop_prolog(unsigned long *in_purr,
> +				    unsigned long *in_spurr)
>  {
>  	ppc64_runlatch_off();
>  	*in_purr = mfspr(SPRN_PURR);
> +	*in_spurr = mfspr(SPRN_SPURR);
>  	/*
>  	 * Indicate to the HV that we are idle. Now would be
>  	 * a good time to find other work to dispatch.
> @@ -42,13 +45,16 @@ static inline void idle_loop_prolog(unsigned long *in_purr)
>  	get_lppaca()->idle = 1;
>  }
>  
> -static inline void idle_loop_epilog(unsigned long in_purr)
> +static inline void idle_loop_epilog(unsigned long in_purr,
> +				    unsigned long in_spurr)
>  {
>  	u64 wait_cycles;
> +	u64 *idle_spurr_cycles_ptr = this_cpu_ptr(&idle_spurr_cycles);
>  
>  	wait_cycles = be64_to_cpu(get_lppaca()->wait_state_cycles);
>  	wait_cycles += mfspr(SPRN_PURR) - in_purr;
>  	get_lppaca()->wait_state_cycles = cpu_to_be64(wait_cycles);
> +	*idle_spurr_cycles_ptr += mfspr(SPRN_SPURR) - in_spurr;

... and the sampling and increment logic probably should be further
encapsulated in accessor functions that can be used in both the cpuidle
driver and the default/generic idle implementation. Or is there some
reason this is specific to the pseries cpuidle driver?