Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752765AbdGSCOm (ORCPT ); Tue, 18 Jul 2017 22:14:42 -0400 Received: from mail-pg0-f65.google.com ([74.125.83.65]:33325 "EHLO mail-pg0-f65.google.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752694AbdGSCOY (ORCPT ); Tue, 18 Jul 2017 22:14:24 -0400 Date: Wed, 19 Jul 2017 12:14:12 +1000 From: Nicholas Piggin To: "Gautham R. Shenoy" Cc: Michael Ellerman , Michael Neuling , Vaidyanathan Srinivasan , Shilpasri G Bhat , Akshay Adiga , linuxppc-dev@lists.ozlabs.org, linux-kernel@vger.kernel.org Subject: Re: [PATCH 2/2] powernv/powerpc: Clear PECE1 in LPCR via stop-api only on Hotplug Message-ID: <20170719121412.1d89857c@roar.ozlabs.ibm.com> In-Reply-To: References: Organization: IBM X-Mailer: Claws Mail 3.14.1 (GTK+ 2.24.31; x86_64-pc-linux-gnu) MIME-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Content-Length: 4338 Lines: 115 Thanks for working on these patches. We really need to get this stuff merged and tested asap :) On Tue, 18 Jul 2017 19:58:49 +0530 "Gautham R. Shenoy" wrote: > From: "Gautham R. Shenoy" > > > Currently we use the stop-api provided by the firmware to program the > SLW engine to restore the values of hypervisor resources that get lost > on deeper idle states (such as winkle). Since the deep states were > only used for CPU-Hotplug on POWER8 systems, we would program the LPCR > to have the PECE1 bit since Hotplugged CPUs shouldn't be spuriously > woken up by decrementer. > > On POWER9, some of the deep platform idle states such as stop4 can be > used in cpuidle as well. In this case, we want the CPU in stop4 to be > woken up by the decrementer when some timer on the CPU expires. > > In this patch, for POWER9, we program the stop-api for LPCR with PECE1 > bit cleared only when we are offlining the CPU. > > Signed-off-by: Gautham R. Shenoy > --- > arch/powerpc/platforms/powernv/idle.c | 12 +++++++++++- > arch/powerpc/platforms/powernv/smp.c | 28 ++++++++++++++++++++++++---- > 2 files changed, 35 insertions(+), 5 deletions(-) > > diff --git a/arch/powerpc/platforms/powernv/idle.c b/arch/powerpc/platforms/powernv/idle.c > index 2abee07..f4a29d4 100644 > --- a/arch/powerpc/platforms/powernv/idle.c > +++ b/arch/powerpc/platforms/powernv/idle.c > @@ -68,7 +68,7 @@ static int pnv_save_sprs_for_deep_states(void) > * all cpus at boot. Get these reg values of current cpu and use the > * same across all cpus. > */ > - uint64_t lpcr_val = mfspr(SPRN_LPCR) & ~(u64)LPCR_PECE1; > + uint64_t lpcr_val = mfspr(SPRN_LPCR); > uint64_t hid0_val = mfspr(SPRN_HID0); > uint64_t hid1_val = mfspr(SPRN_HID1); > uint64_t hid4_val = mfspr(SPRN_HID4); > @@ -85,6 +85,16 @@ static int pnv_save_sprs_for_deep_states(void) > if (rc != 0) > return rc; > > + /* > + * On POWER8, the only state that uses SLW engine is > + * winkle. This is only used for CPU-Hotplug. So we > + * clear the decrementer bit from LPCR since we > + * don't want to be woken up on decrementer when in > + * winkle. > + */ > + if (!cpu_has_feature(CPU_FTR_ARCH_300)) > + lpcr_val = lpcr_val & ~(u64)LPCR_PECE1; > + > rc = opal_slw_set_reg(pir, SPRN_LPCR, lpcr_val); > if (rc != 0) > return rc; > diff --git a/arch/powerpc/platforms/powernv/smp.c b/arch/powerpc/platforms/powernv/smp.c > index 40dae96..5f2a712 100644 > --- a/arch/powerpc/platforms/powernv/smp.c > +++ b/arch/powerpc/platforms/powernv/smp.c > @@ -143,7 +143,8 @@ static void pnv_smp_cpu_kill_self(void) > { > unsigned int cpu; > unsigned long srr1, wmask; > - > + uint64_t lpcr_val; > + uint64_t pir; > /* Standard hot unplug procedure */ > /* > * This hard disables local interurpts, ensuring we have no lazy > @@ -164,13 +165,17 @@ static void pnv_smp_cpu_kill_self(void) > if (cpu_has_feature(CPU_FTR_ARCH_207S)) > wmask = SRR1_WAKEMASK_P8; > > + pir = get_hard_smp_processor_id(cpu); > /* We don't want to take decrementer interrupts while we are offline, > * so clear LPCR:PECE1. We keep PECE2 (and LPCR_PECE_HVEE on P9) > * enabled as to let IPIs in. > */ > - mtspr(SPRN_LPCR, mfspr(SPRN_LPCR) & ~(u64)LPCR_PECE1); > + lpcr_val = mfspr(SPRN_LPCR) & ~(u64)LPCR_PECE1; > + mtspr(SPRN_LPCR, lpcr_val); > > while (!generic_check_cpu_restart(cpu)) { > + > + > /* > * Clear IPI flag, since we don't handle IPIs while > * offline, except for those when changing micro-threading > @@ -180,8 +185,15 @@ static void pnv_smp_cpu_kill_self(void) > */ > kvmppc_set_host_ipi(cpu, 0); > > + /* > + * If the CPU gets woken up by a special wakeup, > + * ensure that the SLW engine sets LPCR with > + * decrementer bit cleared, else we will get spurious > + * wakeups. > + */ > + if (cpu_has_feature(CPU_FTR_ARCH_300)) > + opal_slw_set_reg(pir, SPRN_LPCR, lpcr_val); Can you put these details into pnv_cpu_offline? Possibly even from there into another special SPR save function? E.g., pnv_save_sprs_for_deep_state_decrementer_wakeup(bool decrementer_wakeup) I'd like to put the LPCR manipulation for idle wake settings into idle.c as well (pnv_cpu_offline), I think it fits better in there. Thanks, Nick