Received: by 2002:a25:6193:0:0:0:0:0 with SMTP id v141csp296169ybb; Fri, 3 Apr 2020 02:56:10 -0700 (PDT) X-Google-Smtp-Source: APiQypJ/gJwOvAy1fJwu+1xXj7LWmtjZvBCI6Hk/oDyXt8B8+OWIFDyIuC7L4FYT/vaC5v9D/p7f X-Received: by 2002:a05:6830:17ca:: with SMTP id p10mr5859796ota.165.1585907770175; Fri, 03 Apr 2020 02:56:10 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1585907770; cv=none; d=google.com; s=arc-20160816; b=xTTDVLwZ/aA/ONpMHVU28jenOxSHJWViZ2FdkMjVrGwFzGNxtHkxARJw1EEfNFZ3Fl mFpjOjMwN0kZJUIF/6HD3XnjPjOG82/Ad6bK6JPpmoMt8l7pme6hb9adJ54rFE+QAUUw tYEwEmNDBp/H4SESsCmKsocI7pW2+61Uy5bi5tu6DdA0LshgOsHhK3JCZyb2Fsa9kWqb aWu8BQdDfhqQgMu4ys99R0iVw0+lzV+oJd5j5AQwkEnnUhXzqqVtXDnPIw+dz1JslFgS ojGBiCA2qkhIp2jUDBRr0HJP5CIAz0U4IBjGb/wJaq6tsyhbxDTzTRguZ2dWXa8DJ/ra BH9g== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:message-id:date:subject:cc:to:from; bh=NknkQdbz9UYUsOQLvf3+jamqRZX4HThmHVoupeix7Ic=; b=NBOEOs9ixJg8syyenBcreQjBpDUUEDNZxa+Wu2u5lpJnsLTyfZLOLT3iAUD2BACJnD omj5TR9y2iHtJvU95tWdly/fNU3o5tBw6pg29XsN5/qvGqcXMztdRJ2/ZqJ8rL5ByYD8 M+ecorjhEKP7ebqJ5DeejrOcN147mIVVoBrUkaMqDPZldJJn1IuwPI4ae+BP5Dl8jztt PFj3pnx5c50q8+eZ7P8TMESrIIK2iP68pkgd2erThjHNtGKLX44zGZyuMXlKw5a1zQhw 2/Qy9zecj2PM/VG26pGV1f7OaZ+b4yRnmlSfNjTE0jOqWftudPIUR/yY1oYkuoTsoE2W JsCg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id n64si3337843oih.256.2020.04.03.02.55.57; Fri, 03 Apr 2020 02:56:10 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=ibm.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S2390291AbgDCJ1f (ORCPT + 99 others); Fri, 3 Apr 2020 05:27:35 -0400 Received: from mx0b-001b2d01.pphosted.com ([148.163.158.5]:20170 "EHLO mx0a-001b2d01.pphosted.com" rhost-flags-OK-OK-OK-FAIL) by vger.kernel.org with ESMTP id S1727774AbgDCJ1f (ORCPT ); Fri, 3 Apr 2020 05:27:35 -0400 Received: from pps.filterd (m0098416.ppops.net [127.0.0.1]) by mx0b-001b2d01.pphosted.com (8.16.0.42/8.16.0.42) with SMTP id 03393atN172042 for ; Fri, 3 Apr 2020 05:27:33 -0400 Received: from e06smtp01.uk.ibm.com (e06smtp01.uk.ibm.com [195.75.94.97]) by mx0b-001b2d01.pphosted.com with ESMTP id 302072cskx-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Fri, 03 Apr 2020 05:27:33 -0400 Received: from localhost by e06smtp01.uk.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Fri, 3 Apr 2020 10:27:14 +0100 Received: from b06cxnps3074.portsmouth.uk.ibm.com (9.149.109.194) by e06smtp01.uk.ibm.com (192.168.101.131) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Fri, 3 Apr 2020 10:27:12 +0100 Received: from d06av22.portsmouth.uk.ibm.com (d06av22.portsmouth.uk.ibm.com [9.149.105.58]) by b06cxnps3074.portsmouth.uk.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id 0339RSJP66257150 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=OK); Fri, 3 Apr 2020 09:27:28 GMT Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id E47FA4C050; Fri, 3 Apr 2020 09:27:27 +0000 (GMT) Received: from d06av22.portsmouth.uk.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 67C9B4C044; Fri, 3 Apr 2020 09:27:26 +0000 (GMT) Received: from ltc-wspoon6.aus.stglabs.ibm.com (unknown [9.40.193.95]) by d06av22.portsmouth.uk.ibm.com (Postfix) with ESMTP; Fri, 3 Apr 2020 09:27:26 +0000 (GMT) From: Abhishek Goel To: linuxppc-dev@ozlabs.org, linux-kernel@vger.kernel.org Cc: npiggin@gmail.com, ego@linux.vnet.ibm.com, svaidy@linux.ibm.com, mpe@ellerman.id.au, oohall@gmail.com, mikey@neuling.org, psampat@linux.ibm.com, Abhishek Goel Subject: [RFC] cpuidle/powernv : Support for pre-entry and post exit of stop state in firmware Date: Fri, 3 Apr 2020 04:27:01 -0500 X-Mailer: git-send-email 2.17.1 X-TM-AS-GCONF: 00 x-cbid: 20040309-4275-0000-0000-000003B8C402 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 20040309-4276-0000-0000-000038CE1CAB Message-Id: <20200403092701.39751-1-huntbag@linux.vnet.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:6.0.138,18.0.676 definitions=2020-04-03_05:2020-04-02,2020-04-03 signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 clxscore=1011 malwarescore=0 spamscore=0 lowpriorityscore=0 adultscore=0 mlxlogscore=866 suspectscore=0 bulkscore=0 phishscore=0 priorityscore=1501 mlxscore=0 impostorscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.12.0-2003020000 definitions=main-2004030073 Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org This patch provides kernel framework fro opal support of save restore of sprs in idle stop loop. Opal support for stop states is needed to selectively enable stop states or to introduce a quirk quickly in case a buggy stop state is present. We make a opal call from kernel if firmware-stop-support for stop states is present and enabled. All the quirks for pre-entry of stop state is handled inside opal. A call from opal is made into kernel where we execute stop afer saving of NVGPRs. After waking up from 0x100 vector in kernel, we enter back into opal. All the quirks in post exit path, if any, are then handled in opal, from where we return successfully back to kernel. For deep stop states in which additional SPRs are lost, saving and restoration will be done in OPAL. This idea was first proposed by Nick here: https://patchwork.ozlabs.org/patch/1208159/ The corresponding skiboot patch for this kernel patch is here: https://patchwork.ozlabs.org/patch/1265959/ When we callback from OPAL into kernel, r13 is clobbered. So, to access PACA we need to restore it from HSPRGO. In future we can handle this into OPAL as in here: https://patchwork.ozlabs.org/patch/1245275/ Signed-off-by: Abhishek Goel Signed-off-by: Nicholas Piggin --- arch/powerpc/include/asm/opal-api.h | 8 ++++- arch/powerpc/include/asm/opal.h | 3 ++ arch/powerpc/kernel/idle_book3s.S | 5 +++ arch/powerpc/platforms/powernv/idle.c | 37 ++++++++++++++++++++++ arch/powerpc/platforms/powernv/opal-call.c | 2 ++ 5 files changed, 54 insertions(+), 1 deletion(-) diff --git a/arch/powerpc/include/asm/opal-api.h b/arch/powerpc/include/asm/opal-api.h index c1f25a760eb1..a2c782c99c9e 100644 --- a/arch/powerpc/include/asm/opal-api.h +++ b/arch/powerpc/include/asm/opal-api.h @@ -214,7 +214,9 @@ #define OPAL_SECVAR_GET 176 #define OPAL_SECVAR_GET_NEXT 177 #define OPAL_SECVAR_ENQUEUE_UPDATE 178 -#define OPAL_LAST 178 +#define OPAL_REGISTER_OS_OPS 181 +#define OPAL_CPU_IDLE 182 +#define OPAL_LAST 182 #define QUIESCE_HOLD 1 /* Spin all calls at entry */ #define QUIESCE_REJECT 2 /* Fail all calls with OPAL_BUSY */ @@ -1181,6 +1183,10 @@ struct opal_mpipl_fadump { struct opal_mpipl_region region[]; } __packed; +struct opal_os_ops { + __be64 os_idle_stop; +}; + #endif /* __ASSEMBLY__ */ #endif /* __OPAL_API_H */ diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h index 9986ac34b8e2..3c340bc4df8e 100644 --- a/arch/powerpc/include/asm/opal.h +++ b/arch/powerpc/include/asm/opal.h @@ -400,6 +400,9 @@ void opal_powercap_init(void); void opal_psr_init(void); void opal_sensor_groups_init(void); +extern int64_t opal_register_os_ops(struct opal_os_ops *os_ops); +extern int64_t opal_cpu_idle(__be64 srr1_addr, uint64_t psscr); + #endif /* __ASSEMBLY__ */ #endif /* _ASM_POWERPC_OPAL_H */ diff --git a/arch/powerpc/kernel/idle_book3s.S b/arch/powerpc/kernel/idle_book3s.S index 22f249b6f58d..8d287d1d06c0 100644 --- a/arch/powerpc/kernel/idle_book3s.S +++ b/arch/powerpc/kernel/idle_book3s.S @@ -49,6 +49,8 @@ _GLOBAL(isa300_idle_stop_noloss) */ _GLOBAL(isa300_idle_stop_mayloss) mtspr SPRN_PSSCR,r3 + mr r6, r13 + mfspr r13, SPRN_HSPRG0 std r1,PACAR1(r13) mflr r4 mfcr r5 @@ -74,6 +76,7 @@ _GLOBAL(isa300_idle_stop_mayloss) std r31,-8*18(r1) std r4,-8*19(r1) std r5,-8*20(r1) + std r6,-8*21(r1) /* 168 bytes */ PPC_STOP b . /* catch bugs */ @@ -91,8 +94,10 @@ _GLOBAL(idle_return_gpr_loss) ld r1,PACAR1(r13) ld r4,-8*19(r1) ld r5,-8*20(r1) + ld r6,-8*21(r1) mtlr r4 mtcr r5 + mr r13,r6 /* * KVM nap requires r2 to be saved, rather than just restoring it * from PACATOC. This could be avoided for that less common case diff --git a/arch/powerpc/platforms/powernv/idle.c b/arch/powerpc/platforms/powernv/idle.c index 78599bca66c2..1841027b25c5 100644 --- a/arch/powerpc/platforms/powernv/idle.c +++ b/arch/powerpc/platforms/powernv/idle.c @@ -35,6 +35,7 @@ static u32 supported_cpuidle_states; struct pnv_idle_states_t *pnv_idle_states; int nr_pnv_idle_states; +static bool firmware_stop_supported; /* * The default stop state that will be used by ppc_md.power_save @@ -602,6 +603,25 @@ struct p9_sprs { u64 uamor; }; +/* + * This function is called from OPAL if firmware support for stop + * states is present and enabled. It provides a fallback for idle + * stop states via OPAL. + */ +static uint64_t os_idle_stop(uint64_t psscr, bool save_gprs) +{ + /* + * For lite state which does not lose even GPRS we call + * idle_stop_noloss while for all other states we call + * idle_stop_mayloss. Saving and restoration of other additional + * SPRs if required is handled in OPAL. All the quirks are also + * handled in OPAL. + */ + if (!save_gprs) + return isa300_idle_stop_noloss(psscr); + return isa300_idle_stop_mayloss(psscr); +} + static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on) { int cpu = raw_smp_processor_id(); @@ -613,6 +633,16 @@ static unsigned long power9_idle_stop(unsigned long psscr, bool mmu_on) unsigned long mmcr0 = 0; struct p9_sprs sprs = {}; /* avoid false used-uninitialised */ bool sprs_saved = false; + int rc = 0; + + /* + * Kernel takes decision whether to make OPAL call or not. This logic + * will be combined with the logic for BE opal to take decision. + */ + if (firmware_stop_supported) { + rc = opal_cpu_idle(cpu_to_be64(__pa(&srr1)), (uint64_t) psscr); + goto out; + } if (!(psscr & (PSSCR_EC|PSSCR_ESL))) { /* EC=ESL=0 case */ @@ -1232,6 +1262,10 @@ static int pnv_parse_cpuidle_dt(void) pr_warn("opal: PowerMgmt Node not found\n"); return -ENODEV; } + + if (of_device_is_compatible(np, "firmware-stop-supported")) + firmware_stop_supported = true; + nr_idle_states = of_property_count_u32_elems(np, "ibm,cpu-idle-state-flags"); @@ -1326,6 +1360,7 @@ static int pnv_parse_cpuidle_dt(void) static int __init pnv_init_idle_states(void) { + struct opal_os_ops os_ops; int cpu; int rc = 0; @@ -1349,6 +1384,8 @@ static int __init pnv_init_idle_states(void) } } + os_ops.os_idle_stop = be64_to_cpu(os_idle_stop); + rc = opal_register_os_ops((struct opal_os_ops *)(&os_ops)); /* In case we error out nr_pnv_idle_states will be zero */ nr_pnv_idle_states = 0; supported_cpuidle_states = 0; diff --git a/arch/powerpc/platforms/powernv/opal-call.c b/arch/powerpc/platforms/powernv/opal-call.c index 5cd0f52d258f..c885e607ba62 100644 --- a/arch/powerpc/platforms/powernv/opal-call.c +++ b/arch/powerpc/platforms/powernv/opal-call.c @@ -293,3 +293,5 @@ OPAL_CALL(opal_mpipl_query_tag, OPAL_MPIPL_QUERY_TAG); OPAL_CALL(opal_secvar_get, OPAL_SECVAR_GET); OPAL_CALL(opal_secvar_get_next, OPAL_SECVAR_GET_NEXT); OPAL_CALL(opal_secvar_enqueue_update, OPAL_SECVAR_ENQUEUE_UPDATE); +OPAL_CALL(opal_register_os_ops, OPAL_REGISTER_OS_OPS); +OPAL_CALL(opal_cpu_idle, OPAL_CPU_IDLE); -- 2.17.1