Received: by 2002:a25:d7c1:0:0:0:0:0 with SMTP id o184csp2257026ybg; Sun, 27 Oct 2019 14:32:37 -0700 (PDT) X-Google-Smtp-Source: APXvYqysxSFw37FMKnKtqtucVIOGBdhBVeWP4qbx5NoHDjXIEMT2NRdOA7upzA11oG1m68MDZPfM X-Received: by 2002:a05:6402:689:: with SMTP id f9mr13865370edy.79.1572211957811; Sun, 27 Oct 2019 14:32:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1572211957; cv=none; d=google.com; s=arc-20160816; b=fvnfKKkSLcG96eILf5KEafp+Vzq+19P7U3sxzdZa280ARB7ujeLRLKcrwYUhl8sQvM p8ZCeEev/+aTbAMitz6Lu08sI3axxD93b63Hi9kihsSk6u+87WvaZoPDiQsiJrnBnxBH Uuwj9w95daIBQO7W1FWJpax2pKTf1vVTLRBKBhVurDE/sPNmyb+kyWV4axFBqqpgWRky doQthze0Kkndmqfssxp7FtBZ0PhaQf6ExPU6gIjeEULWizN+6PdcUjxFVNturXQFt48q lj2uDEANEvYl/ZiN7rTyOpq32bm0xCgy1i2YJPgs5gUcMWavrVKC4swPSxchUpW2memo OblA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:content-transfer-encoding:mime-version :user-agent:references:in-reply-to:message-id:date:subject:cc:to :from:dkim-signature; bh=eL2I+54Zcok9O3pMiNElgQ7uirM3Zy3MUnN/9u/NFao=; b=fDtQrQsg/tbKI8Pr5ok6aoiqeL/JsQ3c7GqiAtoAoHKZWd2h3rfYWROYMWNZQAVjsg 5nO0kEd09hU4sKyEgeJ5voWzssB8ka9gz633d6flOP3plDc5AfWxChCua0sW07HFLbIW u/q7TLKxCKTm2Bb9b/L51s0MAO32CkYDJdCpTAZP1wfi3e09jZJfuvlN4dLwerDS25GH MYQFZb6Z7gLaW0dpUpx5P9yy9eb3FW3V9ZqjeJjb1AhhkMlrMNZWJrlIDWqNQJjcjXnh sGovb+m+BfYFsFal7XQOsbE2R1itv7011VMJP2Wz2DV8R80E00CtOE+Rv+KTAmyrPtrz YS6A== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=OtSQpCOn; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id f36si6026516ede.159.2019.10.27.14.32.14; Sun, 27 Oct 2019 14:32:37 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; dkim=pass header.i=@kernel.org header.s=default header.b=OtSQpCOn; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1732032AbfJ0VV4 (ORCPT + 99 others); Sun, 27 Oct 2019 17:21:56 -0400 Received: from mail.kernel.org ([198.145.29.99]:42816 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1732018AbfJ0VVy (ORCPT ); Sun, 27 Oct 2019 17:21:54 -0400 Received: from localhost (100.50.158.77.rev.sfr.net [77.158.50.100]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPSA id F19842070B; Sun, 27 Oct 2019 21:21:51 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=default; t=1572211312; bh=GB5Y1RO3WZ4j6evENjy3M0atHRNjnllqBD/BTpyO7YM=; h=From:To:Cc:Subject:Date:In-Reply-To:References:From; b=OtSQpCOn9wCkHU0ts4pB0dvdOyz2eBcgIFmcZ1B41ndiQtZ4AdLobdNMi3qgkwuzs c3L7AM/efn4fEpdKUUVP2UGAa8Uyl0pLkw/wEzPzl918dJ5PevnYaxB8UW/g+1PGA7 EyYJlflE6dLoxoHXhQU4ERSkJjfXAdIzihaNzeik= From: Greg Kroah-Hartman To: linux-kernel@vger.kernel.org Cc: Greg Kroah-Hartman , stable@vger.kernel.org, Paul Burton , Dmitry Korotin , linux-mips@vger.kernel.org Subject: [PATCH 5.3 107/197] MIPS: tlbex: Fix build_restore_pagemask KScratch restore Date: Sun, 27 Oct 2019 22:00:25 +0100 Message-Id: <20191027203357.532696395@linuxfoundation.org> X-Mailer: git-send-email 2.23.0 In-Reply-To: <20191027203351.684916567@linuxfoundation.org> References: <20191027203351.684916567@linuxfoundation.org> User-Agent: quilt/0.66 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Paul Burton commit b42aa3fd5957e4daf4b69129e5ce752a2a53e7d6 upstream. build_restore_pagemask() will restore the value of register $1/$at when its restore_scratch argument is non-zero, and aims to do so by filling a branch delay slot. Commit 0b24cae4d535 ("MIPS: Add missing EHB in mtc0 -> mfc0 sequence.") added an EHB instruction (Execution Hazard Barrier) prior to restoring $1 from a KScratch register, in order to resolve a hazard that can result in stale values of the KScratch register being observed. In particular, P-class CPUs from MIPS with out of order execution pipelines such as the P5600 & P6600 are affected. Unfortunately this EHB instruction was inserted in the branch delay slot causing the MFC0 instruction which performs the restoration to no longer execute along with the branch. The result is that the $1 register isn't actually restored, ie. the TLB refill exception handler clobbers it - which is exactly the problem the EHB is meant to avoid for the P-class CPUs. Similarly build_get_pgd_vmalloc() will restore the value of $1/$at when its mode argument equals refill_scratch, and suffers from the same problem. Fix this by in both cases moving the EHB earlier in the emitted code. There's no reason it needs to immediately precede the MFC0 - it simply needs to be between the MTC0 & MFC0. This bug only affects Cavium Octeon systems which use build_fast_tlb_refill_handler(). Signed-off-by: Paul Burton Fixes: 0b24cae4d535 ("MIPS: Add missing EHB in mtc0 -> mfc0 sequence.") Cc: Dmitry Korotin Cc: stable@vger.kernel.org # v3.15+ Cc: linux-mips@vger.kernel.org Cc: linux-kernel@vger.kernel.org Signed-off-by: Greg Kroah-Hartman --- arch/mips/mm/tlbex.c | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-) --- a/arch/mips/mm/tlbex.c +++ b/arch/mips/mm/tlbex.c @@ -655,6 +655,13 @@ static void build_restore_pagemask(u32 * int restore_scratch) { if (restore_scratch) { + /* + * Ensure the MFC0 below observes the value written to the + * KScratch register by the prior MTC0. + */ + if (scratch_reg >= 0) + uasm_i_ehb(p); + /* Reset default page size */ if (PM_DEFAULT_MASK >> 16) { uasm_i_lui(p, tmp, PM_DEFAULT_MASK >> 16); @@ -669,12 +676,10 @@ static void build_restore_pagemask(u32 * uasm_i_mtc0(p, 0, C0_PAGEMASK); uasm_il_b(p, r, lid); } - if (scratch_reg >= 0) { - uasm_i_ehb(p); + if (scratch_reg >= 0) UASM_i_MFC0(p, 1, c0_kscratch(), scratch_reg); - } else { + else UASM_i_LW(p, 1, scratchpad_offset(0), 0); - } } else { /* Reset default page size */ if (PM_DEFAULT_MASK >> 16) { @@ -923,6 +928,10 @@ build_get_pgd_vmalloc64(u32 **p, struct } if (mode != not_refill && check_for_high_segbits) { uasm_l_large_segbits_fault(l, *p); + + if (mode == refill_scratch && scratch_reg >= 0) + uasm_i_ehb(p); + /* * We get here if we are an xsseg address, or if we are * an xuseg address above (PGDIR_SHIFT+PGDIR_BITS) boundary. @@ -941,12 +950,10 @@ build_get_pgd_vmalloc64(u32 **p, struct uasm_i_jr(p, ptr); if (mode == refill_scratch) { - if (scratch_reg >= 0) { - uasm_i_ehb(p); + if (scratch_reg >= 0) UASM_i_MFC0(p, 1, c0_kscratch(), scratch_reg); - } else { + else UASM_i_LW(p, 1, scratchpad_offset(0), 0); - } } else { uasm_i_nop(p); }