Received: by 2002:a05:6a10:5bc5:0:0:0:0 with SMTP id os5csp1580618pxb; Tue, 26 Oct 2021 11:38:20 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxQsrF182cF7bcV0gbVcKnyyLwMkxAKkzjrjtnedSU42Zo9jTwFHbZn1rgtLOJQI7oGCqpc X-Received: by 2002:a17:90a:1548:: with SMTP id y8mr449105pja.151.1635273500693; Tue, 26 Oct 2021 11:38:20 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1635273500; cv=none; d=google.com; s=arc-20160816; b=Q+CW/GwTZOTtjw+pOC6EAa0TakH4yOZgfAwJTYrq1dAIZa4+qhjAIYO2msX1P1koRR vzrrGWSHzyoPfTw3TyCQuueZxeMAu1aRY+O9ZfLgpm/CbL8QLApkw0iGzfVbvd5qqaHD zQDg30ENBxk9nZQU1y/CtvPhh1D9kF/1wzbJhHiw6GoQwRl4YQ8Fqod00H97AkZ2J2Ce YfokoqMLOFxRWBWIt/nxrDfHn9Z5sRK0srLTKjQ/QBrB8Mp6xUeYjoD+ITtFEkZahPXw 6LL3aqCAWkRaWPskvyZcuJlUe1tJYZSfULtHiGzO9h9uQq0C5mb/19e2gT1IDdRkgUzO 7L4Q== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=CRFKrmGwY5n2TXCZOXai744qmHmWAyuISZC1rBVd2oI=; b=OoPa5zEHC/pElGyoFsAIVBDOJrNZ5w/ez2kDEAKo5tn2bc02j96+42xusEo0Uflxlo 1m4XJEaXgbVbSjFetblDsiS9MTTo3eyGlFsNGa2nNi1pkhqvxFdf5RLOVLWCW4p+2Wg7 vZ+U74Q+barRy2MoDoc2MWDz/Ki/0NOiSmB8slbvCZ/mCfaS+M74l+hTyFkckzIOkE7E 4wcg3LZfqKqy1/PFYCMdZzmx6sAenJ9sLnSfYU1CBzA8l+kViGOLgL/dxST/ls/aNC1M 4BG0AuGSc4lIqMmHc+5wNQo0l3DlIKaZJWX9evDxw5itmDxRyR0moBIPBjGr0Z9tUs6W a/8w== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b="bXJqex/5"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [23.128.96.18]) by mx.google.com with ESMTP id m16si2838661pgu.423.2021.10.26.11.38.07; Tue, 26 Oct 2021 11:38:20 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18; Authentication-Results: mx.google.com; dkim=pass header.i=@gmail.com header.s=20210112 header.b="bXJqex/5"; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=QUARANTINE dis=NONE) header.from=gmail.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S236739AbhJZOif (ORCPT + 99 others); Tue, 26 Oct 2021 10:38:35 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:57014 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S236777AbhJZOiR (ORCPT ); Tue, 26 Oct 2021 10:38:17 -0400 Received: from mail-pf1-x431.google.com (mail-pf1-x431.google.com [IPv6:2607:f8b0:4864:20::431]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 74110C061348 for ; Tue, 26 Oct 2021 07:35:53 -0700 (PDT) Received: by mail-pf1-x431.google.com with SMTP id a26so3708944pfr.11 for ; Tue, 26 Oct 2021 07:35:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112; h=from:to:cc:subject:date:message-id:in-reply-to:references :mime-version:content-transfer-encoding; bh=CRFKrmGwY5n2TXCZOXai744qmHmWAyuISZC1rBVd2oI=; b=bXJqex/5xyUspH70qPE8HsYV6QVZTHknr7piy4gEU/TGqExgvDfY20YFp6e8o4Nd25 NJnDzfsHmheWW5HLSXlKRBJYzxLlI1gzxn7r5ypzq17GUB5jFLN6S2A9QW52y1jbIyny 99J+xaSL1DiDW69CIgmdc9F3sNp1qEziYabRYtlr1sNR2uAv2w/CTsLSfKROnEDZjC2q dKHDvtr/xlIla9PFq1iLw3NX9fND1wa+cf5skchP9saxiueqqGwUrq4+Tfz/s6NtVMfB eQ8x1uQtV6kcK7XuJ1XH/YA84qTAAKFzutWDWxPpF7/aNLBv38h+XH1RysCvPeBXEpaM csjg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=CRFKrmGwY5n2TXCZOXai744qmHmWAyuISZC1rBVd2oI=; b=3hnDm2V6sn9W/CjOmNJszbKJSL6DUmblazzsAL2GLdkgNGP9cC0Kku8+QwnEf0ziko pPCyboZEt3SyCcy2df55oesmM7ZYarL3oT/FIaZ63+G6jmoy/WDrPNwpUi7cQM2haIky XV/kEx5d4TtWsSZ4Js2WTQX5IiDwnLzMQWmyDT4FKEtOL5Y2jjz2j98LzXTryJNziIg+ sifoDpZhG1+72Grn+GKn1C1iAZHP+2jGwd4kn2po3//OWUOCB0MLCACaV+rU82+DNzDp Jnm+mqUBktt1FjfRuMyuUhcRlftAFatQ9c7jtLqVx5W9p3Tf+19N5CuzFMyItQbscPsh VE4g== X-Gm-Message-State: AOAM530FdD4Gbfc1W2PvVGZQ3FdqzBR58Hlyf5yS9oBeGeqiauKCQm8z woWcvdMdmxa57koGHlR8qKBNLjezWaU= X-Received: by 2002:aa7:8c0d:0:b0:47b:dc76:38c1 with SMTP id c13-20020aa78c0d000000b0047bdc7638c1mr20847868pfd.22.1635258952828; Tue, 26 Oct 2021 07:35:52 -0700 (PDT) Received: from localhost ([47.88.5.130]) by smtp.gmail.com with ESMTPSA id me18sm1108820pjb.33.2021.10.26.07.35.51 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128); Tue, 26 Oct 2021 07:35:52 -0700 (PDT) From: Lai Jiangshan To: linux-kernel@vger.kernel.org Cc: x86@kernel.org, Lai Jiangshan , Andy Lutomirski , Thomas Gleixner , Ingo Molnar , Borislav Petkov , Dave Hansen , "H. Peter Anvin" Subject: [PATCH V4 35/50] x86/entry: Add the C version ist_switch_to_kernel_gsbase() Date: Tue, 26 Oct 2021 22:34:21 +0800 Message-Id: <20211026143436.19071-10-jiangshanlai@gmail.com> X-Mailer: git-send-email 2.19.1.6.gb485710b In-Reply-To: <20211026141420.17138-1-jiangshanlai@gmail.com> References: <20211026141420.17138-1-jiangshanlai@gmail.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Lai Jiangshan It implements the second half of paranoid_entry() whose functionality is to switch to kernel gsbase. Not functional difference intended. Signed-off-by: Lai Jiangshan --- arch/x86/entry/entry64.c | 51 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/arch/x86/entry/entry64.c b/arch/x86/entry/entry64.c index 8f3a9f2d9c6a..448b9947ed9c 100644 --- a/arch/x86/entry/entry64.c +++ b/arch/x86/entry/entry64.c @@ -223,3 +223,54 @@ static __always_inline unsigned long get_percpu_base(void) return pcpu_unit_offsets; } #endif + +/* + * Handle GSBASE depends on the availability of FSGSBASE. + * + * Without FSGSBASE the kernel enforces that negative GSBASE + * values indicate kernel GSBASE. With FSGSBASE no assumptions + * can be made about the GSBASE value when entering from user + * space. + */ +static __always_inline unsigned long ist_switch_to_kernel_gsbase(void) +{ + unsigned long gsbase; + + if (static_cpu_has(X86_FEATURE_FSGSBASE)) { + /* + * Read the current GSBASE for return. + * Retrieve and set the current CPUs kernel GSBASE. + * + * The unconditional write to GS base below ensures that + * no subsequent loads based on a mispredicted GS base can + * happen, therefore no LFENCE is needed here. + */ + gsbase = rdgsbase(); + wrgsbase(get_percpu_base()); + return gsbase; + } + + gsbase = __rdmsr(MSR_GS_BASE); + + /* + * The kernel-enforced convention is a negative GSBASE indicates + * a kernel value. No SWAPGS needed on entry and exit. + */ + if ((long)gsbase < 0) { + fence_swapgs_kernel_entry(); + /* no SWAPGS required on exit */ + return 1; + } + + /* + * SWAPGS is needed. + * + * The above ist_switch_to_kernel_cr3() doesn't do an unconditional + * CR3 write, even in the PTI case. So do an lfence to prevent GS + * speculation, regardless of whether PTI is enabled. + */ + kernel_entry_swapgs_and_fence(); + + /* SWAPGS required on exit */ + return 0; +} -- 2.19.1.6.gb485710b