Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 23.128.96.18 as permitted sender) client-ip=23.128.96.18;
From:   Rick Edgecombe <rick.p.edgecombe@intel.com>
To:     x86@kernel.org, "H . Peter Anvin" <hpa@zytor.com>,
        Thomas Gleixner <tglx@linutronix.de>,
        Ingo Molnar <mingo@redhat.com>, linux-kernel@vger.kernel.org,
        linux-doc@vger.kernel.org, linux-mm@kvack.org,
        linux-arch@vger.kernel.org, linux-api@vger.kernel.org,
        Arnd Bergmann <arnd@arndb.de>,
        Andy Lutomirski <luto@kernel.org>,
        Balbir Singh <bsingharora@gmail.com>,
        Borislav Petkov <bp@alien8.de>,
        Cyrill Gorcunov <gorcunov@gmail.com>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Eugene Syromiatnikov <esyr@redhat.com>,
        Florian Weimer <fweimer@redhat.com>,
        "H . J . Lu" <hjl.tools@gmail.com>, Jann Horn <jannh@google.com>,
        Jonathan Corbet <corbet@lwn.net>,
        Kees Cook <keescook@chromium.org>,
        Mike Kravetz <mike.kravetz@oracle.com>,
        Nadav Amit <nadav.amit@gmail.com>,
        Oleg Nesterov <oleg@redhat.com>, Pavel Machek <pavel@ucw.cz>,
        Peter Zijlstra <peterz@infradead.org>,
        Randy Dunlap <rdunlap@infradead.org>,
        "Ravi V . Shankar" <ravi.v.shankar@intel.com>,
        Dave Martin <Dave.Martin@arm.com>,
        Weijiang Yang <weijiang.yang@intel.com>,
        "Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>,
        joao.moreira@intel.com, John Allen <john.allen@amd.com>,
        kcc@google.com, eranian@google.com
Cc:     rick.p.edgecombe@intel.com
Subject: [PATCH 27/35] x86/fpu: Add unsafe xsave buffer helpers
Date:   Sun, 30 Jan 2022 13:18:30 -0800
Message-Id: <20220130211838.8382-28-rick.p.edgecombe@intel.com>
In-Reply-To: <20220130211838.8382-1-rick.p.edgecombe@intel.com>
References: <20220130211838.8382-1-rick.p.edgecombe@intel.com>
Precedence: bulk

CET will need to modify the xsave buffer of a new FPU that was just
created in the process of copying a thread. In this case the normal
helpers will not work, because they operate on the current thread's FPU.

So add unsafe helpers to allow for this kind of modification. Make the
unsafe helpers operate on the MSR like the safe helpers for symmetry and
to avoid exposing the underling xsave structures. Don't add a read
helper because it is not needed at this time.

Signed-off-by: Rick Edgecombe <rick.p.edgecombe@intel.com>
---
 arch/x86/include/asm/fpu/api.h |  9 ++++++---
 arch/x86/kernel/fpu/xstate.c   | 27 ++++++++++++++++++++++-----
 2 files changed, 28 insertions(+), 8 deletions(-)

diff --git a/arch/x86/include/asm/fpu/api.h b/arch/x86/include/asm/fpu/api.h
index 6aec27984b62..5cb557b9d118 100644
--- a/arch/x86/include/asm/fpu/api.h
+++ b/arch/x86/include/asm/fpu/api.h
@@ -167,7 +167,10 @@ extern long fpu_xstate_prctl(struct task_struct *tsk, int option, unsigned long
 
 void *start_update_xsave_msrs(int xfeature_nr);
 void end_update_xsave_msrs(void);
-int xsave_rdmsrl(void *state, unsigned int msr, unsigned long long *p);
-int xsave_wrmsrl(void *state, u32 msr, u64 val);
-int xsave_set_clear_bits_msrl(void *state, u32 msr, u64 set, u64 clear);
+int xsave_rdmsrl(void *xstate, unsigned int msr, unsigned long long *p);
+int xsave_wrmsrl(void *xstate, u32 msr, u64 val);
+int xsave_set_clear_bits_msrl(void *xstate, u32 msr, u64 set, u64 clear);
+
+void *get_xsave_buffer_unsafe(struct fpu *fpu, int xfeature_nr);
+int xsave_wrmsrl_unsafe(void *xstate, u32 msr, u64 val);
 #endif /* _ASM_X86_FPU_API_H */
diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c
index 25b1b0c417fd..71b08026474c 100644
--- a/arch/x86/kernel/fpu/xstate.c
+++ b/arch/x86/kernel/fpu/xstate.c
@@ -1881,6 +1881,17 @@ static u64 *__get_xsave_member(void *xstate, u32 msr)
 	}
 }
 
+/*
+ * Operate on the xsave buffer directly. It makes no gaurantees that the
+ * buffer will stay valid now or in the futre. This function is pretty
+ * much only useful when the caller knows the fpu's thread can't be
+ * scheduled or otherwise operated on concurrently.
+ */
+void *get_xsave_buffer_unsafe(struct fpu *fpu, int xfeature_nr)
+{
+	return get_xsave_addr(&fpu->fpstate->regs.xsave, xfeature_nr);
+}
+
 /*
  * Return a pointer to the xstate for the feature if it should be used, or NULL
  * if the MSRs should be written to directly. To do this safely, using the
@@ -1971,14 +1982,11 @@ int xsave_rdmsrl(void *xstate, unsigned int msr, unsigned long long *p)
 	return 0;
 }
 
-int xsave_wrmsrl(void *xstate, u32 msr, u64 val)
+
+int xsave_wrmsrl_unsafe(void *xstate, u32 msr, u64 val)
 {
 	u64 *member_ptr;
 
-	__xsave_msrl_prepare_write();
-	if (!xstate)
-		return wrmsrl_safe(msr, val);
-
 	member_ptr = __get_xsave_member(xstate, msr);
 	if (!member_ptr)
 		return 1;
@@ -1988,6 +1996,15 @@ int xsave_wrmsrl(void *xstate, u32 msr, u64 val)
 	return 0;
 }
 
+int xsave_wrmsrl(void *xstate, u32 msr, u64 val)
+{
+	__xsave_msrl_prepare_write();
+	if (!xstate)
+		return wrmsrl_safe(msr, val);
+
+	return xsave_wrmsrl_unsafe(xstate, msr, val);
+}
+
 int xsave_set_clear_bits_msrl(void *xstate, u32 msr, u64 set, u64 clear)
 {
 	u64 val, new_val;
-- 
2.17.1