Received-SPF: pass (google.com: domain of linux-kernel+bounces-173969-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:40f1:3f00::1 as permitted sender) client-ip=2604:1380:40f1:3f00::1;
Message-ID: <7e589b35-4ff8-43fa-99dd-d3b17f56d3ea@intel.com>
Date: Wed, 8 May 2024 17:29:05 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/2] x86/fpu: Extend kernel_fpu_begin_mask() to
 initialize AMX state
To: Dave Hansen <dave.hansen@intel.com>, <linux-kernel@vger.kernel.org>
CC: <x86@kernel.org>, <platform-driver-x86@vger.kernel.org>,
	<tglx@linutronix.de>, <mingo@redhat.com>, <bp@alien8.de>,
	<dave.hansen@linux.intel.com>, <hdegoede@redhat.com>,
	<ilpo.jarvinen@linux.intel.com>, <tony.luck@intel.com>,
	<ashok.raj@intel.com>, <jithu.joseph@intel.com>
References: <20240430212508.105117-1-chang.seok.bae@intel.com>
 <20240507235344.249103-1-chang.seok.bae@intel.com>
 <20240507235344.249103-2-chang.seok.bae@intel.com>
 <f82879a5-f3ca-436f-8c4a-96d4c5d90354@intel.com>
Content-Language: en-US
From: "Chang S. Bae" <chang.seok.bae@intel.com>
In-Reply-To: <f82879a5-f3ca-436f-8c4a-96d4c5d90354@intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 8bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?YjcxZHRweHVvRXRnd1p3VVVLK3BCcnZiS1NXOHhnTmdROTN4YXVsb3JPdkY5?=
 =?utf-8?B?d25mL1BrM28zM0o1OTg3RHROWHU1dTZwWmJWNnprQWJvVHR6OUdRWVY5RFZr?=
 =?utf-8?B?dWFKVUtjdjFEOWt4bEJCMHQzcEI0ZHlwNGxScEtSRXlrcm1icGtSMWZrOEJu?=
 =?utf-8?B?U0ZaTHBnZFJRdmI0U2o5dWxZalNQTDZ0RnNwWHY5UzI5K1c3V3BYeW0vRm1v?=
 =?utf-8?B?dlBOVXpyeVBhTUx4b2RHS2hpMEx4SnVlU3FrbVlObC8xN2xuTE1uQnJQK3E0?=
 =?utf-8?B?akd2Kzd6dUJxUEQwTStpWXliT3ZYUFRTYXdlakYvcTZUTEw1RjhOQTBFbGtj?=
 =?utf-8?B?aC9BaEFTaVFuQWlVOHVCU0VzWWtKY2ZnWFV1eVlmRW1WQ1dZNS9mcUtHdmND?=
 =?utf-8?B?N3hUQkgyRWw4VHgvSlZNZjlZRnhielhKQisrbXFsR1pSelFROHVDMXoxcFdx?=
 =?utf-8?B?M0hUbnE0elNoZ2gyV0NuSVJsVHZiRlBGSHFWcXhieGtsYm5iZ3ZXdWdxb3NT?=
 =?utf-8?B?Q0RoaVBnZkZ1d1Q0VE4rOEYrQjRvUjl4b0xGS3pmbUQ4cUJtTFhQRXlJSlFm?=
 =?utf-8?B?aXQ5N2ZyQ2ZZQ3RVcWs2aHZYOWgwS0ZrZTRHMlZqaW4zUFQvWldJSEdwSi8r?=
 =?utf-8?B?S3dpMUl3REdlR0liTzNTMzFCWi9OTHBMR2V2L25JbEI5YjRVdXcwUmpyZklX?=
 =?utf-8?B?N3doS3U3Sndhb0pmeWlSZmpGQXlnN080bGtSOXF0SGwzbW4xd2orZWl2OFkv?=
 =?utf-8?B?VDNwTUVjRzVKamhvdGN6c000ZnNTa01oTDVsUndrSmNrenRLSy8xb2orVFdi?=
 =?utf-8?B?VkxnRVhMdmJnNG1UR0ZGL1FWOS93TzA3NG1ySGpKTG5FYlFSa0NKNVNiek5I?=
 =?utf-8?B?UWR1TlFxQi9aTCs1MmtxdDkvVG16OHVkdmZPY0wxZVBuSnhTZWVUOUVHZ01w?=
 =?utf-8?B?a2lLT3p6bnRoTys5dGZ2UWd5Z1g0aEViSEFPRVc3VWM4Vi9jOG8zeitGVEZh?=
 =?utf-8?B?QzlSOEVRTVVGYUNONU1vR1c2d1RQZG1KaEdNcHhLVithSzRTT001UDFZbC8y?=
 =?utf-8?B?NmttUExZV2JOeWVtOW1KOWVGVngzdWU4UEhDMlZRc0FkTzhmQnU4Qjlvc0FC?=
 =?utf-8?B?Y0pUNGpCT0NWZXI4QUQ5ZzY2aFlOWjNyL2FjWEZFWnNqMHhSQ1VzaEFZK0ZM?=
 =?utf-8?B?Q05NSkJPZzRFVzUrQ1pwQTFBSDVHdjR4cU5TVFFxNW5LRHAvMXN6ZEEzejdC?=
 =?utf-8?B?Si9zUWxoaUpaa21mcWZIK3lKOFJBV3h2WTBIVFhiNitBZU4yOUpjVWo3RnAy?=
 =?utf-8?B?UVpoY1lEZ0Mrc1B5aTBRYTczdXd0SFhobm5lSFNzbG1RL3lUWW9lQ2ovSmx6?=
 =?utf-8?B?MSt6Q0lqZm8zb3FpYmZMKzhrTlh1TEdQSFVydkoyelhtS2VkVHBEVWx2RFJh?=
 =?utf-8?B?dVFZZFk1aEJUK3RRNVhIT055ZVN3c1Q0cHpYNDVZaWVTYVlPTUZJdWRJZWc4?=
 =?utf-8?B?QVl5UVh3WlBRUmk4cWo0NTlsRVZwaFlaNEQ5R2ZnSUtlUmZQZmt3d1ZCem9U?=
 =?utf-8?B?aGc4RHBlR0tTQjBtS3VJdmpLVDQ1YVZxejhVZ0R3WXdhQzRJQU5lZnlVcyt0?=
 =?utf-8?B?dis5NVpKWE9vNnIycWJKRklobmtvblp3Yk9JR1lBWFpVeUc5WmhCcE5uMDh6?=
 =?utf-8?B?bWZhWllFSVQvWFE4ZzdqNGFUSVAvOGJNWmtXbkg1RmRNaGVTR2hMZlp1OHNC?=
 =?utf-8?B?dDNYNXpmUG04cXhidm11V1ZGV2dOQlZ2bVd3VDFnSDlCTTAvU3l2QXlnSEh6?=
 =?utf-8?B?bU1kK0I4OE9vdXhXU0FSTGZTaUordXdXcEMvdFFjVXFEUnF1RS81aXhIc01B?=
 =?utf-8?B?Vk9Wa1dtZDhKWHFxeVZ6US9wRWF0Zkp0OFI3T1RBRnEwMjVsWlk4YW5aSXhT?=
 =?utf-8?B?aUpGSGxsYXRvbEM0SmJBRWU5LzRqY0VEUHJNemNmRlQzQXZOcDFETUEvQWY3?=
 =?utf-8?B?WC8vc3lNejRQTGNtSXBCQ3RqTmM2YVMzeHpCdGprcFliakdkNC9JRW4wTVNZ?=
 =?utf-8?B?YVh3aC9RU0tmMmFnQ3JyRWFqdSt3M3RwYXZ4R0V1aktDdE0veDBKODBsVzBo?=
 =?utf-8?B?UmIzdHVxSWFVNGMySzd3b0tpUEluN2VwaS9GVy9IRDQ5ZCt0ZVVaVUYvZTZK?=
 =?utf-8?B?aGc9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 55bda192-b1b4-42c6-f5cb-08dc6fbf09d6
X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB7164.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 09 May 2024 00:29:07.5681
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 552N3O4/vI+RTPmvjQiF+IW0S327FzHG+r9yhlBOy9ugGyjfI7OFiquxzrcCJ3zT+GCLCcLWJvmMZkcx9TZb4IDewKUSX+iim1xflQXP4GA=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB5808
X-OriginatorOrg: intel.com

On 5/8/2024 7:40 AM, Dave Hansen wrote:
> 
> As I look closer, I'm not sure I think this is a good match for the two
> other KFPU_* flags.  I don't think either KFPU_387 or KFPU_MXCSR causes
> any XFEATURE to be tracked as init.  The SDM says that FNINIT "sets the
> FPU control, status, tag, instruction pointer, and data pointer
> registers to their default states."
> 
> Off the top of my head, I don't know how that maps to the XSAVE init
> tracking but I do know that MXCSR has a very weird mapping to the first
> two XSAVE features.

FNINIT does *not* set the component to be tracked as init. Indeed, the 
SSE component is somewhat convoluted. AMX state will be likely tracked 
as init.  But, I believe this init tracking mechanism is 
implementation-specific, not architectural. Beyond this intricacy of 
init-tracking, KFPU_* flags all initialize each state:

/* Kernel FPU states to initialize in kernel_fpu_begin_mask() */
#define KFPU_387	_BITUL(0)	/* 387 state will be initialized */
#define KFPU_MXCSR	_BITUL(1)	/* MXCSR will be initialized */

> I really think it would be simplest to just have this whole thing do this:
> 
> 	kernel_fpu_begin();
> 
> 	// Zap AMX state
> 
> 	kernel_fpu_end();
> 
> Where the zap is either an os_xrstor() or an explicit tile release
> instruction.

If I understand correctly, the change could be something like this, 
although I'm not sure about this new API naming and prototype at this point:

diff --git a/arch/x86/include/asm/fpu/api.h b/arch/x86/include/asm/fpu/api.h
index a2be3aefff9f..906634402ea6 100644
--- a/arch/x86/include/asm/fpu/api.h
+++ b/arch/x86/include/asm/fpu/api.h
@@ -28,6 +28,7 @@

  extern void kernel_fpu_begin_mask(unsigned int kfpu_mask);
  extern void kernel_fpu_end(void);
+extern void kernel_fpu_reset(void);
  extern bool irq_fpu_usable(void);
  extern void fpregs_mark_activate(void);

diff --git a/arch/x86/kernel/fpu/core.c b/arch/x86/kernel/fpu/core.c
index 1209c7aebb21..0351f9ee3e49 100644
--- a/arch/x86/kernel/fpu/core.c
+++ b/arch/x86/kernel/fpu/core.c
@@ -452,6 +452,15 @@ void kernel_fpu_end(void)
  }
  EXPORT_SYMBOL_GPL(kernel_fpu_end);

+void kernel_fpu_reset(void)
+{
+       kernel_fpu_begin();
+       if (cpu_feature_enabled(X86_FEATURE_AMX_TILE))
+               tile_release();
+       kernel_fpu_end();
+}
+EXPORT_SYMBOL(kernel_fpu_reset);
+
  /*
   * Sync the FPU register state to current's memory register state when the
   * current task owns the FPU. The hardware register state is preserved.
diff --git a/drivers/platform/x86/intel/ifs/ifs.h 
b/drivers/platform/x86/intel/ifs/ifs.h
index 56b9f3e3cf76..5e7ba94b4054 100644
--- a/drivers/platform/x86/intel/ifs/ifs.h
+++ b/drivers/platform/x86/intel/ifs/ifs.h
@@ -129,6 +129,7 @@
   */
  #include <linux/device.h>
  #include <linux/miscdevice.h>
+#include <asm/fpu/api.h>

  #define MSR_ARRAY_BIST                         0x00000105
  #define MSR_COPY_SCAN_HASHES                   0x000002c2
diff --git a/drivers/platform/x86/intel/ifs/runtest.c 
b/drivers/platform/x86/intel/ifs/runtest.c
index 95b4b71fab53..33ef4962aeba 100644
--- a/drivers/platform/x86/intel/ifs/runtest.c
+++ b/drivers/platform/x86/intel/ifs/runtest.c
@@ -188,6 +188,8 @@ static int doscan(void *data)
         /* Only the first logical CPU on a core reports result */
         first = cpumask_first(cpu_smt_mask(cpu));

+       kernel_fpu_reset();
+
         wait_for_sibling_cpu(&scan_cpus_in, NSEC_PER_SEC);

         /*

> It's just a matter of whether you want this to work like a regular old
> kernel FPU user or you want to tie it to *only* being able to run in its
> own kernel thread. -- Aside: I don't think I realized IFS had its own
> thread earlier.

While both approaches aim for the same functionality, the code change 
seems to be smaller for the above. At the same time, it is notable that 
these new APIs, fpu_idle_fpregs() and the other, are *only* for AMX.

1. The diff state of this series' approach:
  arch/x86/include/asm/fpu/api.h           |  1 +
  arch/x86/kernel/fpu/core.c               |  3 +++
  drivers/platform/x86/intel/ifs/ifs.h     | 15 +++++++++++++++
  drivers/platform/x86/intel/ifs/runtest.c |  6 ++++++
  4 files changed, 25 insertions(+)

2. The above code changes:
  arch/x86/include/asm/fpu/api.h           | 1 +
  arch/x86/kernel/fpu/core.c               | 9 +++++++++
  drivers/platform/x86/intel/ifs/ifs.h     | 1 +
  drivers/platform/x86/intel/ifs/runtest.c | 2 ++
  4 files changed, 13 insertions(+)

Thanks,
Chang