Received-SPF: pass (google.com: domain of linux-kernel+bounces-173688-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <62e2ebd9-4afa-4bb1-a30c-5ba34bc6b94e@intel.com>
Date: Wed, 8 May 2024 11:03:22 -0700
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v2 1/2] x86/fpu: Extend kernel_fpu_begin_mask() to
 initialize AMX state
To: Dave Hansen <dave.hansen@intel.com>, <linux-kernel@vger.kernel.org>
CC: <x86@kernel.org>, <platform-driver-x86@vger.kernel.org>,
	<tglx@linutronix.de>, <mingo@redhat.com>, <bp@alien8.de>,
	<dave.hansen@linux.intel.com>, <hdegoede@redhat.com>,
	<ilpo.jarvinen@linux.intel.com>, <tony.luck@intel.com>,
	<ashok.raj@intel.com>, <jithu.joseph@intel.com>
References: <20240430212508.105117-1-chang.seok.bae@intel.com>
 <20240507235344.249103-1-chang.seok.bae@intel.com>
 <20240507235344.249103-2-chang.seok.bae@intel.com>
 <f82879a5-f3ca-436f-8c4a-96d4c5d90354@intel.com>
Content-Language: en-US
From: "Chang S. Bae" <chang.seok.bae@intel.com>
In-Reply-To: <f82879a5-f3ca-436f-8c4a-96d4c5d90354@intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?d0NIWEpsdXRqaWZNcFFtWW1lazJKa3d0Z0VLRURsUDk1b21tLyswTXdHdEVy?=
 =?utf-8?B?d2N6ZlYzeDRiVnBHUDVURW1icmNRbS9KM0tKQ2Q0NFZhQ09ycWRELzFYVHJy?=
 =?utf-8?B?MWVla1hMYmRkdnEwWkF0elRPc3p4UkRzYXRjWUdtWndaUDFqcnhHc08vSVI1?=
 =?utf-8?B?emFWT25XSnU5SWl3a0NvRmRyQWNLQzh0ejdQUmMvODBWL0NIK0ZCUzJXQ3V6?=
 =?utf-8?B?cXNaRnZTbXk1MWhXdVJsd0wvVCtQVW9EcXBacXJQREZKZjdnM3o2TnhaNEdn?=
 =?utf-8?B?WExwMGpSdDNrSzY3TTFDM3lyT3E1bFFxZGFrWlFVK2NRbWRsbTFBenVtVXA1?=
 =?utf-8?B?NkZFcUs0U0tMaHJ1NDZMWm9GbWdzWUZwdXg4MXJWVkJ3bEI5VFJ3ZmhvR2l3?=
 =?utf-8?B?Y01sVTBKV010SjY5WCtSL0dwT2RSNlBpbktzZ2VEeEl6L0VFZkpNRVNHTXpC?=
 =?utf-8?B?ZnhvRVg3VHBsOHJOUDFVcFRqRlBlYmN1dDZUQ1I0NUhRZXViWkRvWXZCWS9q?=
 =?utf-8?B?ME5ZSW9TaDF2QWZSWEdidDdvaXlRQ3YyRkU0M0JpaUZPL1hsUDlKUXNTWWFN?=
 =?utf-8?B?ancrV291VkRoeEdRUTZlUmdBQVFjcTlNTzVNZjNhQy84ZllCeUhRRGkrcEVM?=
 =?utf-8?B?VWRZR21IZ0RJVWlxQWYzenJBUkxabDdCNHhiYWhPem5JMzNjcU5qWFlwUE9U?=
 =?utf-8?B?SkFucTdhRUJmeVdPUEF5Q1dNWFhGS29qU09RY0ZXQnRnWVRCOUF6ZG1ZQXFK?=
 =?utf-8?B?UTBFUU14c2V2NklpR2t2Ujc4N3JYQWZrd1k1dS9zTEZPa25DbFArVk5CV3dX?=
 =?utf-8?B?M3R4UXVEVElkOElZQWpWNFJudmdLWU1ia1hiYitTTTNzYlpnNjcxOU03dThl?=
 =?utf-8?B?T0tYVkJrYXdlb3FYclo0UmYxRjIzd3BIa0ZPNTRNNTB2bW0rYUJFdGExSlUx?=
 =?utf-8?B?NjZxNkl6aXRYVTNsb0l6S2ZGV2tGNFlxUkZoZFZNa3doeC83TzFrdkVqMytw?=
 =?utf-8?B?SDU4VzVIL2xkYWlrcDhiZ3piSUh2cjY5SGVneEpWaXJuV09uenZzeDl2dCth?=
 =?utf-8?B?RTFkRmcrR3plVmtYREN2RFZFTnBxbHVuYjNkeVpSdFRHZUhXRDE3QW44Mzgz?=
 =?utf-8?B?VEdKV0QreFc2OXh3eU5HQ0tXRjdEemUvYXdoRTlRellzOURtemROMnVxRnFU?=
 =?utf-8?B?QlNNQXpJOFR3MWYwNFp5YTBpRWpEM2RhWjljdkNtUldwUEsvcWVkRU51MGFp?=
 =?utf-8?B?OFd2dmxqUEMwS1kzWXBjTXoyZS9GVk9vb1ZwS1ZLcTd3NUpDNUQ3OC9wTWdI?=
 =?utf-8?B?cERoajFDRStRS1pVbTNJRVdkUnV2VGRCWWtBNmw0a09yVFVlTW1ob0ZBZXFX?=
 =?utf-8?B?VG1nY0NiK0lKZStuRVppZ29SQkxyR2hyTU1wV2tFbkc0U1BEWkgzRWtIc2Q1?=
 =?utf-8?B?dEdPNGRJKy9Qc29yQ1U1N1JLU3B3dHUweWk1YlV6cHRhTkdvbXkwRjc1NHA0?=
 =?utf-8?B?WlpmSHdhZjBoZGtrQkEvT1luQUxxU0t3eTgxbkhoTUU4c3Q0eFVJNUU3dGty?=
 =?utf-8?B?MjRFNmRUYjZ5QW9iekFnaEQvYmJZY3F0bkF3SjRWS3N1WFFhckZQd2NTWmJ2?=
 =?utf-8?B?Zi94cGlkUWwyeFVaSnAwelhMMi8vd0Z0eEZxQ2x1Y1NzT1EwR2kzNlgxTW40?=
 =?utf-8?B?NFRQakt4elJ2aVQrQ25ZM3luVkVZSU5RQkJrSUhXUHkzS1BUeHp3YVcveWtO?=
 =?utf-8?B?TURETTcyN3U4R3M0ZTJNU3BzS1dORE5ka1hWZFNWejlaZ1RqeW1ZZTc1dzJH?=
 =?utf-8?B?RVBIQzFKaFQvK3IvVStxYTZKOUpBRVlFYXdrRTdIODBjZE1WQTVjb3BiTG95?=
 =?utf-8?B?S0xqcE1WNERQZDNpWkdTNnd0ZlFmR01sZ2NMNGNnWXdTenF5VDBGSmR0L3pw?=
 =?utf-8?B?bW83cUtlNVhQdnFKSi84RkZpL05VZ1RZS2ZGK0JFaTJlZXZRS1RzWHMxbzNr?=
 =?utf-8?B?RUhmYXhqTDR4NTJtekExVFBvYU5LSzY1NDgzRW9Tc3FsY3o3SmhEc0MvdW5P?=
 =?utf-8?B?VnJIcjVvZHpYS2VnbXpjL0Y0Ym5PdUQ2enY0MEhRclA1NzFTS0c1RXlHK0Yw?=
 =?utf-8?B?Wlorc0pVdTE4UlZyazAzbDJvMnZuVUorWGsyK05DL09DK3d6Tnc1RFpIVmtj?=
 =?utf-8?B?S1E9PQ==?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 3f5bde7f-6793-4368-a03d-08dc6f892abc
X-MS-Exchange-CrossTenant-AuthSource: MW4PR11MB7164.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 08 May 2024 18:03:29.9358
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: wXXu+TWZjDi/qQGaakEiAoyJhgJ2rPGjWj0fIoZCt2oYRE2oKGafaManqeU42oX6paihsGIHMHeB0Sdh2AaYAAdzs7xUfEZTERka4pm1yVk=
X-MS-Exchange-Transport-CrossTenantHeadersStamped: PH0PR11MB7471
X-OriginatorOrg: intel.com

On 5/8/2024 7:40 AM, Dave Hansen wrote:
> On 5/7/24 16:53, Chang S. Bae wrote:
> 
>> However, due to resource constraints in storage, AMX state is excluded
>> from the scope of state recovery. Consequently, AMX state must be in its
>> initialized state for the IFS test to run.
> 
> This doesn't mention how this issue got introduced.  Are we all bad at
> reading the SDM? :)

Ah, I'd rather zap out this SDM sentence.

>> When AMX workloads are running, an active user AMX state remains even
>> after a context switch, optimizing to reduce the state reload cost. In
>> such cases, the test cannot proceed if it is scheduled.
> 
> This is a bit out of the blue.  What does scheduling have do do with IFS?

$ echo <cpu#> > /sys/devices/virtual/misc/intel_ifs_0/run_test

Then,
run_test_store()
-> do_core_test()
   -> ifs_test_core()
     -> stop_core_cpuslocked()
       -> stop_cpus()
         -> queue_stop_cpus_work()
           -> cpu_stop_queue_work()
             -> wake_q_add()
             -> wake_up_q()

So, the CPU stopper threads for <cpu#> and its sibling to execute 
doscan() are queued up with the highest priority.

queue_stop_cpus_work() has

	for_each_cpu(cpu, cpumask) {
		work = &per_cpu(cpu_stopper.stop_work, cpu);
		work->fn = fn;
		work->arg = arg;
		work->done = done;
		work->caller = _RET_IP_;
		if (cpu_stop_queue_work(cpu, work))
			queued = true;
	}

Those threads are created during early boot via 
smpboot_register_percpu_thread().

> I'm not sure those last two paragraphs add much value.  I'd try to
> banish most of that content to *after* you talk about the solution.  Or
> maybe put it in the cover letter.

It looks like lots of distractions coming from bunch of alternatives in 
different levels.

Thanks,
Chang

PS: Let me respond the solution discussion separately. I do want to 
experiment the init-track behavior a bit.