Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <34600873-3716-eedd-84d0-dada88dc1a70@intel.com>
Date:   Tue, 10 Jan 2023 17:01:59 -0800
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:102.0) Gecko/20100101
 Firefox/102.0 Thunderbird/102.6.1
Subject: Re: [PATCH 3/7] x86/cpu: Disable kernel LASS when patching kernel
 alternatives
Content-Language: en-US
To:     Peter Zijlstra <peterz@infradead.org>
CC:     <linux-kernel@vger.kernel.org>, <x86@kernel.org>,
        Andy Lutomirski <luto@kernel.org>,
        Dave Hansen <dave.hansen@linux.intel.com>,
        Ravi Shankar <ravi.v.shankar@intel.com>,
        Tony Luck <tony.luck@intel.com>,
        Sohil Mehta <sohil.mehta@intel.com>,
        Paul Lai <paul.c.lai@intel.com>
References: <20230110055204.3227669-1-yian.chen@intel.com>
 <20230110055204.3227669-4-yian.chen@intel.com>
 <Y73S56t/wDIGEPlK@hirez.programming.kicks-ass.net>
From:   "Chen, Yian" <yian.chen@intel.com>
In-Reply-To: <Y73S56t/wDIGEPlK@hirez.programming.kicks-ass.net>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?VkJiT2dOYTF6eEQvazVUTXdUaVVPdGszbk1vQ0VTUUg1SSsyTlhYWGozb3ZM?=
 =?utf-8?B?QVdudWxhbXlqeGo3MVFDMW9WUW5iSTR0QkNBL2t1cnU3dW5lZmRDL3JOa0ZR?=
 =?utf-8?B?dTVPTVBRRTc1MHhCZHNTZmYwUzA3c2tIS3h0czJnbnRmSmlHUkRqQVlKaUtJ?=
 =?utf-8?B?V2dZM2N3K0lvenhkU0tpUWJkOUw0Y3Q5K00wNHgzdFNKaGRTM09VZUQ1bE1x?=
 =?utf-8?B?N1IwTmF2UkM1dzZDV0FGeUxudGFWMlBycG0vYk9xdkdYRVliVFUxNDVQT0RD?=
 =?utf-8?B?ME1GTjNLdU92L2hSbVFiSFBRNDR4S09kS3A1MnVkMTNBdlpDVENHTHJJTnA1?=
 =?utf-8?B?b3NLVkdlWitJeC9vQUI0MFc2cENWT2cwcXgxMnNuM2EzaGNyamhzU1FxTUpV?=
 =?utf-8?B?QW5MSWFWL2IrNnlBc2VBbFJWaWVrN0pEcWJMcjdTK041K1NkRzlGb2x2NU5p?=
 =?utf-8?B?ejRlSkE0aGcvV003OW1QMlIxQmwxT1l0RHBDK1I5SFJWVzZsREUvQ2ZUcWI0?=
 =?utf-8?B?ZHhEaEU4MnFESWUzaXdQZUQ3S3BOVHJKZHd6WVNCTlM0NEJYUVdPN3VnbFFL?=
 =?utf-8?B?cjZXVHJEcjBqSkUxZGswZDVTdldDNnlvU3JRSHlUbTlIYWNweVBTZXp3NTE4?=
 =?utf-8?B?dEFYNkJhcC9OdjlGRUpHTlB2a1VBNldQNEEyU2VpQ2ZOTXU5SXBSbnduMkxE?=
 =?utf-8?B?NTViR2ROT29QMDQ4M3hJS2Zvbld3KzZKN09MZVJ2cFA3aEp6T3lGSXpDa1cr?=
 =?utf-8?B?TzhuemdpMWJzcEN1NDg3ZDdOK2VIRjF3Y3Z4bzYxL0cwSk5VOHRhSGp4eldZ?=
 =?utf-8?B?ejUxMU1SNzNIdnk1azd1KzZSSldBNnZ1d2E3a29OWW5DK0VjRWc4WjFqNE52?=
 =?utf-8?B?L200QjJQdFRDMko3eWErd2Q1TC95MEw1UFh4VkIxSHVZcjY3dTAvb2hqMTNB?=
 =?utf-8?B?MlB4dElYKzFSaDkvN3dEVG0zWm9zQllyRXhSMmZDOElsQ1VMM0Y5R0paSEZK?=
 =?utf-8?B?TUZYanFmaDc2Y0VDbGx5dEdNRnBCaS8yUzRlZTgxZU56THBkTkgvM1JDOTFL?=
 =?utf-8?B?aWxieUoyUDdtQzFYZWc1M20vaDhVT25oa3Q5eFdSYkRVdldTUlpScTgwaDhS?=
 =?utf-8?B?RXpRZTRRNU9kWDlXOHNEdm5KSWlkS1ozM3J0RWFaUndVejNPd2Jha2tYazgz?=
 =?utf-8?B?eEF5dGIvY1MwRElFZkFDakRkcXpaMDYwazJ6a2pTeEUzVjNTVmpqeUFIYjdI?=
 =?utf-8?B?elNNczFWb3BJa1B5bDZsdmE1OUdHUDFXaDNSUlhtcTBteUVTaXpoZUNGMmxR?=
 =?utf-8?B?blV4bk5SY1A0U0JyZHNzanpUekxnVEY4bVBtUENCMlJKVEtSTWVjNVZQMFNR?=
 =?utf-8?B?YnE1WU1CSkhkLytYUEVHNnJybTR2L0lZd3cwRU8xRGJKT09nck1aYjhOZHBR?=
 =?utf-8?B?SFVhMFZpQUhWK1BLbnZQZXFFOTRWejRseGlYUGoyaEdFUzVvcVlCN3VrNkcz?=
 =?utf-8?B?dTZ2dENRdzhZTGV6UHh2b0xkejVPaUVSSWxkVUxVMjRMcFJSTUZUazVFQ2VB?=
 =?utf-8?B?eDNXWGVBeDVucmcvSlljaXprazFHcHljZHpJUHZCVlRoc3dvUU1iS0hyR1FQ?=
 =?utf-8?B?SXdFeUFUZjFSbU9zdk13R2FheDVaSEhCdmQwR292S2luV3lGUHhFL0NlMzdy?=
 =?utf-8?B?QnI5alhVb2hoOXplRWxxL1dITUVHa0EvR2pQWklsT2cxb29lT3BtTFVTZXRO?=
 =?utf-8?B?K0pEY1Nnb21laXkxYm82YTRBZzF0eUpuNGVGQzJ6TVdGMnZrb3RXODB1dTEx?=
 =?utf-8?B?NHNyc2taMFNwTGVsTTlRL0xidHZRVEdySnpOUnJJZmxLTTEraVdvQTh5TG1L?=
 =?utf-8?B?ZlN4dytDNjdwck1Cem5zMSszZDJ3RktMcTFtamdGTENUTmJwOVVsVlk4aTZX?=
 =?utf-8?B?SFZ5NGhJeXFpd2Qxbm9Na3cremdEck1QS2VxNGQ2NzMyZVI0ZDhyZ3pnRGNu?=
 =?utf-8?B?dm52eUE2dTNBa3U1ck5RVmRtOGpBTXNCazJkVjRCaVVhQVJreUFPMERzaElz?=
 =?utf-8?B?RnJBYVZzZkFmRFJFaHNjYU56dVMyVituT2dpQVBDbXIvUUhES3hjOG5CcytX?=
 =?utf-8?Q?WzW4JbbmLc6H1u/E6JS3GJH0O?=
X-MS-Exchange-CrossTenant-Network-Message-Id: cb857ce9-8563-45c0-4527-08daf36f730c
X-MS-Exchange-CrossTenant-AuthSource: DM4PR11MB5262.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 11 Jan 2023 01:02:02.5012
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: bZEF3PsnYZY4WmF06wf5sbUGAUs9KdYTM0Gk+02X8SeF4CqXF6K80jehVoh9TwrfbuU7utsbjDsNSV9GAS6vtg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: SJ0PR11MB6695
X-OriginatorOrg: intel.com
X-Spam-Status: No, score=-2.1 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
        DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,
        SPF_HELO_NONE,SPF_NONE autolearn=ham autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org


On 1/10/2023 1:04 PM, Peter Zijlstra wrote:
> On Mon, Jan 09, 2023 at 09:52:00PM -0800, Yian Chen wrote:
>> Most of the kernel is mapped at virtual addresses
>> in the upper half of the address range. But kernel
>> deliberately initialized a temporary mm area
>> within the lower half of the address range
>> for text poking, see commit 4fc19708b165
>> ("x86/alternatives: Initialize temporary mm
>> for patching").
>>
>> LASS stops access to a lower half address in kernel,
>> and this can be deactivated if AC bit in EFLAGS
>> register is set. Hence use stac and clac instructions
>> around access to the address to avoid triggering a
>> LASS #GP fault.
>>
>> Kernel objtool validation warns if the binary calls
>> to a non-whitelisted function that exists outside of
>> the stac/clac guard, or references any function with a
>> dynamic function pointer inside the guard; see section
>> 9 in the document tools/objtool/Documentation/objtool.txt.
>>
>> For these reasons, also considering text poking size is
>> usually small, simple modifications have been done
>> in function text_poke_memcpy() and text_poke_memset() to
>> avoid non-whitelisted function calls inside the stac/clac
>> guard.
>>
>> Gcc may detect and replace the target with its built-in
>> functions. However, the replacement would break the
>> objtool validation criteria. Hence, add compiler option
>> -fno-builtin for the file.
> 
> Please reflow to 72 characters consistently, this is silly.
> 
Sure. I will format the commit msg guideline.

>> Co-developed-by: Tony Luck <tony.luck@intel.com>
>> Signed-off-by: Tony Luck <tony.luck@intel.com>
>> Signed-off-by: Yian Chen <yian.chen@intel.com>
>> ---
>>   arch/x86/include/asm/smap.h      | 13 +++++++++++++
>>   arch/x86/kernel/Makefile         |  2 ++
>>   arch/x86/kernel/alternative.c    | 21 +++++++++++++++++++--
>>   tools/objtool/arch/x86/special.c |  2 ++
>>   4 files changed, 36 insertions(+), 2 deletions(-)
>>
>> diff --git a/arch/x86/include/asm/smap.h b/arch/x86/include/asm/smap.h
>> index bab490379c65..6f7ac0839b10 100644
>> --- a/arch/x86/include/asm/smap.h
>> +++ b/arch/x86/include/asm/smap.h
>> @@ -39,6 +39,19 @@ static __always_inline void stac(void)
>>   	alternative("", __ASM_STAC, X86_FEATURE_SMAP);
>>   }
>>   
>> +/* Deactivate/activate LASS via AC bit in EFLAGS register */
>> +static __always_inline void low_addr_access_begin(void)
>> +{
>> +	/* Note: a barrier is implicit in alternative() */
>> +	alternative("", __ASM_STAC, X86_FEATURE_LASS);
>> +}
>> +
>> +static __always_inline void low_addr_access_end(void)
>> +{
>> +	/* Note: a barrier is implicit in alternative() */
>> +	alternative("", __ASM_CLAC, X86_FEATURE_LASS);
>> +}
> 
> Can't say I like the name. 
Indeed, there are alternative ways to name the functions. for example,
enable_kernel_lass()/disable_kernel_lass(), or simply keep no change to 
use stac()/clac().

I choose this name because it is straight forward to the purpose and 
helps in understanding when to use the functions.

Also if you look at bit 63 as a sign bit,
> it's actively wrong since -1 is lower than 0.
>This could be a trade-off choice. While considering address manipulation 
and calculation, it is likely an unsigned. I would be happy to get input 
for better naming.

>> +
>>   static __always_inline unsigned long smap_save(void)
>>   {
>>   	unsigned long flags;
>> diff --git a/arch/x86/kernel/Makefile b/arch/x86/kernel/Makefile
>> index 96d51bbc2bd4..f8a455fc56a2 100644
>> --- a/arch/x86/kernel/Makefile
>> +++ b/arch/x86/kernel/Makefile
>> @@ -7,6 +7,8 @@ extra-y	+= vmlinux.lds
>>   
>>   CPPFLAGS_vmlinux.lds += -U$(UTS_MACHINE)
>>   
>> +CFLAGS_alternative.o	+= -fno-builtin
>> +
>>   ifdef CONFIG_FUNCTION_TRACER
>>   # Do not profile debug and lowlevel utilities
>>   CFLAGS_REMOVE_tsc.o = -pg
>> diff --git a/arch/x86/kernel/alternative.c b/arch/x86/kernel/alternative.c
>> index 7d8c3cbde368..4de8b54fb5f2 100644
>> --- a/arch/x86/kernel/alternative.c
>> +++ b/arch/x86/kernel/alternative.c
>> @@ -1530,14 +1530,31 @@ __ro_after_init unsigned long poking_addr;
>>   
>>   static void text_poke_memcpy(void *dst, const void *src, size_t len)
>>   {
>> -	memcpy(dst, src, len);
>> +	const char *s = src;
>> +	char *d = dst;
>> +
>> +	/* The parameter dst ends up referencing to the global variable
>> +	 * poking_addr, which is mapped to the low half address space.
>> +	 * In kernel, accessing the low half address range is prevented
>> +	 * by LASS. So relax LASS prevention while accessing the memory
>> +	 * range.
>> +	 */
>> +	low_addr_access_begin();
>> +	while (len-- > 0)
>> +		*d++ = *s++;
>> +	low_addr_access_end();
>>   }
>>   
>>   static void text_poke_memset(void *dst, const void *src, size_t len)
>>   {
>>   	int c = *(const int *)src;
>> +	char *d = dst;
>>   
>> -	memset(dst, c, len);
>> +	/* The same comment as it is in function text_poke_memcpy */
>> +	low_addr_access_begin();
>> +	while (len-- > 0)
>> +		*d++ = c;
>> +	low_addr_access_end();
>>   }
> 
> This is horrific tinkering :/
>
This part seems difficult to have a perfect solution since function call 
or function pointer inside the guard of instruction stac and clac will 
trigger objtool warning (stated the reasons in the commit msg)

To avoid the warning, I considered this might be okay since the poking 
text usually seems a few bytes.

> Also, what about the EFI mm? IIRC EFI also lives in the user address
> space.

I didn't encounter EFI mm related problem while I tested the 
implementation. I will update you later after I investigate more around 
the EFI mm.

Thanks,
Yian