Subject: Re: [PATCH] x86/mm: Fix PAGE_KERNEL_IO removal breakage
To:     Joerg Roedel <joro@8bytes.org>, x86@kernel.org,
        Dave Hansen <dave.hansen@linux.intel.com>
Cc:     Borislav Petkov <bp@alien8.de>, Ingo Molnar <mingo@redhat.com>,
        Thomas Gleixner <tglx@linutronix.de>, hpa@zytor.com,
        Andy Lutomirski <luto@kernel.org>,
        Peter Zijlstra <peterz@infradead.org>,
        Brijesh Singh <brijesh.singh@amd.com>,
        linux-kernel@vger.kernel.org, Joerg Roedel <jroedel@suse.de>,
        Lucas De Marchi <lucas.demarchi@intel.com>
References: <20211202144646.23186-1-joro@8bytes.org>
From:   Tom Lendacky <thomas.lendacky@amd.com>
Message-ID: <a155df16-ce8f-e884-221b-c3ff9b11eca7@amd.com>
Date:   Thu, 2 Dec 2021 09:52:28 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.14.0
In-Reply-To: <20211202144646.23186-1-joro@8bytes.org>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?RlBKdjFWdk1KSDVtSUtWZmNOc28yRkVlVnZWNFAyMlJuWlNyMk1NWm1vWXZa?=
 =?utf-8?B?Z2FBSkI2QTVYNzBiZElRQWRWaHM2ZXdESHJUbGowM0ZiMUVhVll3Y1F1N3Fa?=
 =?utf-8?B?Qzd2d1owY0lyVG8xTldyUHJiak9HbCtadm80dFY4ZW9EWW5UZ2hNdGlua21I?=
 =?utf-8?B?NVBGNWorQ2ZYdk5hSTc0T052UXpYVXRVQ1crbThrNnRORjU2WjNweXhaRVlk?=
 =?utf-8?B?aXFpN2VkZENXb0lGc1lLNU4wVWRNT0VNdDh6clBwNWRxZTZYY0tHam1OV21I?=
 =?utf-8?B?TnJvUjFuUXFZZFo2RHh4aktXZDhpVlU3RG9kMFFZU2hZWjhUSXdseFJXUWJz?=
 =?utf-8?B?cTJENkxycHo2Nit5WEZvSXYvZW9KSFZWWWsvM2NFTGgrYWNua3ZrakhVeGtn?=
 =?utf-8?B?Z0tXT09QVklCMGtyZi9JU1VDNEdoQWY4TjQ2OWI1VGFGMHg1YzMvandyaWtV?=
 =?utf-8?B?cytSQks3V1hxU3cra0huOW92TjhyRThKU052cUJzTGI1c0NuYlAwaGk0L2Zs?=
 =?utf-8?B?KzE5Z1pVSEkya2JCQWgvTXZGS09XU1lxRHRlZUdKZHZXdk9JTVh5WkkwdEZy?=
 =?utf-8?B?ZU5QNXkxQzZzNzZHZUs5STI0N3BsZlZpSHNFTVgwNVROTkMyZysrMlgwMjBL?=
 =?utf-8?B?ZWI1QzVjLzluWnFvaHZ6TlpCSDJNbW9oUXB0QXpqSjhIWThyeUlwRDVQcjNR?=
 =?utf-8?B?dzFBVFJFMjA0dXJUQzQxVDlVK2N0c3ByMXJKcm5pR3B6aEZKOTFDYXpaOTlv?=
 =?utf-8?B?QnAxQ2pwTmIxY0hDSXU2cDdMYnMzV2t4a3FVR09iVWMrdlh5T0w4b0FsNGNz?=
 =?utf-8?B?WDltSEY5cUkzSFJLRERZSkxIME9OWHk2NjNuaHY2YUVtZ1dsbXhvVHhXK0hq?=
 =?utf-8?B?T3lxM3VlN1dYZWdjQmRWb1kwNlZ2SS9jRTdMWkFxLytWRFpKZEo4VW0wMWFo?=
 =?utf-8?B?MElmN0F6MHowYW9WUHJ5TDJkTHJ0NzRzdXdIUXlsWEQ0c2RxUEw1ZlkvQ015?=
 =?utf-8?B?ZWVkQTVOWmtEZkltR29PUE1zUmxUeHRDZmJXUWdYQUxKTE8yL3BjQ2tjQXIy?=
 =?utf-8?B?eEhrU0VHS0t0enppdDZFaWdENnl2akdXdnRvUno1MVQzejArRnl5SDB5NDIv?=
 =?utf-8?B?V3JOTFVDbW15ZittTEt5RTlsZ29HMnRlWnVtTC9SSHY4K1NPeGc0NHk3dmN4?=
 =?utf-8?B?OFZaRmxaSHNLcUI0S3NHcTB2RHd1bDFVU2lSVTd2SEVYeHo4dVlJYXhzSjhC?=
 =?utf-8?B?eUphYTZZZE85R0NSbGxUWGRXRDFseXdMeWsvREpTWGJWdFlwa0sydHNIc0ZW?=
 =?utf-8?B?RTgxTnl1bzNwbVBaVDVBV2tJUGNhZ2VDWTNyYytmZE9xUklpanBqTnBaWWVV?=
 =?utf-8?B?SG5DR2tOOUpxeHhocXRQaVhRUC8zRGFGNjk2K1dtckQ3MXd1VXpObHI4c1dj?=
 =?utf-8?B?TFpNblpjU2pwRnZ1S1BLTktlL3ZVT3U4UXppYVliZ1BZdnlucEJpZWVYbTBv?=
 =?utf-8?B?V05SZVkyYmtCS2RudGt4SnBiK1dwU3dCWG1qdWFnaUJScFgrK1FBWUpJSS9v?=
 =?utf-8?B?WWZyajRldmVSUlMwcVluVDNvVEVNeFl6WjR0dHdsQkxCS3NVWHByVllFeTBX?=
 =?utf-8?B?MjlYTE5wTkFsVzVNY3NnYm1jUk1Kdk0yYVhHYndFQWFPTUt3U2w3eGFLcjdS?=
 =?utf-8?B?cC9adWJOSDd3S2RIR1pURVlEdzM3OVdDazBNQmpRNFhMYklCVU4rdUROTWlN?=
 =?utf-8?B?NC9CUkpSUy91Nm1qTmZlS2Q3dERuT05xSExaU0NuWTdXeTN0NUROZFlTWTIz?=
 =?utf-8?B?MTlGODVObzJrV0tnR0VMNGl6dW1PUmF3VnplRFlhL29WbEdMK0hsRGhFYUtY?=
 =?utf-8?B?cHN3M3JHdUl0L2s4YU5sTnJOaHdMaFpCVUFkY2lqUHFEaXR1Tk43bFJ3clFa?=
 =?utf-8?B?NkxoOEdQYlBOcHUvejZCaVdWM0NKcjZXMDN3NlJJZ1Mya21QUFk1ZkViVXZZ?=
 =?utf-8?B?TmpyRVBORHJnRXBVVXpoMXNHUWpnQU5kaklTcEIxNFR4MHgxbm9kc2htd3NR?=
 =?utf-8?B?VXRWNUlNUTRuL0FlVDl1Q0l2Rm01NHBKY0YraDdua0wvMEpkMHlpWk5tdG4z?=
 =?utf-8?B?MEhMelZrRlNKZ25valFqdVA3OGdpVGQrWTd2S0VEUDZsYzdLV0xCNWVxZGkv?=
 =?utf-8?Q?hUFGlgkyMqMCsoCBYTwZQmI=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: cdd6f352-b579-48e5-9cbd-08d9b5abc015
X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB5229.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 Dec 2021 15:52:31.7706
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: cC7XFuCFY+2sFfPsxAnuoeax51AetmY6onhcBIowg3I21Lqa9nMMlnsczeCcesWhGfZYzfBYHE98oIzmLo2l+w==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM8PR12MB5479
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 12/2/21 8:46 AM, Joerg Roedel wrote:
> From: Joerg Roedel <jroedel@suse.de>
> 
> The removal of PAGE_KERNEL_IO broke SEV-ES because it changed the
> mapping of ioremap and some fixmap areas (like the local APIC page)
> from unencrypted to encrypted. Change those mappings back to
> be unencrypted.

SME was also broken and I can verify this patch fixes SME, too.

Tested-by: Tom Lendacky <thomas.lendacky@amd.com>

Thanks,
Tom

> 
> Cc: Lucas De Marchi <lucas.demarchi@intel.com>
> Fixes: 27dff0f58bde ("x86/mm: Nuke PAGE_KERNEL_IO")
> Signed-off-by: Joerg Roedel <jroedel@suse.de>
> ---
>   arch/x86/include/asm/fixmap.h        |  2 +-
>   arch/x86/include/asm/pgtable_types.h | 21 +++++++++++----------
>   arch/x86/mm/ioremap.c                |  2 +-
>   3 files changed, 13 insertions(+), 12 deletions(-)
> 
> diff --git a/arch/x86/include/asm/fixmap.h b/arch/x86/include/asm/fixmap.h
> index 5e186a69db10..a2eaf265f784 100644
> --- a/arch/x86/include/asm/fixmap.h
> +++ b/arch/x86/include/asm/fixmap.h
> @@ -173,7 +173,7 @@ static inline void __set_fixmap(enum fixed_addresses idx,
>    * supported for MMIO addresses, so make sure that the memory encryption
>    * mask is not part of the page attributes.
>    */
> -#define FIXMAP_PAGE_NOCACHE PAGE_KERNEL_NOCACHE
> +#define FIXMAP_PAGE_NOCACHE PAGE_KERNEL_NOCACHE_NOENC
>   
>   /*
>    * Early memremap routines used for in-place encryption. The mappings created
> diff --git a/arch/x86/include/asm/pgtable_types.h b/arch/x86/include/asm/pgtable_types.h
> index a87224767ff3..fc9b6995cb22 100644
> --- a/arch/x86/include/asm/pgtable_types.h
> +++ b/arch/x86/include/asm/pgtable_types.h
> @@ -208,16 +208,17 @@ enum page_cache_mode {
>   
>   #define __pgprot_mask(x)	__pgprot((x) & __default_kernel_pte_mask)
>   
> -#define PAGE_KERNEL		__pgprot_mask(__PAGE_KERNEL            | _ENC)
> -#define PAGE_KERNEL_NOENC	__pgprot_mask(__PAGE_KERNEL            |    0)
> -#define PAGE_KERNEL_RO		__pgprot_mask(__PAGE_KERNEL_RO         | _ENC)
> -#define PAGE_KERNEL_EXEC	__pgprot_mask(__PAGE_KERNEL_EXEC       | _ENC)
> -#define PAGE_KERNEL_EXEC_NOENC	__pgprot_mask(__PAGE_KERNEL_EXEC       |    0)
> -#define PAGE_KERNEL_ROX		__pgprot_mask(__PAGE_KERNEL_ROX        | _ENC)
> -#define PAGE_KERNEL_NOCACHE	__pgprot_mask(__PAGE_KERNEL_NOCACHE    | _ENC)
> -#define PAGE_KERNEL_LARGE	__pgprot_mask(__PAGE_KERNEL_LARGE      | _ENC)
> -#define PAGE_KERNEL_LARGE_EXEC	__pgprot_mask(__PAGE_KERNEL_LARGE_EXEC | _ENC)
> -#define PAGE_KERNEL_VVAR	__pgprot_mask(__PAGE_KERNEL_VVAR       | _ENC)
> +#define PAGE_KERNEL			__pgprot_mask(__PAGE_KERNEL            | _ENC)
> +#define PAGE_KERNEL_NOENC		__pgprot_mask(__PAGE_KERNEL            |    0)
> +#define PAGE_KERNEL_RO			__pgprot_mask(__PAGE_KERNEL_RO         | _ENC)
> +#define PAGE_KERNEL_EXEC		__pgprot_mask(__PAGE_KERNEL_EXEC       | _ENC)
> +#define PAGE_KERNEL_EXEC_NOENC		__pgprot_mask(__PAGE_KERNEL_EXEC       |    0)
> +#define PAGE_KERNEL_ROX			__pgprot_mask(__PAGE_KERNEL_ROX        | _ENC)
> +#define PAGE_KERNEL_NOCACHE		__pgprot_mask(__PAGE_KERNEL_NOCACHE    | _ENC)
> +#define PAGE_KERNEL_NOCACHE_NOENC	__pgprot_mask(__PAGE_KERNEL_NOCACHE    |    0)
> +#define PAGE_KERNEL_LARGE		__pgprot_mask(__PAGE_KERNEL_LARGE      | _ENC)
> +#define PAGE_KERNEL_LARGE_EXEC		__pgprot_mask(__PAGE_KERNEL_LARGE_EXEC | _ENC)
> +#define PAGE_KERNEL_VVAR		__pgprot_mask(__PAGE_KERNEL_VVAR       | _ENC)
>   
>   #endif	/* __ASSEMBLY__ */
>   
> diff --git a/arch/x86/mm/ioremap.c b/arch/x86/mm/ioremap.c
> index 3102dda4b152..4fe8d43d53bb 100644
> --- a/arch/x86/mm/ioremap.c
> +++ b/arch/x86/mm/ioremap.c
> @@ -243,7 +243,7 @@ __ioremap_caller(resource_size_t phys_addr, unsigned long size,
>   	 * make sure the memory encryption attribute is enabled in the
>   	 * resulting mapping.
>   	 */
> -	prot = PAGE_KERNEL;
> +	prot = PAGE_KERNEL_NOENC;
>   	if ((io_desc.flags & IORES_MAP_ENCRYPTED) || encrypted)
>   		prot = pgprot_encrypted(prot);
>   
>