Subject: Re: [PATCH 19/26] x86/tdx: Make pages shared in ioremap()
To:     "Kirill A. Shutemov" <kirill.shutemov@linux.intel.com>,
        tglx@linutronix.de, mingo@redhat.com, bp@alien8.de,
        dave.hansen@intel.com, luto@kernel.org, peterz@infradead.org
Cc:     sathyanarayanan.kuppuswamy@linux.intel.com, aarcange@redhat.com,
        ak@linux.intel.com, dan.j.williams@intel.com, david@redhat.com,
        hpa@zytor.com, jgross@suse.com, jmattson@google.com,
        joro@8bytes.org, jpoimboe@redhat.com, knsathya@kernel.org,
        pbonzini@redhat.com, sdeep@vmware.com, seanjc@google.com,
        tony.luck@intel.com, vkuznets@redhat.com, wanpengli@tencent.com,
        x86@kernel.org, linux-kernel@vger.kernel.org
References: <20211214150304.62613-1-kirill.shutemov@linux.intel.com>
 <20211214150304.62613-20-kirill.shutemov@linux.intel.com>
From:   Tom Lendacky <thomas.lendacky@amd.com>
Message-ID: <87c288d6-9bf8-5a94-a628-1e0aaa7de690@amd.com>
Date:   Wed, 22 Dec 2021 11:26:59 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101
 Thunderbird/78.11.0
In-Reply-To: <20211214150304.62613-20-kirill.shutemov@linux.intel.com>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?QnQ5VkptaXlvd2w1SmhMVXo0eDduNEsyZ3ZmQWw0ZFVYOVhRejl2NnAxN3pE?=
 =?utf-8?B?dkhDcGZFYkw0ZjkraEpwRmZVZ3VEUmFBU2lYcUIrY2c4cDJvMEh6SU9tWnRr?=
 =?utf-8?B?OE00UzR3WnFxcTdLaG1MbjA5dzh2ZEpkTjRIdWxwR2VsSVlsOC81OG9JellJ?=
 =?utf-8?B?MUp1c3h0aklxVGllQ20rcGlNSVRLWkQ4YzVYREp6OGRyclNyOXVIemlQNXFQ?=
 =?utf-8?B?TVdNRjZkV2ZibnBnUEdhNjh1QlQ5am9NdGdTM3Z3WDRVS1pid2x0MkRlZkFO?=
 =?utf-8?B?OGcyZC9Vc1FHQllzaEcxSzJhZ1RKMkdLSWZQZlZvdTkrVEROWXFjaERxb3BN?=
 =?utf-8?B?VG5iT0ZQSmFBdVhITUVEdHVJYldOeHViM1Z4WDBGeGJqbk4zVzhoaVlZenh6?=
 =?utf-8?B?ajcrV1dpUzBtbDJTNlA0ZGQvWk56S01CUUhzc0EyQjJRV0pqQ1JZSjhwYm5t?=
 =?utf-8?B?VnB2Q1V5WHBIUUIxR0JQdEVOVTRJRGVHVFFud09nTWhhVmdTdUVMQytvK3Bs?=
 =?utf-8?B?UkllbEJFakZSemZEc2VwZkhIWDJ6VnRidEJDbEVDdktQTFNHeTlMeWo0QlVM?=
 =?utf-8?B?UGNSbXgzSGlXQ1R2VmhIcjUxL01zdDF6a0o3d0xVSjQ3cVQrWEM5TTlRQllM?=
 =?utf-8?B?L2VpOGdPZzFaVlNrZ0tsQzA1M3pDZHV3QkVDM3M3ZENFZnVOU3FDWitXdDlF?=
 =?utf-8?B?TWk5VDhrTTM5djQvOUlxMUNGQ2d3cENZUVBzaFYxV2poaVBEZmhKQ1BhcEhh?=
 =?utf-8?B?U3BBREcvdXcxanRUY3VqYUtRZk9ZaXk0cWN2RURpeGY3WDJkRjBleDlRWHNq?=
 =?utf-8?B?S2R0R3BUc1NZdnZybFlqWjdWSi9NUFlQSGdUbWVxMUxBSlRna2F1aFg4Umlj?=
 =?utf-8?B?bWZBWis5N0tXOHdSbDFtM0VFQW43QThXbnZ4SG9CVG9BdjlkYkcvV01CL0h0?=
 =?utf-8?B?dG5icTFSbk1iTTF4TEpHNCsxd3h0dmkrTFdwSlVMcnVxZGFuSDB2U25kejFP?=
 =?utf-8?B?Z2gzRVNyY2FVWndsNjdZOEs1KytSYXpKRHhhdDBWQ2wvaDdQYy9RSWhmYTFi?=
 =?utf-8?B?QS9Kc09zRE1aM05ZK3lhSmhIc2RwZCttQXJHOVVodGlLdFBsZ3RiQ2lkcEhk?=
 =?utf-8?B?NVAvYkN2dGY0WE9kRkFNbGRBRDY5YWZUWlE4enJ1WHRXU2xKTGZLeFBvenNa?=
 =?utf-8?B?ayt4Ry9DMk5NWGY4VFRvUjdzNVZxdnFqUU5UMFVDSWY2azlKalIvenFwTWxU?=
 =?utf-8?B?bjg2Vk5FaXRkODJwUVZRYkFtK21SYlN6V0F3WWFDNVJRNkRtUVNDWkp2ZmtO?=
 =?utf-8?B?bExlN1EwcGx2M2xsYmdDMW1XTkZDeVdxd1hxNUdGc2Q3RFF5Z2lsbnFBSGZ0?=
 =?utf-8?B?d2F2ZjQraUFNQXhKZlRINzFUaUFjdG12MkFTN2w3QXFSWUVsTFltdnh0bFV3?=
 =?utf-8?B?SWxyTkN5OTg2eDdTRFFMRmxmdThZTWZkRnMyZm5qZUNzRlNVNjREdUpKRmkw?=
 =?utf-8?B?c1MxaE80bkxRYjBmODhxQmt5dUw2cjJ2R09HNmtBR1VtMzl5NTZlSGVnZ2Vs?=
 =?utf-8?B?NGs5R2JGN1o1Q2ZRY043RFpoNTFUNlhUdUJ6cVJIZTBhaWxqTkxiakhiMUFx?=
 =?utf-8?B?VTlnS1BzWDY0L0MyOHgyOU9aOGRaOHB5YzdDSXRUbC93amluWjN1aU1kNVZO?=
 =?utf-8?B?L1Vta0ozVGhpVHpyR1BrU3ZvMWRBOHZmVDNQYzJGNDhsOWN1cVFuMVJqVzFF?=
 =?utf-8?B?dVRGbE5jR3dHU0lSb3ZuS1F5OGorQWliQllMdjdYR0ZCeDc1WTJFcnNSZC8v?=
 =?utf-8?B?Z200OVcwdUNXaTQ5ZEl0ZWROWDJSSmNSOUE2bmRodEJ6RERrTUNXdWwyL2dp?=
 =?utf-8?B?Ymtyeml3V3dKVlduVFRlUDhNOUdNMnFpUU5ZY2JYNDRoRmlsY1hOdlRjYUNT?=
 =?utf-8?B?dkhjS0xKSUNTOTlPRDl3UmY3OXR4N2RiR2xKRUZyaWFwQXlFZ1dRUnlWQkxG?=
 =?utf-8?B?Z2JrYU5sOFZrSXU4ZmZCV3Uwa25ldmFCaS9qdTJqT08zeXZTK2xZRGQxUDBk?=
 =?utf-8?B?ZXVSU2Y0VDhuMXNrT1Jub01DUzcySUhKRkhhTW9jYUcydmY2YjN2L3hnRGlM?=
 =?utf-8?B?c3ZsQXhDZmk3U2l2dWRVQ0RuSXg3M1NOYVpKbmJMdUl0UnZWSjVSSVJXQzVZ?=
 =?utf-8?Q?JktgSMRnPBYDBYeCA3p9vFs=3D?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 4e65a88a-761a-41c1-e9c6-08d9c5704467
X-MS-Exchange-CrossTenant-AuthSource: DM4PR12MB5229.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 22 Dec 2021 17:27:02.5554
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: iit5xeufsrr6IknFGngzYV9Qu6xAuLSFKlbbtQQ0viI7hO4oYkrpK597rbKFa2+3Aq5LFYV72n1QcxS3OJzPbg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DM6PR12MB5568
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 12/14/21 9:02 AM, Kirill A. Shutemov wrote:
> In TDX guests, guest memory is protected from host access. If a guest
> performs I/O, it needs to explicitly share the I/O memory with the host.
> 
> Make all ioremap()ed pages that are not backed by normal memory
> (IORES_DESC_NONE or IORES_DESC_RESERVED) mapped as shared.
> 
> Since TDX memory encryption support is similar to AMD SEV architecture,
> reuse the infrastructure from AMD SEV code. Introduce CC_ATTR_GUEST_TDX
> to add TDX-specific changes to the AMD SEV/SME memory encryption code.
> 
> Add tdx_shared_mask() interface to get the TDX guest shared bitmask.
> 
> pgprot_decrypted() is used by drivers (i915, virtio_gpu, vfio). Export
> both pgprot_encrypted() and pgprot_decrypted().
> 

> --- a/arch/x86/mm/mem_encrypt.c
> +++ b/arch/x86/mm/mem_encrypt.c
> @@ -14,6 +14,33 @@
>   #include <linux/mem_encrypt.h>
>   #include <linux/virtio_config.h>
>   
> +#include <asm/tdx.h>
> +
> +/*
> + * Set or unset encryption attribute in vendor agnostic way.
> + */
> +pgprot_t pgprot_cc_encrypted(pgprot_t prot)
> +{
> +	if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
> +		return __pgprot(__sme_set(pgprot_val(prot)));
> +	else if (cc_platform_has(CC_ATTR_GUEST_TDX))
> +		return __pgprot(pgprot_val(prot) & ~tdx_shared_mask());
> +

Hmmm... I believe this breaks SEV guests. __sme_set() uses sme_me_mask 
which is used for both SME and SEV. With the current checks, an SEV guest 
will end up never setting an encrypted address through this path. Ditto 
below on the decrypted path.

Thanks,
Tom

> +	return prot;
> +}
> +EXPORT_SYMBOL_GPL(pgprot_cc_encrypted);
> +
> +pgprot_t pgprot_cc_decrypted(pgprot_t prot)
> +{
> +	if (cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT))
> +		return __pgprot(__sme_clr(pgprot_val(prot)));
> +	else if (cc_platform_has(CC_ATTR_GUEST_TDX))
> +		return __pgprot(pgprot_val(prot) | tdx_shared_mask());
> +
> +	return prot;
> +}
> +EXPORT_SYMBOL_GPL(pgprot_cc_decrypted);
> +
>   /* Override for DMA direct allocation check - ARCH_HAS_FORCE_DMA_UNENCRYPTED */
>   bool force_dma_unencrypted(struct device *dev)
>   {
> diff --git a/include/linux/cc_platform.h b/include/linux/cc_platform.h
> index 63b15108bc85..5fed077cc5f4 100644
> --- a/include/linux/cc_platform.h
> +++ b/include/linux/cc_platform.h
> @@ -82,6 +82,15 @@ enum cc_attr {
>   	 * Examples include TDX Guest.
>   	 */
>   	CC_ATTR_HOTPLUG_DISABLED,
> +
> +	/**
> +	 * @CC_ATTR_GUEST_TDX: Trust Domain Extension Support
> +	 *
> +	 * The platform/OS is running as a TDX guest/virtual machine.
> +	 *
> +	 * Examples include Intel TDX.
> +	 */
> +	CC_ATTR_GUEST_TDX = 0x100,
>   };
>   
>   #ifdef CONFIG_ARCH_HAS_CC_PLATFORM
>