Received-SPF: pass (google.com: domain of linux-kernel+bounces-17885-linux.lists.archive=gmail.com@vger.kernel.org designates 147.75.199.223 as permitted sender) client-ip=147.75.199.223;
Message-ID: <d2cc19fc-f8c6-4b3a-ba88-5db947fb1604@amd.com>
Date: Fri, 5 Jan 2024 20:39:52 +0700
User-Agent: Mozilla Thunderbird
From: "Suthikulpanit, Suravee" <suravee.suthikulpanit@amd.com>
Subject: Re: [RFC PATCH 6/6] iommu/amd: Introduce nested translation support
To: "Tian, Kevin" <kevin.tian@intel.com>,
 "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
 "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
 "joro@8bytes.org" <joro@8bytes.org>, "jgg@nvidia.com" <jgg@nvidia.com>
Cc: "Liu, Yi L" <yi.l.liu@intel.com>,
 "nicolinc@nvidia.com" <nicolinc@nvidia.com>,
 "eric.auger@redhat.com" <eric.auger@redhat.com>,
 "vasant.hegde@amd.com" <vasant.hegde@amd.com>,
 "jon.grimm@amd.com" <jon.grimm@amd.com>,
 "santosh.shukla@amd.com" <santosh.shukla@amd.com>,
 "Dhaval.Giani@amd.com" <Dhaval.Giani@amd.com>,
 "pandoh@google.com" <pandoh@google.com>,
 "loganodell@google.com" <loganodell@google.com>
References: <20231212160139.174229-1-suravee.suthikulpanit@amd.com>
 <20231212160139.174229-7-suravee.suthikulpanit@amd.com>
 <BN9PR11MB5276BA85118EECCCD89681EB8C93A@BN9PR11MB5276.namprd11.prod.outlook.com>
Content-Language: en-US
In-Reply-To: <BN9PR11MB5276BA85118EECCCD89681EB8C93A@BN9PR11MB5276.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0:
	=?utf-8?B?dnk0dFR0djFJd21jcFR3aFV3alNzNVVKMkd1c2laMTViY0pyUks1UTR4WkdB?=
 =?utf-8?B?NFU2ajBMRS9yWGtZa0R0Q0hPRFRmSjJSZytldVJtcUVXTjlaKzZWekY0MHVt?=
 =?utf-8?B?MXFYWDdnM1hPcUNnOE55cXpKWU5iWFdQQXZJWSt2d2dwcDJ3THVOcnBWR3Rh?=
 =?utf-8?B?a2lMTVk4ZC9GS2Y5SVZ6NWE5cm8vT0J3VktkN0lmaVdESGQraFdXWXFsclYr?=
 =?utf-8?B?V1Z3MTRzQVV6dk5UbUNUT0EvKzg0akJVV2owY3RSK2Y0L0xXRmlVRDRlV29l?=
 =?utf-8?B?TTlpdTl2dE00cVZpdnlEc0hEVTlTZjZnQXNPd2FJUXZ3RTh3VmJIeE1wNjZu?=
 =?utf-8?B?Y1BneHpzNzJXK3crQWFqSG4yaGhjZ1kwZHlHSE4zUWxZZXRubmdpMDVvQjVL?=
 =?utf-8?B?QjNkUi9IY29XQmQ3eEFGWEpkYlF0VnNPRWRtZHZKWFMxa2I4T1J4c3pURlVo?=
 =?utf-8?B?K052L0Z2M21PUVlDdHAySWlVeWFXdUxVS3Qxb3VLamgwaW9ReThnUmNMUHVS?=
 =?utf-8?B?VXViYXJwb2NhZEFpdGE1LzhWQXQ4RTllQ0JOR3RxYzdCMHQzMXNvUlJER2JH?=
 =?utf-8?B?MXhiRTMyK202Y1dOVTZld2pDamtiYk8wWDhQVTZmOGgrSUw2Yk43SXZRa3or?=
 =?utf-8?B?cGl5d0NUamY2ZE1zV1A1N0Z0YTJLYkVSd0NTcngxMHRLNVZtZ3NwZ0Q0ZS80?=
 =?utf-8?B?Um55THR6djF4K1JNbG14MjZLMDBvdndwMi9jbTBWZkxoRDE4ZnpwUmNiUHlU?=
 =?utf-8?B?LzFPeUIvOXlwbjRSemFSNEo1d3FsY29taVpnc0k1Q1NNVmMxK3JKNnpRSjM0?=
 =?utf-8?B?dTR4NU02UlVCZFJPUytvc0c2ZHZidnRlakUvVVdBOVRwYldEVUlkMFBocjky?=
 =?utf-8?B?SDlmL1VJTm1Wc2pNOEdnRW9QMlloaWxKN3BES1pZb0xXbitFSEhidVA2aWJa?=
 =?utf-8?B?dlFiTWxPQlpTelMyUFZZSVUxMi8zY2tyaFZOczcwNnBkVVdBUW1RWGo4a002?=
 =?utf-8?B?M0lTUDFvVVhWWXVyNnZtc0V6OTN6QnoweDhVTlRQSnJNbUxpd2sxbDV2V0Nq?=
 =?utf-8?B?Tm5PdEpqU1pQNVJ5ZGh6cWV6aHdsNzlXWU1IdGVUakk1b0hnK0t2cnJGcTl2?=
 =?utf-8?B?dHdBcm9ISEplRHQyTlY1cnRPUS9KSDdGOGF4VHo1b0tJZHJWaEE4bEJ0SUZk?=
 =?utf-8?B?c1g4bjYrM3FDM2RDOHlDQWUyZUJXN3VTejRRV0d2MURxNUdjOHZNRlZOQURV?=
 =?utf-8?B?dWJ4Ym1yTlRrSzhjU2xMckdjZ2RZSDdKUEFqRlBHRXZkRUlsTVNJT09NdTRo?=
 =?utf-8?B?WFl2YTZ1VmpzRmtQdXVWSjczNFhUK0N3TnByb2M2cytJZnA5aC8vWFRnV3d5?=
 =?utf-8?B?eVo3RTM5ZVN4dmltUktGS2ZZSWpKV0NRQUFQL1pHZC81TUZBeWJkYlcyZW44?=
 =?utf-8?B?SFduNHRLU29IdjRLV0p3dXBhVVNJQjMvR3lwdXdzTHl6NElWY1FSVnl4RzFp?=
 =?utf-8?B?YnpFSE1WYkV6eVdibUNNR0FnT1JpbDZkUkZURjloU3crcUJiZ3pORFBUOHh5?=
 =?utf-8?B?Qmx1MFVnS1dJRzhjVy9HRThJK05MS3B2Zm90dVlXMFNldTBkV0dheXlIbXpp?=
 =?utf-8?B?cUJOZHVJRFBaVnJEK2g3NlBGVXprYjlYbHl4UUU5YU9KVzhQclV2eGVlVFow?=
 =?utf-8?B?NnRmVGhIQlRobnZvYk41UEgxRjVHMkxVbHRLSFJJQnd0cmYrdXRYckxpaGk0?=
 =?utf-8?B?NkJDRWlpQ1FHT3ZSSVpPZmcvVHN6OWZ1VFA2NUUwUHpuUTNSTStHS2NyMFVC?=
 =?utf-8?B?YzcxRCtjZDVYb3F2NE9PaXFNWFRvQnRqQVBTYmY5Y2gzWG5DcTBoMnYwNWFS?=
 =?utf-8?B?QU5BeXFpekovL1QwYnVHZml4WWtEMHZxTTJIMmdkYm5yQ09ZZVhiRkF3VVNy?=
 =?utf-8?B?RnM2dXRoOE5HbVRaOXNCb0RlY2pOM3hKdHhsbm1lUUc3UGlZbklpajFZR211?=
 =?utf-8?B?QnkwVUdOUzFQZWtBSDFQSURDdnppcXdmMEVRcXZpUnZpbXlUUW1NVjVmUC8z?=
 =?utf-8?B?L0grQW9JY0o0MWt4bm9Xbnlzai9XREEzMTEvaHpURDZRanZJcmRpaHhBQWd4?=
 =?utf-8?Q?MhouI6xNkeUINuaOJrNgQFxPr?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: e8f9dfd7-7868-425e-45f5-08dc0df3ced1
X-MS-Exchange-CrossTenant-AuthSource: DM8PR12MB5445.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 05 Jan 2024 13:39:58.0891
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: CDjq5KqmLHrGuW84nGTOY+9dWt4TVCbm5z6M/TZKJCVRgWeulbRuVg0GGy3oT0/pfCDVWzeUUIcgG3IRWpwNtw==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: IA1PR12MB6233

Hi Kevin,

On 12/15/2023 2:45 PM, Tian, Kevin wrote:
>> From: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
>> Sent: Wednesday, December 13, 2023 12:02 AM
>>
>> To support nested translation on AMD IOMMU, the driver needs to
>> program DTE[GCR3 Table Root Pointer] with the address provided by
>> the guest via struct iommu_hwpt_amd_v2, which is passed as a parameter
>> of the struct iommu_ops.domain_alloc_user() with the flag
>> IOMMU_HWPT_ALLOC_NEST_PARENT.
>>
>> Note that current implementation only support GCR3TRPMode for
>> nested translation, which uses GPA to program GCR3 Table Root Pointer.
>>
> 
> means there is a plan to support another mode in the future or
> actually the nested translation requires GCR3TRPMode as a
> functional requirement? imho the point of GPA is assumed
> in the nested configuration in concept...

On (older) system, which does not support GCR3TRPMode, the IOMMU driver 
needs to program the device's DTE[GCR3 Table Root Pointer] field w/ SPA.

When QEMU presents an AMD vIOMMU device to a guest, the guest programs 
the guest DTE[GCR3 Table Root Pointer] with GPA. Then we need to :
1. Traps the DTE write
2. Translate the GPA->SPA
3. Program DTE with SPA.

With the GCR3TRPMode, we can skip step 2 above and directly program step 
3 with GPA.

Suravee