Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <da56ae0c-8f72-b39f-95b8-8870ec9fb336@amd.com>
Date:   Fri, 20 Jan 2023 13:55:38 -0600
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:91.0) Gecko/20100101
 Thunderbird/91.2.1
Subject: Re: [PATCH 1/4] iommu/amd: Introduce Protection-domain flag VFIO
Content-Language: en-US
To:     Jason Gunthorpe <jgg@ziepe.ca>
Cc:     Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>,
        linux-kernel@vger.kernel.org, iommu@lists.linux.dev,
        joro@8bytes.org, robin.murphy@arm.com, thomas.lendacky@amd.com,
        vasant.hegde@amd.com, jon.grimm@amd.com
References: <20230110143137.54517-1-suravee.suthikulpanit@amd.com>
 <20230110143137.54517-2-suravee.suthikulpanit@amd.com>
 <Y8F53dzdebKLTlOy@ziepe.ca> <90762dee-1559-58ac-220d-a13635d5032e@amd.com>
 <Y8mBczFH/Hw6xot0@ziepe.ca> <c3fbec52-1281-eb1c-40f8-588438146c14@amd.com>
 <Y8q9ocj2IZB2r6Np@ziepe.ca> <1ba09b11-8a07-24dd-a99f-eeacb2f5c96c@amd.com>
 <Y8rUYTjVhksAu+i9@ziepe.ca>
From:   "Kalra, Ashish" <ashish.kalra@amd.com>
In-Reply-To: <Y8rUYTjVhksAu+i9@ziepe.ca>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?UDNvY1VXZytOdC9qd292bDFWTHhaT05aT3NLUElxWXpjams1eTBWWW8xZkVU?=
 =?utf-8?B?bDNZY1VvZUp6bWJKeEhzMldDLzBaaThXcUtUcERBb0ZFVXBBdFY1b3lPRVQz?=
 =?utf-8?B?RFZHL1l3Qlk2ek91K3hWNXpIQ0taK053VWlvVWdoZW1ZNi9kK1ZYdlJlRU5E?=
 =?utf-8?B?OXMva3ZTb3pUQXRaYi9KSjZXUUpzajFiYXRDUThSaE9ValFWbHhXSE1CcEdT?=
 =?utf-8?B?aU90NUlsNGFJdEJyeXcxZEZrbEo2UElTVEd5K0JnQUErR0gvQ3ppZWdsdE1j?=
 =?utf-8?B?Z0xFdS9YNHkrUXZBVEdtZWFCYjZoL1NkdnRpYW90ZE1LanpWdFpDNmtkSC9r?=
 =?utf-8?B?cXluckl5VlpWdlNyazhzSitGckZ4RlhJNFBMY0RRVVgvTjQ3Z1lON3M1QkVo?=
 =?utf-8?B?QUVOVjhRU2FyV0VjZGFzaVN2NTFHaDdCR1U2UUpITVpXMUo2MjRLZ1NRNUww?=
 =?utf-8?B?SjVqWG9yTEVXWjhrdkNYQjRBWjJLL3ZmdXFuVkxQREVpNUQxM2tUTlJJT241?=
 =?utf-8?B?Mmthbm5tSXNVTVBoaHBWbWxWZmZIcmgrb0RMTk1USG01UkkvS1ZodFBmdFRm?=
 =?utf-8?B?OTF2TDNIeU9aYytGeEpTZENRUnpOSmFPUTFkVzhyMGRVcEIwL25DTVAvcTg5?=
 =?utf-8?B?UEVGK2s5VmdRUjJqUzh2c2VnYzduN2tCTENaa3JvNkFSSHNpYk9LYlo0VnlI?=
 =?utf-8?B?RnlsSHlWV1lYa3Nqbm4wZFRodFVaMVljdTJZTThBZEtVTEErLzR2elJTMmVh?=
 =?utf-8?B?V2h1ZTh3M2JqOEpWUk92bS9pT3phdGhzM0ZuVGpRTDRRY2loQU0yeVBjcjBv?=
 =?utf-8?B?ZTcvemMrbHVmWTU3NHhOMGQ4ZjBnTlUvQXNWUHVmNU9UTzd6LytFRXhZUldq?=
 =?utf-8?B?d2hLTXBEaDZwMWtWZkV5S1pRTTB1NWdrWVBIYU50UDdwTDVXVEkvSE9wM1dh?=
 =?utf-8?B?MTNOVlVjbVErUFJvWlFoQ0cwb3VWZGVNVFRuOFI4MmNOb1ZXT0owYVBWSjM2?=
 =?utf-8?B?elg4V3FwQ21haWVaUVdRTHRVOGtTbCs1bXdPVEhON1pET3NtUlNOZndjUm1J?=
 =?utf-8?B?S2phZStMeU1SSGRPanRJOGdPdTQ5WWN6NFdySi92MSs5bVNqVmhOanBERzZo?=
 =?utf-8?B?QkFLT1pYeUYxVmhMZ1hYdmdTSFd1c3B4Z2JFZWZ3eTRBM1E3SDdDQ0dPZGtM?=
 =?utf-8?B?MDRPNEtxS1YrT3JmYTlTUVpNRUw0Qy9Ub1hxdnUrVUM0VVprc3dsZkpQeVZX?=
 =?utf-8?B?ZWxkZFlVTERhcnIraExENVdLWDVWckFpU1I1SWJMbERLdmlKR1RjUmpOeHdq?=
 =?utf-8?B?aThUMTdVNHNDNjQzQ0E2c29NT0VISUhoUWdsZGZkQmpVR2tKRGFQa254Nkli?=
 =?utf-8?B?YmhoZldZelFBb2UrUGszaWRtWk5abTRsYWpRUVZxZXcxRDVWSXFncGo2TWdQ?=
 =?utf-8?B?aGRndWV3Y2xPbVM5NzJJa2FQL0NkSVlqZUd2WTBXeTJLS1BJQldSaG5kT3Er?=
 =?utf-8?B?dXJlTHRQSTFTUWlVWXpkRmdDR2x3bWF0T1dhWGMvNCt6dkE2dU9EQjRwaHZO?=
 =?utf-8?B?T0l4QjNoaGMvRzRYV0hFQmg3dEYwMWZVY05mMG1XWjZqc1JCeXNNczY1TDVw?=
 =?utf-8?B?ZkloVjN3RVQyaDA1cXZUWXoyVEhkendNMWRLc1E4OHFCcThZY245cjZVUTEx?=
 =?utf-8?B?YklkWDFDVllaaG9FaVFmTzB3M0xSK21Qa1BrYUplS0M1YUZzN3hjRHBXazBl?=
 =?utf-8?B?VnZleWwwZXAyZjlpLzNlalJTMWdaRWwxK3pWQkUxRGVtcE8xQWNyY21HV0JS?=
 =?utf-8?B?RFB4MjRsTmJHT3BOdHA4Sk5SM3JMSm5vcWRIcEpVRWptWWhPSzZhRlUwVjZm?=
 =?utf-8?B?OFZxeFpXRVNHNlloMUlVNFBJOTY4MWlWMG4zWGJ6YnlqUU5CUEtpZHN6QWM5?=
 =?utf-8?B?UzFCQUh3UE1nQi91NHdITXFJMExqL2ovTnZkRDBlWGV4SXYxZkV2NTJvZWFq?=
 =?utf-8?B?NFBUSHlsSEpMNnFaNm12cjlXR3AxM2VwTnFPSE1rY1lRNllhbnZHK1NRRVl3?=
 =?utf-8?B?RVUrVVFMTTZYaWRSMXdjc1U0TkRIR0daTjBwT2NmRVZwT3ErRTIrN2JRN2Fp?=
 =?utf-8?Q?iqiEgmoMqJj7rkRGpAjKd+Hx5?=
X-OriginatorOrg: amd.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 6265bb23-7a98-437c-dcb0-08dafb204ef4
X-MS-Exchange-CrossTenant-AuthSource: SN6PR12MB2767.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 20 Jan 2023 19:55:41.2447
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 3dd8961f-e488-4e60-8e11-a82d994e183d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: 39+CWl5xtNYvvVajrVFpUyVOWIxOeigYEocmQDZ3ANsJx5aJs4B231p0JC1wcCou9ADqrjfyeSBfBW/EFRXYQg==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CY8PR12MB7732
X-Spam-Status: No, score=-2.2 required=5.0 tests=BAYES_00,DKIM_SIGNED,
        DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_NONE,
        RCVD_IN_MSPIKE_H2,SPF_HELO_PASS,SPF_PASS autolearn=ham
        autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
        lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On 1/20/2023 11:50 AM, Jason Gunthorpe wrote:
> On Fri, Jan 20, 2023 at 11:01:21AM -0600, Kalra, Ashish wrote:
> 
>> We basically get the RMP #PF from the IOMMU because there is a page size
>> mismatch between the RMP table and the IOMMU page table. The RMP table's
>> large page entry has been smashed to 4K PTEs to handle page state change to
>> shared on 4K mappings, so this change has to be synced up with the IOMMU
>> page table, otherwise there is now a page size mismatch between RMP table
>> and IOMMU page table which causes the RMP #PF.
> 
> I understand that, you haven't answered my question:
> 
> Why is the IOMMU being programmed with pages it cannot access in the
> first place?
>

I believe the IOMMU page tables are setup as part of device pass-through 
to be able to do DMA to all of the guest memory, but i am not an IOMMU 
expert, so i will let Suravee elaborate on this.

Thanks,
Ashish

> Don't do that is the obvious solution there, and preserves huge page
> IO performance.
> 
> Jason
>