Received-SPF: pass (google.com: domain of linux-kernel+bounces-144610-linux.lists.archive=gmail.com@vger.kernel.org designates 2604:1380:45e3:2400::1 as permitted sender) client-ip=2604:1380:45e3:2400::1;
Message-ID: <66c09ac9-0107-4b86-993b-70ab6907fc21@intel.com>
Date: Mon, 15 Apr 2024 14:56:16 +0800
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH v3 1/1] iommu/vt-d: Remove caching mode check before
 device TLB flush
Content-Language: en-US
To: Baolu Lu <baolu.lu@linux.intel.com>, "Zhang, Tina" <tina.zhang@intel.com>,
	"iommu@lists.linux.dev" <iommu@lists.linux.dev>
CC: "Tian, Kevin" <kevin.tian@intel.com>, Jacob Pan
	<jacob.jun.pan@linux.intel.com>, Joerg Roedel <joro@8bytes.org>, Will Deacon
	<will@kernel.org>, Robin Murphy <robin.murphy@arm.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
References: <20240415013835.9527-1-baolu.lu@linux.intel.com>
 <MW5PR11MB58818970CB8E9A37A6AF563889092@MW5PR11MB5881.namprd11.prod.outlook.com>
 <f3502244-5f37-4f44-94a5-39939ba20eec@intel.com>
 <a5c4b57b-c1f8-4f75-95e4-9a462f4673c9@linux.intel.com>
From: Yi Liu <yi.l.liu@intel.com>
In-Reply-To: <a5c4b57b-c1f8-4f75-95e4-9a462f4673c9@linux.intel.com>
Content-Type: text/plain; charset="UTF-8"; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?utf-8?B?NllmeVR2eUtaYmludVNpWUNtV2FMdHpnU3pNU1dBMHUvZXZBT2ZRMEtYU1Rn?=
 =?utf-8?B?cWUrRk53dUMycHhmQjlSS213YTBSR1F6NUxFRk9nU2FaR01aV1BGbU1LZE1Q?=
 =?utf-8?B?eE5xeGZmMTNzZ2l5NllRYnRiUDFZMHg0dlJRWXJIYko1VVVsUHR5Sk0wdnZ0?=
 =?utf-8?B?aEFWMExCY3JIYlNudGhGaXhxUzkyNVdSVHgrVzFsbjFjWFFQMmt3THJ6aVR3?=
 =?utf-8?B?UHEybXVBWDVsY20wWnBPNms2ZmxibkNzL1lDYW9MT1FOY3ZrakhIMFF0V2RN?=
 =?utf-8?B?Y21TcXRoN0U3M2piTEZPV1A1ZXFPL0JtQkZSZyt4dzRDVFRoS0hhWlQwc2p6?=
 =?utf-8?B?QUsvZG9uWVlpTTM5bmY2cUhITTFHU2ZzMHVUeGkxU2JVbzhZOTd6TzhodFky?=
 =?utf-8?B?UWx1TGVXcnR6Mkh5dzVYUWpMazBNdVdJYlUySzV5aU9pMFhUeXR4K1JzZlZl?=
 =?utf-8?B?V0thTktKTW9TclZaSXd5SWNoSFVwQXdsNllxb3BFNUptNjlvS3JWa0VFNElO?=
 =?utf-8?B?bUdNdS9yaDhlTDlkZkJwSENCR1VXYjdoSGdaY3ZtZEp1U1ZKWHVtVjlKSm5I?=
 =?utf-8?B?cEVrMTUrYzE3UU4ySlRYSUhrN1lSaTd0ckVsbGhIYWs2anJ5MGhsZUQrcUZ0?=
 =?utf-8?B?T2JDbnZQQU5NL1EyYWNXSjQ5Q2pubkxUNFpVcktycEJNcjEyUEdkMFFjcTN0?=
 =?utf-8?B?enVRcTc5cUxxbG1qMXNXOGkyWUp1V3BqcytyeHA4Q2UvSmVSWDJpKzlpK0tp?=
 =?utf-8?B?UmN6ZC9kd3RGRnBsNVYwQWFmeDZPK0VLQjI4TU5FQWZ5alRjOXpBbERWRjVn?=
 =?utf-8?B?YjFtS1RxbEtqVER5Ny9sVVVFWkJSbm9KWGhPck1IL2tNVlBiQTFlYUhYY0Zh?=
 =?utf-8?B?YkNQRzRET05IZ1h6MlpMTzhhZi94MUlleklPQUdRYW9xRndxZzRsdTZtTWw3?=
 =?utf-8?B?MmZJSXM0OFRRekgrRVJRU3RMZE9zTW8ybHdQSVYvUlNxejRzaDR2bUhyUU0z?=
 =?utf-8?B?Y2xEVzNFbWFudDRYNkw1TjFUYTY1cEtpOWZRQWh0RUxHbnZlaXRISTVLUkFN?=
 =?utf-8?B?UHZ0UDk2bllQUVRZcmZESWNoV3FpNFNCaXZVa05nMXM5QXQ1QnBheFAydy83?=
 =?utf-8?B?UnU0bms3dEgrVXlsVVkrZFBjdkhrR2RvZWJoT1lMTkFkWUlaRHQ4bDIvM1hG?=
 =?utf-8?B?cTRBcU5pZUh3WHMxVFAzYjMrdllMSVFQMURuWll3WWkveGhScGN0dXNlL1A3?=
 =?utf-8?B?WGJUU1owMVFEcEJtNkFBaC84UzNjaktYQTdQaGdmUDAxcGRZOEtiVzMrUFA2?=
 =?utf-8?B?VlhIemdLK2V3bURvMDFwZ21LZE5IZjJtUytnWS84ODJFNTNHVXpTZEwyUE5K?=
 =?utf-8?B?RXBxQm5VMWovSUxKQ252d1BhTlpUSG5nUzVFRVM0V1VEcnNWaVM4SGE1c3Rr?=
 =?utf-8?B?OGNDcTZ2MHN5eFV0c1l1STRtR0ZQZ0xWV0s2czBMdG9GZmE5cit4TTRIUHJi?=
 =?utf-8?B?UVV5TktWS0hwUDlSSE5KRmZXL2JRcnpGVDFlYnd6Tk8vS2lIUTR3UW5xay9P?=
 =?utf-8?B?bnhtdFJnb2kzVE9GMXVyZXgrem9JL09qMTNFTWtVdkkwWFYrTGRUS2N1ZnNz?=
 =?utf-8?B?Z0dRaVdjdmdjV2o4aUFmVHpTMmtWZzhoM3BuMHVNcUpqcmt6dUQ5SEdxZWZP?=
 =?utf-8?B?RWY2cGh1VTJSdHgyLzFUckJvTlhndkRQRit3TDVmeXhpRWNoVG5xZk93YzZM?=
 =?utf-8?B?ekM0VWRibHdVSXF3VlhNNXZ6K2IzZmlYQndBekdYWGhqbTI5dE5BUnZxQXNn?=
 =?utf-8?B?QWdpVXN5QmtLazdPK0IzNUhTT2EvZ00zYlA4cmVHN0l3ZERMWUI4aktOYXRh?=
 =?utf-8?B?Zm1xN2hwYzVIdGVxNG0wVWhocmNiYUNPRHk1V0RDSTJuT0R2VzIra2dqWkg0?=
 =?utf-8?B?VDVpZmRsUWt0RXpKWnVXVmMwMHlCYUI0dktQbStISXAvZ1FMQ28vd3BwOGpo?=
 =?utf-8?B?a0tIS3FRdERUZjdQTlg4NVJQM2lOUFVXaDJJK3NYeU9pQ2R6OW9oa0sxKzY2?=
 =?utf-8?B?Ulh3UmRGZGtKWG9qODlsODlLU054cHJhRFdaUkt5aUVvVjRPYnR5aXZWQ3RW?=
 =?utf-8?Q?Ia6L5QZp+jyFdflQ63joz/BS9?=
X-MS-Exchange-CrossTenant-Network-Message-Id: 938bc54b-1a4c-4bab-0ba2-08dc5d18a6b6
X-MS-Exchange-CrossTenant-AuthSource: DS0PR11MB7529.namprd11.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 15 Apr 2024 06:52:44.0231
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 46c98d88-e344-4ed4-8496-4ed7712e255d
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: gA8KTlsJth3PPiLya61EDu6Du7DMpuK0/rG4WxUk8k/gZ0eHruWizRZFWlBu8gHyGlFvHcnrJWMnIgKA1C+3IQ==
X-MS-Exchange-Transport-CrossTenantHeadersStamped: DS7PR11MB5991
X-OriginatorOrg: intel.com

On 2024/4/15 13:00, Baolu Lu wrote:
> On 4/15/24 10:43 AM, Yi Liu wrote:
>> On 2024/4/15 10:22, Zhang, Tina wrote:
>>>
>>>
>>>> -----Original Message-----
>>>> From: Lu Baolu <baolu.lu@linux.intel.com>
>>>> Sent: Monday, April 15, 2024 9:39 AM
>>>> To: iommu@lists.linux.dev
>>>> Cc: Tian, Kevin <kevin.tian@intel.com>; Liu, Yi L <yi.l.liu@intel.com>; 
>>>> Jacob
>>>> Pan <jacob.jun.pan@linux.intel.com>; Joerg Roedel <joro@8bytes.org>; Will
>>>> Deacon <will@kernel.org>; Robin Murphy <robin.murphy@arm.com>; linux-
>>>> kernel@vger.kernel.org; Lu Baolu <baolu.lu@linux.intel.com>
>>>> Subject: [PATCH v3 1/1] iommu/vt-d: Remove caching mode check before
>>>> device TLB flush
>>>>
>>>> The Caching Mode (CM) of the Intel IOMMU indicates if the hardware
>>>> implementation caches not-present or erroneous translation-structure 
>>>> entries
>>>> except for the first-stage translation. The caching mode is irrelevant 
>>>> to the
>>>> device TLB, therefore there is no need to check it before a device TLB
>>>> invalidation operation.
>>>>
>>>> Remove two caching mode checks before device TLB invalidation in the 
>>>> driver.
>>>> The removal of these checks doesn't change the driver's behavior in 
>>>> critical
>>>> map/unmap paths. Hence, there is no functionality or performance impact,
>>>> especially since commit <29b32839725f> ("iommu/vt-d:
>>>> Do not use flush-queue when caching-mode is on") has already disabled
>>>> flush-queue for caching mode. Therefore, caching mode will never call
>>>> intel_flush_iotlb_all().
>>> The current logic is if the caching mode is being used and a domain 
>>> isn't using first level I/O page table, then flush-queue won't be used. 
>>> Otherwise, the flush-queue can be enabled.
>>> See https://github.com/torvalds/linux/commit/257ec29074
>>>
>>> In other words, if the caching mode is being used and a domain is using 
>>> first level I/O page table, the flush-queue can be used for this domain 
>>> to flush iotlb. Could the code change in this patch bring any 
>>> performance impact to this case?
>>
>> This seems to have performance deduction in the nested translation case.
>> The iommufd nested support bas been merged in 6.8, while the Qemu side
>> is wip. So this performance deduction does not happen until Qemu is
>> done. Should this also be considered as a performance regression? TBH.
> 
> Caching mode is irrelevant to first-stage and nesting translations. If
> the QEMU implementation still relies on caching mode for nesting
> support, it's already broken.

For first-stage, yes, caching mode is irrelevant. But for nesting, still
be relevant. Because it relies on caching mode to capture the mofifications
of the pasid entry. Without caching mode, vIOMMU device model within QEMU 
will miss the newly setup pasid entries, hence no way to setup nested
translation at all.

-- 
Regards,
Yi Liu