From: Nadav Amit <[email protected]>
Do not use flush-queue on virtualized environments, where the NpCache
capability of the IOMMU is set. This is required to reduce
virtualization overheads.
This change follows a similar change to Intel's VT-d and a detailed
explanation as for the rationale is described in commit 29b32839725f
("iommu/vt-d: Do not use flush-queue when caching-mode is on").
Cc: Joerg Roedel <[email protected]>
Cc: Will Deacon <[email protected]>
Cc: Jiajun Cao <[email protected]>
Cc: Robin Murphy <[email protected]>
Cc: Lu Baolu <[email protected]>
Cc: [email protected]
Cc: [email protected]
Signed-off-by: Nadav Amit <[email protected]>
---
drivers/iommu/amd/init.c | 7 ++++++-
1 file changed, 6 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index d006724f4dc2..ba3b76ed776d 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -1850,8 +1850,13 @@ static int __init iommu_init_pci(struct amd_iommu *iommu)
if (ret)
return ret;
- if (iommu->cap & (1UL << IOMMU_CAP_NPCACHE))
+ if (iommu->cap & (1UL << IOMMU_CAP_NPCACHE)) {
+ if (!amd_iommu_unmap_flush)
+ pr_warn_once("IOMMU batching is disabled due to virtualization");
+
amd_iommu_np_cache = true;
+ amd_iommu_unmap_flush = true;
+ }
init_iommu_perf_ctr(iommu);
--
2.25.1
On 2021-06-07 19:25, Nadav Amit wrote:
> From: Nadav Amit <[email protected]>
>
> Do not use flush-queue on virtualized environments, where the NpCache
> capability of the IOMMU is set. This is required to reduce
> virtualization overheads.
>
> This change follows a similar change to Intel's VT-d and a detailed
> explanation as for the rationale is described in commit 29b32839725f
> ("iommu/vt-d: Do not use flush-queue when caching-mode is on").
>
> Cc: Joerg Roedel <[email protected]>
> Cc: Will Deacon <[email protected]>
> Cc: Jiajun Cao <[email protected]>
> Cc: Robin Murphy <[email protected]>
> Cc: Lu Baolu <[email protected]>
> Cc: [email protected]
> Cc: [email protected]
> Signed-off-by: Nadav Amit <[email protected]>
> ---
> drivers/iommu/amd/init.c | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
> index d006724f4dc2..ba3b76ed776d 100644
> --- a/drivers/iommu/amd/init.c
> +++ b/drivers/iommu/amd/init.c
> @@ -1850,8 +1850,13 @@ static int __init iommu_init_pci(struct amd_iommu *iommu)
> if (ret)
> return ret;
>
> - if (iommu->cap & (1UL << IOMMU_CAP_NPCACHE))
> + if (iommu->cap & (1UL << IOMMU_CAP_NPCACHE)) {
> + if (!amd_iommu_unmap_flush)
> + pr_warn_once("IOMMU batching is disabled due to virtualization");
Nit: you can just use pr_warn() (or arguably pr_info()) since the
explicit conditions already only match once. Speaking of which, it might
be better to use amd_iommu_np_cache instead, since other patches are
planning to clean up the last remnants of amd_iommu_unmap_flush.
Robin.
> +
> amd_iommu_np_cache = true;
> + amd_iommu_unmap_flush = true;
> + }
>
> init_iommu_perf_ctr(iommu);
>
>
> On Jun 15, 2021, at 6:08 AM, Robin Murphy <[email protected]> wrote:
>
> On 2021-06-07 19:25, Nadav Amit wrote:
>> From: Nadav Amit <[email protected]>
>> Do not use flush-queue on virtualized environments, where the NpCache
>> capability of the IOMMU is set. This is required to reduce
>> virtualization overheads.
>> This change follows a similar change to Intel's VT-d and a detailed
>> explanation as for the rationale is described in commit 29b32839725f
>> ("iommu/vt-d: Do not use flush-queue when caching-mode is on").
>> Cc: Joerg Roedel <[email protected]>
>> Cc: Will Deacon <[email protected]>
>> Cc: Jiajun Cao <[email protected]>
>> Cc: Robin Murphy <[email protected]>
>> Cc: Lu Baolu <[email protected]>
>> Cc: [email protected]
>> Cc: [email protected]
>> Signed-off-by: Nadav Amit <[email protected]>
>> ---
>> drivers/iommu/amd/init.c | 7 ++++++-
>> 1 file changed, 6 insertions(+), 1 deletion(-)
>> diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
>> index d006724f4dc2..ba3b76ed776d 100644
>> --- a/drivers/iommu/amd/init.c
>> +++ b/drivers/iommu/amd/init.c
>> @@ -1850,8 +1850,13 @@ static int __init iommu_init_pci(struct amd_iommu *iommu)
>> if (ret)
>> return ret;
>> - if (iommu->cap & (1UL << IOMMU_CAP_NPCACHE))
>> + if (iommu->cap & (1UL << IOMMU_CAP_NPCACHE)) {
>> + if (!amd_iommu_unmap_flush)
>> + pr_warn_once("IOMMU batching is disabled due to virtualization");
>
> Nit: you can just use pr_warn() (or arguably pr_info()) since the explicit conditions already only match once.
Yes, my bad. I will fix it in the next version.
> Speaking of which, it might be better to use amd_iommu_np_cache instead, since other patches are planning to clean up the last remnants of amd_iommu_unmap_flush.
I prefer that the other patches (that remove amd_iommu_unmap_flush) would address this code as well. I certainly do not want to embed amd_iommu_np_cache deep into the flushing logic. IOW: I don’t know what you have exactly in mind, but I prefer the code would be clear.
This code follows (copies?) the same pattern+logic from commit 5f3116ea8b5 ("iommu/vt-d: Do not use flush-queue when caching-mode is on”). I see that changed the code in commit 53255e545807c ("iommu: remove DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE”), but did not get rid of intel_iommu_strict, so please allow me to use amd_iommu_unmap_flush.
To remind you/me/whoever: disabling batching due to caching-mode/NP-cache is not inherently needed. It was not needed for quite some time on Intel, but somehow along the way the consolidated flushing code broke it, and now it is needed (without intrusive code changes).
On 2021-06-15 19:26, Nadav Amit wrote:
>
>
>> On Jun 15, 2021, at 6:08 AM, Robin Murphy <[email protected]> wrote:
>>
>> On 2021-06-07 19:25, Nadav Amit wrote:
>>> From: Nadav Amit <[email protected]>
>>> Do not use flush-queue on virtualized environments, where the NpCache
>>> capability of the IOMMU is set. This is required to reduce
>>> virtualization overheads.
>>> This change follows a similar change to Intel's VT-d and a detailed
>>> explanation as for the rationale is described in commit 29b32839725f
>>> ("iommu/vt-d: Do not use flush-queue when caching-mode is on").
>>> Cc: Joerg Roedel <[email protected]>
>>> Cc: Will Deacon <[email protected]>
>>> Cc: Jiajun Cao <[email protected]>
>>> Cc: Robin Murphy <[email protected]>
>>> Cc: Lu Baolu <[email protected]>
>>> Cc: [email protected]
>>> Cc: [email protected]
>>> Signed-off-by: Nadav Amit <[email protected]>
>>> ---
>>> drivers/iommu/amd/init.c | 7 ++++++-
>>> 1 file changed, 6 insertions(+), 1 deletion(-)
>>> diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
>>> index d006724f4dc2..ba3b76ed776d 100644
>>> --- a/drivers/iommu/amd/init.c
>>> +++ b/drivers/iommu/amd/init.c
>>> @@ -1850,8 +1850,13 @@ static int __init iommu_init_pci(struct amd_iommu *iommu)
>>> if (ret)
>>> return ret;
>>> - if (iommu->cap & (1UL << IOMMU_CAP_NPCACHE))
>>> + if (iommu->cap & (1UL << IOMMU_CAP_NPCACHE)) {
>>> + if (!amd_iommu_unmap_flush)
>>> + pr_warn_once("IOMMU batching is disabled due to virtualization");
>>
>> Nit: you can just use pr_warn() (or arguably pr_info()) since the explicit conditions already only match once.
>
> Yes, my bad. I will fix it in the next version.
>
>> Speaking of which, it might be better to use amd_iommu_np_cache instead, since other patches are planning to clean up the last remnants of amd_iommu_unmap_flush.
>
> I prefer that the other patches (that remove amd_iommu_unmap_flush) would address this code as well. I certainly do not want to embed amd_iommu_np_cache deep into the flushing logic. IOW: I don’t know what you have exactly in mind, but I prefer the code would be clear.
>
> This code follows (copies?) the same pattern+logic from commit 5f3116ea8b5 ("iommu/vt-d: Do not use flush-queue when caching-mode is on”). I see that changed the code in commit 53255e545807c ("iommu: remove DOMAIN_ATTR_DMA_USE_FLUSH_QUEUE”), but did not get rid of intel_iommu_strict, so please allow me to use amd_iommu_unmap_flush.
Sure, it was just a suggestion to pre-resolve one line of merge conflict
with another series[1] which is also almost ready, and removes those
local variables for both AMD and Intel. But there will still be other
conflicts either way, so it's not a big deal.
Robin.
[1]
https://lore.kernel.org/linux-iommu/[email protected]/
> To remind you/me/whoever: disabling batching due to caching-mode/NP-cache is not inherently needed. It was not needed for quite some time on Intel, but somehow along the way the consolidated flushing code broke it, and now it is needed (without intrusive code changes).
>