2021-06-11 12:28:23

by John Garry

[permalink] [raw]
Subject: [PATCH v12 0/5] Enhance IOMMU default DMA mode build options

This is a reboot of Zhen Lei's series from a couple of years ago, which
never made it across the line.

I still think that it has some value, so taking up the mantle.

Motivation:
Allow lazy mode be default mode for DMA domains for all ARCHs, and not
only those who hardcode it (to be lazy). For ARM64, currently we must use
a kernel command line parameter to use lazy mode, which is less than
ideal.

I have now included the print for strict/lazy mode, which I originally
sent in:
https://lore.kernel.org/linux-iommu/[email protected]/

There was some concern there about drivers and their custom prints
conflicting with the print in that patch, but I think that it
should be ok.

Difference to v11:
- Rebase to next-20210610
- Drop strict mode globals in Intel and AMD drivers
- Include patch to print strict vs lazy mode
- Include patch to remove argument from iommu_set_dma_strict()

Differences to v10:
- Rebase to v5.13-rc4
- Correct comment and typo in Kconfig (Randy)
- Make Kconfig choice depend on relevant architectures

Differences to v9:
https://lore.kernel.org/linux-iommu/[email protected]/#t
- Rebase to v5.13-rc2
- Remove CONFIG_IOMMU_DEFAULT_PASSTHROUGH from choice:
Since we can dynamically change default domain of group, lazy or strict and
passthrough are not mutually exclusive
- Drop ia64 patch, which I don't think was ever required
- Drop "x86/dma: use IS_ENABLED() to simplify the code", which is no
longer required
- Drop s390/pci patch, as this arch does not use CONFIG_IOMMU_API or even
already honour CONFIG_IOMMU_DEFAULT_PASSTHROUGH
https://lore.kernel.org/linux-iommu/[email protected]/
- Drop powernv/iommu patch, as I no longer think that it is relevant
https://lore.kernel.org/linux-iommu/[email protected]/
- Some tidying

John Garry (2):
iommu: Print strict or lazy mode at init time
iommu: Remove mode argument from iommu_set_dma_strict()

Zhen Lei (3):
iommu: Enhance IOMMU default DMA mode build options
iommu/vt-d: Add support for IOMMU default DMA mode build options
iommu/amd: Add support for IOMMU default DMA mode build options

drivers/iommu/Kconfig | 39 +++++++++++++++++++++++++++++
drivers/iommu/amd/amd_iommu_types.h | 6 -----
drivers/iommu/amd/init.c | 3 +--
drivers/iommu/amd/iommu.c | 6 -----
drivers/iommu/intel/iommu.c | 15 +++++------
drivers/iommu/iommu.c | 13 +++++++---
include/linux/iommu.h | 2 +-
7 files changed, 56 insertions(+), 28 deletions(-)

--
2.26.2


2021-06-11 12:29:08

by John Garry

[permalink] [raw]
Subject: [PATCH v12 2/5] iommu: Enhance IOMMU default DMA mode build options

From: Zhen Lei <[email protected]>

First, add build options IOMMU_DEFAULT_{LAZY|STRICT}, so that we have the
opportunity to set {lazy|strict} mode as default at build time. Then put
the two config options in an choice, as they are mutually exclusive.

[jpg: Make choice between strict and lazy only (and not passthrough)]
Signed-off-by: Zhen Lei <[email protected]>
Signed-off-by: John Garry <[email protected]>
---
drivers/iommu/Kconfig | 38 ++++++++++++++++++++++++++++++++++++++
drivers/iommu/iommu.c | 3 ++-
2 files changed, 40 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index 1f111b399bca..2a71347611d4 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -90,6 +90,44 @@ config IOMMU_DEFAULT_PASSTHROUGH

If unsure, say N here.

+choice
+ prompt "IOMMU default DMA mode"
+ depends on IOMMU_DMA
+
+ default IOMMU_DEFAULT_STRICT
+ help
+ This option allows an IOMMU DMA mode to be chosen at build time, to
+ override the default DMA mode of each ARCH, removing the need to
+ pass in kernel parameters through command line. It is still possible
+ to provide ARCH-specific or common boot options to override this
+ option.
+
+ If unsure, keep the default.
+
+config IOMMU_DEFAULT_STRICT
+ bool "strict"
+ help
+ For every IOMMU DMA unmap operation, the flush operation of IOTLB and
+ the free operation of IOVA are guaranteed to be done in the unmap
+ function.
+
+config IOMMU_DEFAULT_LAZY
+ bool "lazy"
+ help
+ Support lazy mode, where for every IOMMU DMA unmap operation, the
+ flush operation of IOTLB and the free operation of IOVA are deferred.
+ They are only guaranteed to be done before the related IOVA will be
+ reused.
+
+ The isolation provided in this mode is not as secure as STRICT mode,
+ such that a vulnerable time window may be created between the DMA
+ unmap and the mapping finally being torn down in the IOMMU, where the
+ device can still access the system memory. However this mode may
+ provide better performance in high throughput scenarios, and is still
+ considerably more secure than passthrough mode or no IOMMU.
+
+endchoice
+
config OF_IOMMU
def_bool y
depends on OF && IOMMU_API
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index cf58949cc2f3..ccbd5d4c1a50 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -29,7 +29,8 @@ static struct kset *iommu_group_kset;
static DEFINE_IDA(iommu_group_ida);

static unsigned int iommu_def_domain_type __read_mostly;
-static bool iommu_dma_strict __read_mostly = true;
+static bool iommu_dma_strict __read_mostly =
+ IS_ENABLED(CONFIG_IOMMU_DEFAULT_STRICT);
static u32 iommu_cmd_line __read_mostly;

struct iommu_group {
--
2.26.2

2021-06-11 12:29:42

by John Garry

[permalink] [raw]
Subject: [PATCH v12 3/5] iommu/vt-d: Add support for IOMMU default DMA mode build options

From: Zhen Lei <[email protected]>

Make IOMMU_DEFAULT_LAZY default for when INTEL_IOMMU config is set,
as is current behaviour.

Also delete global flag intel_iommu_strict:
- In intel_iommu_setup(), call iommu_set_dma_strict(true) directly. The
accompanying print is removed, as it replicates the print in
iommu_subsys_init(), and, since called from __setup(), should before
iommu_subsys_init() (so desired ordering).

- For cap_caching_mode() set in intel_iommu_setup(), call
iommu_set_dma_strict(true) directly, and reword the accompanying print
and add the missing '\n'.

- For Ironlake GPU, again call iommu_set_dma_strict(true) directly and
keep the accompanying print.

Note that we should not conflict with iommu.strict param conflicting when
we set strict mode, as that param is not for x86.

[jpg: Remove intel_iommu_strict]
Signed-off-by: Zhen Lei <[email protected]>
Signed-off-by: John Garry <[email protected]>
---
drivers/iommu/Kconfig | 1 +
drivers/iommu/intel/iommu.c | 15 ++++++---------
2 files changed, 7 insertions(+), 9 deletions(-)

diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index 2a71347611d4..4467353f981b 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -94,6 +94,7 @@ choice
prompt "IOMMU default DMA mode"
depends on IOMMU_DMA

+ default IOMMU_DEFAULT_LAZY if INTEL_IOMMU
default IOMMU_DEFAULT_STRICT
help
This option allows an IOMMU DMA mode to be chosen at build time, to
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index be35284a2016..6763e516362c 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -360,7 +360,6 @@ int intel_iommu_enabled = 0;
EXPORT_SYMBOL_GPL(intel_iommu_enabled);

static int dmar_map_gfx = 1;
-static int intel_iommu_strict;
static int intel_iommu_superpage = 1;
static int iommu_identity_mapping;
static int iommu_skip_te_disable;
@@ -453,8 +452,7 @@ static int __init intel_iommu_setup(char *str)
pr_warn("intel_iommu=forcedac deprecated; use iommu.forcedac instead\n");
iommu_dma_forcedac = true;
} else if (!strncmp(str, "strict", 6)) {
- pr_info("Disable batched IOTLB flush\n");
- intel_iommu_strict = 1;
+ iommu_set_dma_strict(true);
} else if (!strncmp(str, "sp_off", 6)) {
pr_info("Disable supported super page\n");
intel_iommu_superpage = 0;
@@ -4392,9 +4390,9 @@ int __init intel_iommu_init(void)
* is likely to be much lower than the overhead of synchronizing
* the virtual and physical IOMMU page-tables.
*/
- if (!intel_iommu_strict && cap_caching_mode(iommu->cap)) {
- pr_warn("IOMMU batching is disabled due to virtualization");
- intel_iommu_strict = 1;
+ if (cap_caching_mode(iommu->cap)) {
+ pr_warn("IOMMU batching disallowed due to virtualization\n");
+ iommu_set_dma_strict(true);
}
iommu_device_sysfs_add(&iommu->iommu, NULL,
intel_iommu_groups,
@@ -4403,7 +4401,6 @@ int __init intel_iommu_init(void)
}
up_read(&dmar_global_lock);

- iommu_set_dma_strict(intel_iommu_strict);
bus_set_iommu(&pci_bus_type, &intel_iommu_ops);
if (si_domain && !hw_pass_through)
register_memory_notifier(&intel_iommu_memory_nb);
@@ -5666,8 +5663,8 @@ static void quirk_calpella_no_shadow_gtt(struct pci_dev *dev)
} else if (dmar_map_gfx) {
/* we have to ensure the gfx device is idle before we flush */
pci_info(dev, "Disabling batched IOTLB flush on Ironlake\n");
- intel_iommu_strict = 1;
- }
+ iommu_set_dma_strict(true);
+ }
}
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x0040, quirk_calpella_no_shadow_gtt);
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x0044, quirk_calpella_no_shadow_gtt);
--
2.26.2

2021-06-11 12:29:44

by John Garry

[permalink] [raw]
Subject: [PATCH v12 5/5] iommu: Remove mode argument from iommu_set_dma_strict()

We only ever now set strict mode enabled in iommu_set_dma_strict(), so
just remove the argument.

Signed-off-by: John Garry <[email protected]>
---
drivers/iommu/amd/init.c | 2 +-
drivers/iommu/intel/iommu.c | 6 +++---
drivers/iommu/iommu.c | 5 ++---
include/linux/iommu.h | 2 +-
4 files changed, 7 insertions(+), 8 deletions(-)

diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index 0e6ae6d68f14..27e9677ec303 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -3098,7 +3098,7 @@ static int __init parse_amd_iommu_options(char *str)
{
for (; *str; ++str) {
if (strncmp(str, "fullflush", 9) == 0)
- iommu_set_dma_strict(true);
+ iommu_set_dma_strict();
if (strncmp(str, "force_enable", 12) == 0)
amd_iommu_force_enable = true;
if (strncmp(str, "off", 3) == 0)
diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index 6763e516362c..e77b8b6e7838 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -452,7 +452,7 @@ static int __init intel_iommu_setup(char *str)
pr_warn("intel_iommu=forcedac deprecated; use iommu.forcedac instead\n");
iommu_dma_forcedac = true;
} else if (!strncmp(str, "strict", 6)) {
- iommu_set_dma_strict(true);
+ iommu_set_dma_strict();
} else if (!strncmp(str, "sp_off", 6)) {
pr_info("Disable supported super page\n");
intel_iommu_superpage = 0;
@@ -4392,7 +4392,7 @@ int __init intel_iommu_init(void)
*/
if (cap_caching_mode(iommu->cap)) {
pr_warn("IOMMU batching disallowed due to virtualization\n");
- iommu_set_dma_strict(true);
+ iommu_set_dma_strict();
}
iommu_device_sysfs_add(&iommu->iommu, NULL,
intel_iommu_groups,
@@ -5663,7 +5663,7 @@ static void quirk_calpella_no_shadow_gtt(struct pci_dev *dev)
} else if (dmar_map_gfx) {
/* we have to ensure the gfx device is idle before we flush */
pci_info(dev, "Disabling batched IOTLB flush on Ironlake\n");
- iommu_set_dma_strict(true);
+ iommu_set_dma_strict();
}
}
DECLARE_PCI_FIXUP_HEADER(PCI_VENDOR_ID_INTEL, 0x0040, quirk_calpella_no_shadow_gtt);
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index ccbd5d4c1a50..146cb71c7441 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -350,10 +350,9 @@ static int __init iommu_dma_setup(char *str)
}
early_param("iommu.strict", iommu_dma_setup);

-void iommu_set_dma_strict(bool strict)
+void iommu_set_dma_strict(void)
{
- if (strict || !(iommu_cmd_line & IOMMU_CMD_LINE_STRICT))
- iommu_dma_strict = strict;
+ iommu_dma_strict = true;
}

bool iommu_get_dma_strict(struct iommu_domain *domain)
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 32d448050bf7..754f67d6dd90 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -476,7 +476,7 @@ int iommu_enable_nesting(struct iommu_domain *domain);
int iommu_set_pgtable_quirks(struct iommu_domain *domain,
unsigned long quirks);

-void iommu_set_dma_strict(bool val);
+void iommu_set_dma_strict(void);
bool iommu_get_dma_strict(struct iommu_domain *domain);

extern int report_iommu_fault(struct iommu_domain *domain, struct device *dev,
--
2.26.2

2021-06-11 12:31:32

by John Garry

[permalink] [raw]
Subject: [PATCH v12 4/5] iommu/amd: Add support for IOMMU default DMA mode build options

From: Zhen Lei <[email protected]>

Make IOMMU_DEFAULT_LAZY default for when AMD_IOMMU config is set, which
matches current behaviour.

For "fullflush" param, just call iommu_set_dma_strict(true) directly.
Since this is called from __setup(), it should occur prior to
iommu_subsys_init(). As such, since we get a strict vs lazy mode print
there, drop the prints in amd_iommu_init_dma_ops().

Also drop global flag amd_iommu_unmap_flush, as it has no purpose
any longer.

Note that we should not conflict with iommu.strict param when we set
strict mode, as that param is not for x86.

[jpg: Rebase for relocated file an drop amd_iommu_unmap_flush]
Signed-off-by: Zhen Lei <[email protected]>
Signed-off-by: John Garry <[email protected]>
---
drivers/iommu/Kconfig | 2 +-
drivers/iommu/amd/amd_iommu_types.h | 6 ------
drivers/iommu/amd/init.c | 3 +--
drivers/iommu/amd/iommu.c | 6 ------
4 files changed, 2 insertions(+), 15 deletions(-)

diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index 4467353f981b..8c1e0f2bba0d 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -94,7 +94,7 @@ choice
prompt "IOMMU default DMA mode"
depends on IOMMU_DMA

- default IOMMU_DEFAULT_LAZY if INTEL_IOMMU
+ default IOMMU_DEFAULT_LAZY if (AMD_IOMMU || INTEL_IOMMU)
default IOMMU_DEFAULT_STRICT
help
This option allows an IOMMU DMA mode to be chosen at build time, to
diff --git a/drivers/iommu/amd/amd_iommu_types.h b/drivers/iommu/amd/amd_iommu_types.h
index 94c1a7a9876d..8dbe61e2b3c1 100644
--- a/drivers/iommu/amd/amd_iommu_types.h
+++ b/drivers/iommu/amd/amd_iommu_types.h
@@ -779,12 +779,6 @@ extern u16 amd_iommu_last_bdf;
/* allocation bitmap for domain ids */
extern unsigned long *amd_iommu_pd_alloc_bitmap;

-/*
- * If true, the addresses will be flushed on unmap time, not when
- * they are reused
- */
-extern bool amd_iommu_unmap_flush;
-
/* Smallest max PASID supported by any IOMMU in the system */
extern u32 amd_iommu_max_pasid;

diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
index 46280e6e1535..0e6ae6d68f14 100644
--- a/drivers/iommu/amd/init.c
+++ b/drivers/iommu/amd/init.c
@@ -161,7 +161,6 @@ u16 amd_iommu_last_bdf; /* largest PCI device id we have
to handle */
LIST_HEAD(amd_iommu_unity_map); /* a list of required unity mappings
we find in ACPI */
-bool amd_iommu_unmap_flush; /* if true, flush on every unmap */

LIST_HEAD(amd_iommu_list); /* list of all AMD IOMMUs in the
system */
@@ -3099,7 +3098,7 @@ static int __init parse_amd_iommu_options(char *str)
{
for (; *str; ++str) {
if (strncmp(str, "fullflush", 9) == 0)
- amd_iommu_unmap_flush = true;
+ iommu_set_dma_strict(true);
if (strncmp(str, "force_enable", 12) == 0)
amd_iommu_force_enable = true;
if (strncmp(str, "off", 3) == 0)
diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index b1fbf2c83df5..32b541ee2e11 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -1775,12 +1775,6 @@ void amd_iommu_domain_update(struct protection_domain *domain)
static void __init amd_iommu_init_dma_ops(void)
{
swiotlb = (iommu_default_passthrough() || sme_me_mask) ? 1 : 0;
-
- if (amd_iommu_unmap_flush)
- pr_info("IO/TLB flush on unmap enabled\n");
- else
- pr_info("Lazy IO/TLB flushing enabled\n");
- iommu_set_dma_strict(amd_iommu_unmap_flush);
}

int __init amd_iommu_init_api(void)
--
2.26.2

2021-06-12 01:23:48

by Lu Baolu

[permalink] [raw]
Subject: Re: [PATCH v12 2/5] iommu: Enhance IOMMU default DMA mode build options

On 2021/6/11 20:20, John Garry wrote:
> +config IOMMU_DEFAULT_LAZY
> + bool "lazy"
> + help
> + Support lazy mode, where for every IOMMU DMA unmap operation, the
> + flush operation of IOTLB and the free operation of IOVA are deferred.
> + They are only guaranteed to be done before the related IOVA will be
> + reused.
> +
> + The isolation provided in this mode is not as secure as STRICT mode,
> + such that a vulnerable time window may be created between the DMA
> + unmap and the mapping finally being torn down in the IOMMU, where the
> + device can still access the system memory. However this mode may

" ... and the mappings cached in the IOMMU IOTLB or device TLB finally
being invalidated, where the device probably can still access the memory
which has already been unmapped by the device driver."

Best regards,
baolu

2021-06-12 02:31:08

by Lu Baolu

[permalink] [raw]
Subject: Re: [PATCH v12 2/5] iommu: Enhance IOMMU default DMA mode build options

On 2021/6/11 20:20, John Garry wrote:
> +choice
> + prompt "IOMMU default DMA mode"

This is not explicit.

How about

"IOMMU DMA default cache invalidation policy"

?

Best regards,
baolu

> + depends on IOMMU_DMA
> +
> + default IOMMU_DEFAULT_STRICT
> + help
> + This option allows an IOMMU DMA mode to be chosen at build time, to
> + override the default DMA mode of each ARCH, removing the need to
> + pass in kernel parameters through command line. It is still possible
> + to provide ARCH-specific or common boot options to override this
> + option.
> +
> + If unsure, keep the default.

2021-06-12 02:38:42

by Lu Baolu

[permalink] [raw]
Subject: Re: [PATCH v12 3/5] iommu/vt-d: Add support for IOMMU default DMA mode build options

On 2021/6/11 20:20, John Garry wrote:
> diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
> index 2a71347611d4..4467353f981b 100644
> --- a/drivers/iommu/Kconfig
> +++ b/drivers/iommu/Kconfig
> @@ -94,6 +94,7 @@ choice
> prompt "IOMMU default DMA mode"
> depends on IOMMU_DMA
>
> + default IOMMU_DEFAULT_LAZY if INTEL_IOMMU
> default IOMMU_DEFAULT_STRICT

If two default values are different. Which one will be overridden?

Best regards,
baolu

2021-06-12 02:43:07

by Lu Baolu

[permalink] [raw]
Subject: Re: [PATCH v12 3/5] iommu/vt-d: Add support for IOMMU default DMA mode build options

On 2021/6/11 20:20, John Garry wrote:
> @@ -453,8 +452,7 @@ static int __init intel_iommu_setup(char *str)
> pr_warn("intel_iommu=forcedac deprecated; use iommu.forcedac instead\n");
> iommu_dma_forcedac = true;
> } else if (!strncmp(str, "strict", 6)) {
> - pr_info("Disable batched IOTLB flush\n");
> - intel_iommu_strict = 1;
> + iommu_set_dma_strict(true);

I would like to deprecate this command line and ask users to use
iommu.strict instead.

--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -436,7 +436,7 @@ static int __init intel_iommu_setup(char *str)
pr_warn("intel_iommu=forcedac deprecated; use
iommu.forcedac instead\n");
iommu_dma_forcedac = true;
} else if (!strncmp(str, "strict", 6)) {
- pr_info("Disable batched IOTLB flush\n");
+ pr_warn("intel_iommu=strict deprecated; use
iommu.strict instead\n");
intel_iommu_strict = 1;

Also update Documentation/admin-guide/kernel-parameters.txt accordingly.

Best regards,
baolu

2021-06-12 02:43:52

by Lu Baolu

[permalink] [raw]
Subject: Re: [PATCH v12 5/5] iommu: Remove mode argument from iommu_set_dma_strict()

On 2021/6/11 20:20, John Garry wrote:
> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> index ccbd5d4c1a50..146cb71c7441 100644
> --- a/drivers/iommu/iommu.c
> +++ b/drivers/iommu/iommu.c
> @@ -350,10 +350,9 @@ static int __init iommu_dma_setup(char *str)
> }
> early_param("iommu.strict", iommu_dma_setup);
>
> -void iommu_set_dma_strict(bool strict)
> +void iommu_set_dma_strict(void)
> {
> - if (strict || !(iommu_cmd_line & IOMMU_CMD_LINE_STRICT))
> - iommu_dma_strict = strict;
> + iommu_dma_strict = true;
> }

Will this change break the functionality of iommu.strict?

Best regards,
baolu