Hi,
the two patchsets posted as reply to this email implement KVM device
passthrough support for AMD IOMMU hardware. The changes to the previous
posts are descibed below
The first patchset is version 4 of the generic iommu api patchset which
generalizes the VT-d functions exported to KVM into a common api where
the AMD IOMMU code can plug into. In this version the patchset was
rebased to the latest post of Han Weidong's patches.
The second patchset finally implements the KVM device passthrough
support in the AMD IOMMU code. Together with KVM-79 I successfully
passed an 10GBit network card into an KVM guest. In this version the
patchset was changed to remove any device before a protection domain is
freed instead of printing a BUG. Also the patchset was rebased to the
updated IOMMU-API patches.
These two patchsets apply in order in top of the latest post of
Han Weidong's Multiple device assignement support patches. Anybody who
wants to try this out can pull the whole stuff from
git://git.kernel.org/pub/scm/linux/kernel/git/joro/linux-2.6-iommu.git kvm-amd-iommu
Please give these patches a good review.
Thanks,
Joerg
--
| AMD Saxony Limited Liability Company & Co. KG
Operating | Wilschdorfer Landstr. 101, 01109 Dresden, Germany
System | Register Court Dresden: HRA 4896
Research | General Partner authorized to represent:
Center | AMD Saxony LLC (Wilmington, Delaware, US)
| General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy
Signed-off-by: Joerg Roedel <[email protected]>
---
drivers/pci/intel-iommu.c | 33 ++++++++++++++++++++-------------
include/linux/intel-iommu.h | 4 ----
2 files changed, 20 insertions(+), 17 deletions(-)
diff --git a/drivers/pci/intel-iommu.c b/drivers/pci/intel-iommu.c
index db9a26c..8af6c96 100644
--- a/drivers/pci/intel-iommu.c
+++ b/drivers/pci/intel-iommu.c
@@ -3047,20 +3047,28 @@ static void intel_iommu_detach_device(struct iommu_domain *domain,
vm_domain_remove_one_dev_info(dmar_domain, pdev);
}
-int intel_iommu_map_address(struct dmar_domain *domain, dma_addr_t iova,
- u64 hpa, size_t size, int prot)
+static int intel_iommu_map_range(struct iommu_domain *domain,
+ unsigned long iova, phys_addr_t hpa,
+ size_t size, int iommu_prot)
{
+ struct dmar_domain *dmar_domain = domain->priv;
u64 max_addr;
int addr_width;
+ int prot = 0;
int ret;
+ if (iommu_prot & IOMMU_READ)
+ prot |= DMA_PTE_READ;
+ if (iommu_prot & IOMMU_WRITE)
+ prot |= DMA_PTE_WRITE;
+
max_addr = (iova & VTD_PAGE_MASK) + VTD_PAGE_ALIGN(size);
- if (domain->max_addr < max_addr) {
+ if (dmar_domain->max_addr < max_addr) {
int min_agaw;
u64 end;
/* check if minimum agaw is sufficient for mapped address */
- min_agaw = vm_domain_min_agaw(domain);
+ min_agaw = vm_domain_min_agaw(dmar_domain);
addr_width = agaw_to_width(min_agaw);
end = DOMAIN_MAX_ADDR(addr_width);
end = end & VTD_PAGE_MASK;
@@ -3070,28 +3078,27 @@ int intel_iommu_map_address(struct dmar_domain *domain, dma_addr_t iova,
__func__, min_agaw, max_addr);
return -EFAULT;
}
- domain->max_addr = max_addr;
+ dmar_domain->max_addr = max_addr;
}
- ret = domain_page_mapping(domain, iova, hpa, size, prot);
+ ret = domain_page_mapping(dmar_domain, iova, hpa, size, prot);
return ret;
}
-EXPORT_SYMBOL_GPL(intel_iommu_map_address);
-void intel_iommu_unmap_address(struct dmar_domain *domain,
- dma_addr_t iova, size_t size)
+static void intel_iommu_unmap_range(struct iommu_domain *domain,
+ unsigned long iova, size_t size)
{
+ struct dmar_domain *dmar_domain = domain->priv;
dma_addr_t base;
/* The address might not be aligned */
base = iova & VTD_PAGE_MASK;
size = VTD_PAGE_ALIGN(size);
- dma_pte_clear_range(domain, base, base + size);
+ dma_pte_clear_range(dmar_domain, base, base + size);
- if (domain->max_addr == base + size)
- domain->max_addr = base;
+ if (dmar_domain->max_addr == base + size)
+ dmar_domain->max_addr = base;
}
-EXPORT_SYMBOL_GPL(intel_iommu_unmap_address);
int intel_iommu_found(void)
{
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 9909c5a..6bc26e0 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -330,10 +330,6 @@ extern int qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr,
extern void qi_submit_sync(struct qi_desc *desc, struct intel_iommu *iommu);
-int intel_iommu_map_address(struct dmar_domain *domain, dma_addr_t iova,
- u64 hpa, size_t size, int prot);
-void intel_iommu_unmap_address(struct dmar_domain *domain,
- dma_addr_t iova, size_t size);
u64 intel_iommu_iova_to_phys(struct dmar_domain *domain, u64 iova);
#ifdef CONFIG_DMAR
--
1.5.6.4
This patch series makes the current KVM device passthrough code generic
enough so that other IOMMU implementation can also plug into this code.
It works by factoring the functions Vt-d code exports to KVM into a
generic interface which allows different backends.
This is the fourth version of the patchset. I rebased these patches onto
the 17-patches post of Han Weidongs multiple device assignment work and
included changes to the commit-messages according the comments I got
from the review.
This a basic implementation of a generic interface. It can and should be
improved later to support more types of hardware IOMMUs then VT-d and
AMD IOMMU.
Since I have no VT-d hardware available these patches are only compile
tested on x86 and ia64 for now.
Please review, comment and test these patches.
Thanks,
Joerg
diffstat:
arch/ia64/Kconfig | 3 +
arch/ia64/include/asm/kvm_host.h | 2 +-
arch/ia64/kvm/Makefile | 4 +-
arch/ia64/kvm/kvm-ia64.c | 3 +-
arch/x86/Kconfig | 3 +
arch/x86/include/asm/kvm_host.h | 2 +-
arch/x86/kvm/Makefile | 4 +-
arch/x86/kvm/x86.c | 3 +-
drivers/base/Makefile | 1 +
drivers/base/iommu.c | 100 +++++++++++++++++++++++++++++++
drivers/pci/intel-iommu.c | 121 ++++++++++++++++++++++---------------
include/linux/intel-iommu.h | 21 -------
include/linux/iommu.h | 112 +++++++++++++++++++++++++++++++++++
include/linux/kvm_host.h | 6 +-
virt/kvm/{vtd.c => iommu.c} | 45 +++++++--------
virt/kvm/kvm_main.c | 2 +-
16 files changed, 326 insertions(+), 106 deletions(-)
This patch introduces the API to abstract the exported VT-d functions
for KVM into a generic API. This way the AMD IOMMU implementation can
plug into this API later.
Signed-off-by: Joerg Roedel <[email protected]>
---
include/linux/iommu.h | 112 +++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 112 insertions(+), 0 deletions(-)
create mode 100644 include/linux/iommu.h
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
new file mode 100644
index 0000000..8a7bfb1
--- /dev/null
+++ b/include/linux/iommu.h
@@ -0,0 +1,112 @@
+/*
+ * Copyright (C) 2007-2008 Advanced Micro Devices, Inc.
+ * Author: Joerg Roedel <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published
+ * by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#ifndef __LINUX_IOMMU_H
+#define __LINUX_IOMMU_H
+
+#define IOMMU_READ (1)
+#define IOMMU_WRITE (2)
+
+struct device;
+
+struct iommu_domain {
+ void *priv;
+};
+
+struct iommu_ops {
+ int (*domain_init)(struct iommu_domain *domain);
+ void (*domain_destroy)(struct iommu_domain *domain);
+ int (*attach_dev)(struct iommu_domain *domain, struct device *dev);
+ void (*detach_dev)(struct iommu_domain *domain, struct device *dev);
+ int (*map)(struct iommu_domain *domain, unsigned long iova,
+ phys_addr_t paddr, size_t size, int prot);
+ void (*unmap)(struct iommu_domain *domain, unsigned long iova,
+ size_t size);
+ phys_addr_t (*iova_to_phys)(struct iommu_domain *domain,
+ unsigned long iova);
+};
+
+#ifdef CONFIG_IOMMU_API
+
+extern void register_iommu(struct iommu_ops *ops);
+extern bool iommu_found(void);
+extern struct iommu_domain *iommu_domain_alloc(void);
+extern void iommu_domain_free(struct iommu_domain *domain);
+extern int iommu_attach_device(struct iommu_domain *domain,
+ struct device *dev);
+extern void iommu_detach_device(struct iommu_domain *domain,
+ struct device *dev);
+extern int iommu_map_range(struct iommu_domain *domain, unsigned long iova,
+ phys_addr_t paddr, size_t size, int prot);
+extern void iommu_unmap_range(struct iommu_domain *domain, unsigned long iova,
+ size_t size);
+extern phys_addr_t iommu_iova_to_phys(struct iommu_domain *domain,
+ unsigned long iova);
+
+#else /* CONFIG_IOMMU_API */
+
+static inline void register_iommu(struct iommu_ops *ops)
+{
+}
+
+static inline bool iommu_found(void)
+{
+ return false;
+}
+
+static inline struct iommu_domain *iommu_domain_alloc(void)
+{
+ return NULL;
+}
+
+static inline void iommu_domain_free(struct iommu_domain *domain)
+{
+}
+
+static inline int iommu_attach_device(struct iommu_domain *domain,
+ struct device *dev)
+{
+ return -ENODEV;
+}
+
+static inline void iommu_detach_device(struct iommu_domain *domain,
+ struct device *dev)
+{
+}
+
+static inline int iommu_map_range(struct iommu_domain *domain,
+ unsigned long iova, phys_addr_t paddr,
+ size_t size, int prot)
+{
+ return -ENODEV;
+}
+
+static inline void iommu_unmap_range(struct iommu_domain *domain,
+ unsigned long iova, size_t size)
+{
+}
+
+static inline phys_addr_t iommu_iova_to_phys(struct iommu_domain *domain,
+ unsigned long iova)
+{
+ return 0;
+}
+
+#endif /* CONFIG_IOMMU_API */
+
+#endif /* __LINUX_IOMMU_H */
--
1.5.6.4
Signed-off-by: Joerg Roedel <[email protected]>
---
drivers/pci/intel-iommu.c | 7 ++++---
include/linux/intel-iommu.h | 2 --
2 files changed, 4 insertions(+), 5 deletions(-)
diff --git a/drivers/pci/intel-iommu.c b/drivers/pci/intel-iommu.c
index 8af6c96..7128105 100644
--- a/drivers/pci/intel-iommu.c
+++ b/drivers/pci/intel-iommu.c
@@ -3106,15 +3106,16 @@ int intel_iommu_found(void)
}
EXPORT_SYMBOL_GPL(intel_iommu_found);
-u64 intel_iommu_iova_to_phys(struct dmar_domain *domain, u64 iova)
+static phys_addr_t intel_iommu_iova_to_phys(struct iommu_domain *domain,
+ unsigned long iova)
{
+ struct dmar_domain *dmar_domain = domain->priv;
struct dma_pte *pte;
u64 phys = 0;
- pte = addr_to_dma_pte(domain, iova);
+ pte = addr_to_dma_pte(dmar_domain, iova);
if (pte)
phys = dma_pte_addr(pte);
return phys;
}
-EXPORT_SYMBOL_GPL(intel_iommu_iova_to_phys);
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 6bc26e0..26ccc02 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -330,8 +330,6 @@ extern int qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr,
extern void qi_submit_sync(struct qi_desc *desc, struct intel_iommu *iommu);
-u64 intel_iommu_iova_to_phys(struct dmar_domain *domain, u64 iova);
-
#ifdef CONFIG_DMAR
int intel_iommu_found(void);
#else /* CONFIG_DMAR */
--
1.5.6.4
Impact: file renamed
The code in the vtd.c file can be reused for other IOMMUs as well. So
rename it to make it clear that it handle more than VT-d.
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/ia64/kvm/Makefile | 2 +-
arch/x86/kvm/Makefile | 2 +-
virt/kvm/{vtd.c => iommu.c} | 0
3 files changed, 2 insertions(+), 2 deletions(-)
rename virt/kvm/{vtd.c => iommu.c} (100%)
diff --git a/arch/ia64/kvm/Makefile b/arch/ia64/kvm/Makefile
index 76464dc..cb69dfc 100644
--- a/arch/ia64/kvm/Makefile
+++ b/arch/ia64/kvm/Makefile
@@ -52,7 +52,7 @@ common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o ioapic.o \
coalesced_mmio.o irq_comm.o)
ifeq ($(CONFIG_DMAR),y)
-common-objs += $(addprefix ../../../virt/kvm/, vtd.o)
+common-objs += $(addprefix ../../../virt/kvm/, iommu.o)
endif
kvm-objs := $(common-objs) kvm-ia64.o kvm_fw.o
diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile
index c023435..00f46c2 100644
--- a/arch/x86/kvm/Makefile
+++ b/arch/x86/kvm/Makefile
@@ -8,7 +8,7 @@ ifeq ($(CONFIG_KVM_TRACE),y)
common-objs += $(addprefix ../../../virt/kvm/, kvm_trace.o)
endif
ifeq ($(CONFIG_DMAR),y)
-common-objs += $(addprefix ../../../virt/kvm/, vtd.o)
+common-objs += $(addprefix ../../../virt/kvm/, iommu.o)
endif
EXTRA_CFLAGS += -Ivirt/kvm -Iarch/x86/kvm
diff --git a/virt/kvm/vtd.c b/virt/kvm/iommu.c
similarity index 100%
rename from virt/kvm/vtd.c
rename to virt/kvm/iommu.c
--
1.5.6.4
Signed-off-by: Joerg Roedel <[email protected]>
---
drivers/pci/intel-iommu.c | 6 ------
include/linux/intel-iommu.h | 9 ---------
2 files changed, 0 insertions(+), 15 deletions(-)
diff --git a/drivers/pci/intel-iommu.c b/drivers/pci/intel-iommu.c
index 81e04ec..ecb5fd3 100644
--- a/drivers/pci/intel-iommu.c
+++ b/drivers/pci/intel-iommu.c
@@ -3105,12 +3105,6 @@ static void intel_iommu_unmap_range(struct iommu_domain *domain,
dmar_domain->max_addr = base;
}
-int intel_iommu_found(void)
-{
- return g_num_of_iommus;
-}
-EXPORT_SYMBOL_GPL(intel_iommu_found);
-
static phys_addr_t intel_iommu_iova_to_phys(struct iommu_domain *domain,
unsigned long iova)
{
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 26ccc02..c4f6c10 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -330,15 +330,6 @@ extern int qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr,
extern void qi_submit_sync(struct qi_desc *desc, struct intel_iommu *iommu);
-#ifdef CONFIG_DMAR
-int intel_iommu_found(void);
-#else /* CONFIG_DMAR */
-static inline int intel_iommu_found(void)
-{
- return 0;
-}
-#endif /* CONFIG_DMAR */
-
extern void *intel_alloc_coherent(struct device *, size_t, dma_addr_t *, gfp_t);
extern void intel_free_coherent(struct device *, size_t, void *, dma_addr_t);
extern dma_addr_t intel_map_single(struct device *, phys_addr_t, size_t, int);
--
1.5.6.4
These two IOMMUs can implement the current version of this API. So
select the API if one or both of these IOMMU drivers is selected.
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/ia64/Kconfig | 3 +++
arch/x86/Kconfig | 3 +++
drivers/base/Makefile | 1 +
3 files changed, 7 insertions(+), 0 deletions(-)
diff --git a/arch/ia64/Kconfig b/arch/ia64/Kconfig
index 6bd91ed..6a7b0c9 100644
--- a/arch/ia64/Kconfig
+++ b/arch/ia64/Kconfig
@@ -687,3 +687,6 @@ config IRQ_PER_CPU
config IOMMU_HELPER
def_bool (IA64_HP_ZX1 || IA64_HP_ZX1_SWIOTLB || IA64_GENERIC || SWIOTLB)
+
+config IOMMU_API
+ def_bool (DMAR)
diff --git a/arch/x86/Kconfig b/arch/x86/Kconfig
index ac22bb7..b9f7187 100644
--- a/arch/x86/Kconfig
+++ b/arch/x86/Kconfig
@@ -580,6 +580,9 @@ config SWIOTLB
config IOMMU_HELPER
def_bool (CALGARY_IOMMU || GART_IOMMU || SWIOTLB || AMD_IOMMU)
+config IOMMU_API
+ def_bool (AMD_IOMMU || DMAR)
+
config MAXSMP
bool "Configure Maximum number of SMP Processors and NUMA Nodes"
depends on X86_64 && SMP && BROKEN
diff --git a/drivers/base/Makefile b/drivers/base/Makefile
index c666373..b5b8ba5 100644
--- a/drivers/base/Makefile
+++ b/drivers/base/Makefile
@@ -11,6 +11,7 @@ obj-$(CONFIG_FW_LOADER) += firmware_class.o
obj-$(CONFIG_NUMA) += node.o
obj-$(CONFIG_MEMORY_HOTPLUG_SPARSE) += memory.o
obj-$(CONFIG_SMP) += topology.o
+obj-$(CONFIG_IOMMU_API) += iommu.o
ifeq ($(CONFIG_SYSFS),y)
obj-$(CONFIG_MODULES) += module.o
endif
--
1.5.6.4
Signed-off-by: Joerg Roedel <[email protected]>
---
drivers/pci/intel-iommu.c | 33 ++++++++++++++++++---------------
include/linux/intel-iommu.h | 2 --
2 files changed, 18 insertions(+), 17 deletions(-)
diff --git a/drivers/pci/intel-iommu.c b/drivers/pci/intel-iommu.c
index 772fb22..5c95a5a 100644
--- a/drivers/pci/intel-iommu.c
+++ b/drivers/pci/intel-iommu.c
@@ -34,6 +34,7 @@
#include <linux/mempool.h>
#include <linux/timer.h>
#include <linux/iova.h>
+#include <linux/iommu.h>
#include <linux/intel-iommu.h>
#include <asm/cacheflush.h>
#include <asm/iommu.h>
@@ -2962,32 +2963,34 @@ static void vm_domain_exit(struct dmar_domain *domain)
free_domain_mem(domain);
}
-struct dmar_domain *intel_iommu_alloc_domain(void)
+static int intel_iommu_domain_init(struct iommu_domain *domain)
{
- struct dmar_domain *domain;
+ struct dmar_domain *dmar_domain;
- domain = iommu_alloc_vm_domain();
- if (!domain) {
+ dmar_domain = iommu_alloc_vm_domain();
+ if (!dmar_domain) {
printk(KERN_ERR
- "intel_iommu_domain_alloc: domain == NULL\n");
- return NULL;
+ "intel_iommu_domain_init: dmar_domain == NULL\n");
+ return -ENOMEM;
}
- if (vm_domain_init(domain, DEFAULT_DOMAIN_ADDRESS_WIDTH)) {
+ if (vm_domain_init(dmar_domain, DEFAULT_DOMAIN_ADDRESS_WIDTH)) {
printk(KERN_ERR
- "intel_iommu_domain_alloc: domain_init() failed\n");
- vm_domain_exit(domain);
- return NULL;
+ "intel_iommu_domain_init() failed\n");
+ vm_domain_exit(dmar_domain);
+ return -ENOMEM;
}
+ domain->priv = dmar_domain;
- return domain;
+ return 0;
}
-EXPORT_SYMBOL_GPL(intel_iommu_alloc_domain);
-void intel_iommu_free_domain(struct dmar_domain *domain)
+static void intel_iommu_domain_destroy(struct iommu_domain *domain)
{
- vm_domain_exit(domain);
+ struct dmar_domain *dmar_domain = domain->priv;
+
+ domain->priv = NULL;
+ vm_domain_exit(dmar_domain);
}
-EXPORT_SYMBOL_GPL(intel_iommu_free_domain);
int intel_iommu_attach_device(struct dmar_domain *domain,
struct pci_dev *pdev)
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 07973c4..0a7ba0c 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -330,8 +330,6 @@ extern int qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr,
extern void qi_submit_sync(struct qi_desc *desc, struct intel_iommu *iommu);
-struct dmar_domain *intel_iommu_alloc_domain(void);
-void intel_iommu_free_domain(struct dmar_domain *domain);
int intel_iommu_attach_device(struct dmar_domain *domain,
struct pci_dev *pdev);
void intel_iommu_detach_device(struct dmar_domain *domain,
--
1.5.6.4
This patchset implements KVM device assignment support in the AMD IOMMU
driver. It uses the generic interface from the iommu-api patchset and
was successfully tested using an 10GBit network card passed through to
an KVM guest.
diffstat:
arch/x86/include/asm/amd_iommu_types.h | 15 +-
arch/x86/kernel/amd_iommu.c | 448 +++++++++++++++++++++++++++++---
2 files changed, 416 insertions(+), 47 deletions(-)
Signed-off-by: Joerg Roedel <[email protected]>
---
drivers/pci/intel-iommu.c | 27 +++++++++++++++------------
include/linux/intel-iommu.h | 4 ----
2 files changed, 15 insertions(+), 16 deletions(-)
diff --git a/drivers/pci/intel-iommu.c b/drivers/pci/intel-iommu.c
index 5c95a5a..db9a26c 100644
--- a/drivers/pci/intel-iommu.c
+++ b/drivers/pci/intel-iommu.c
@@ -2992,9 +2992,11 @@ static void intel_iommu_domain_destroy(struct iommu_domain *domain)
vm_domain_exit(dmar_domain);
}
-int intel_iommu_attach_device(struct dmar_domain *domain,
- struct pci_dev *pdev)
+static int intel_iommu_attach_device(struct iommu_domain *domain,
+ struct device *dev)
{
+ struct dmar_domain *dmar_domain = domain->priv;
+ struct pci_dev *pdev = to_pci_dev(dev);
struct intel_iommu *iommu;
int addr_width;
u64 end;
@@ -3006,7 +3008,7 @@ int intel_iommu_attach_device(struct dmar_domain *domain,
old_domain = find_domain(pdev);
if (old_domain) {
- if (domain->flags & DOMAIN_FLAG_VIRTUAL_MACHINE)
+ if (dmar_domain->flags & DOMAIN_FLAG_VIRTUAL_MACHINE)
vm_domain_remove_one_dev_info(old_domain, pdev);
else
domain_remove_dev_info(old_domain);
@@ -3021,28 +3023,29 @@ int intel_iommu_attach_device(struct dmar_domain *domain,
addr_width = agaw_to_width(iommu->agaw);
end = DOMAIN_MAX_ADDR(addr_width);
end = end & VTD_PAGE_MASK;
- if (end < domain->max_addr) {
+ if (end < dmar_domain->max_addr) {
printk(KERN_ERR "%s: iommu agaw (%d) is not "
"sufficient for the mapped address (%llx)\n",
- __func__, iommu->agaw, domain->max_addr);
+ __func__, iommu->agaw, dmar_domain->max_addr);
return -EFAULT;
}
- ret = domain_context_mapping(domain, pdev);
+ ret = domain_context_mapping(dmar_domain, pdev);
if (ret)
return ret;
- ret = vm_domain_add_dev_info(domain, pdev);
+ ret = vm_domain_add_dev_info(dmar_domain, pdev);
return ret;
}
-EXPORT_SYMBOL_GPL(intel_iommu_attach_device);
-void intel_iommu_detach_device(struct dmar_domain *domain,
- struct pci_dev *pdev)
+static void intel_iommu_detach_device(struct iommu_domain *domain,
+ struct device *dev)
{
- vm_domain_remove_one_dev_info(domain, pdev);
+ struct dmar_domain *dmar_domain = domain->priv;
+ struct pci_dev *pdev = to_pci_dev(dev);
+
+ vm_domain_remove_one_dev_info(dmar_domain, pdev);
}
-EXPORT_SYMBOL_GPL(intel_iommu_detach_device);
int intel_iommu_map_address(struct dmar_domain *domain, dma_addr_t iova,
u64 hpa, size_t size, int prot)
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 0a7ba0c..9909c5a 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -330,10 +330,6 @@ extern int qi_flush_iotlb(struct intel_iommu *iommu, u16 did, u64 addr,
extern void qi_submit_sync(struct qi_desc *desc, struct intel_iommu *iommu);
-int intel_iommu_attach_device(struct dmar_domain *domain,
- struct pci_dev *pdev);
-void intel_iommu_detach_device(struct dmar_domain *domain,
- struct pci_dev *pdev);
int intel_iommu_map_address(struct dmar_domain *domain, dma_addr_t iova,
u64 hpa, size_t size, int prot);
void intel_iommu_unmap_address(struct dmar_domain *domain,
--
1.5.6.4
Impact: add helper functions to detach a device from a domain
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 41 +++++++++++++++++++++++++++++++++++++++++
1 files changed, 41 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 61584d5..47158e3 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -1469,3 +1469,44 @@ free_domains:
return ret;
}
+
+/*****************************************************************************
+ *
+ * The following functions belong to the exported interface of AMD IOMMU
+ *
+ * This interface allows access to lower level functions of the IOMMU
+ * like protection domain handling and assignement of devices to domains
+ * which is not possible with the dma_ops interface.
+ *
+ *****************************************************************************/
+
+static void __detach_device(struct protection_domain *domain, u16 devid)
+{
+
+ /* lock domain */
+ spin_lock(&domain->lock);
+
+ /* remove domain from the lookup table */
+ amd_iommu_pd_table[devid] = NULL;
+
+ /* remove entry from the device table seen by the hardware */
+ amd_iommu_dev_table[devid].data[0] = IOMMU_PTE_P | IOMMU_PTE_TV;
+ amd_iommu_dev_table[devid].data[1] = 0;
+ amd_iommu_dev_table[devid].data[2] = 0;
+
+ /* decrease reference counter */
+ domain->dev_cnt -= 1;
+
+ /* ready */
+ spin_unlock(&domain->lock);
+}
+
+static void detach_device(struct protection_domain *domain, u16 devid)
+{
+ unsigned long flags;
+
+ /* lock device table */
+ write_lock_irqsave(&amd_iommu_devtable_lock, flags);
+ __detach_device(domain, devid);
+ write_unlock_irqrestore(&amd_iommu_devtable_lock, flags);
+}
--
1.5.6.4
Impact: bugfix
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 153db6e..e96a2f3 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -600,7 +600,7 @@ static void dma_ops_free_pagetable(struct dma_ops_domain *dma_dom)
continue;
p2 = IOMMU_PTE_PAGE(p1[i]);
- for (j = 0; j < 512; ++i) {
+ for (j = 0; j < 512; ++j) {
if (!IOMMU_PTE_PRESENT(p2[j]))
continue;
p3 = IOMMU_PTE_PAGE(p2[j]);
--
1.5.6.4
Impact: add a generic function for releasing protection domains
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 21 +++++++++++++++++++++
1 files changed, 21 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index cb7db66..d5581bb 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -1552,3 +1552,24 @@ out_free:
return -ENOMEM;
}
+
+static void amd_iommu_domain_destroy(struct iommu_domain *dom)
+{
+ struct protection_domain *domain = dom->priv;
+
+ if (!domain)
+ return;
+
+ if (domain->dev_cnt > 0)
+ cleanup_domain(domain);
+
+ BUG_ON(domain->dev_cnt != 0);
+
+ free_pagetable(domain);
+
+ domain_id_free(domain->id);
+
+ kfree(domain);
+
+ dom->priv = NULL;
+}
--
1.5.6.4
Impact: split one function into three
The separate functions are required synchronize commands across all
hardware IOMMUs in the system.
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 65 +++++++++++++++++++++++++++++--------------
1 files changed, 44 insertions(+), 21 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 11de0f1..db89001 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -195,6 +195,46 @@ static int iommu_queue_command(struct amd_iommu *iommu, struct iommu_cmd *cmd)
}
/*
+ * This function waits until an IOMMU has completed a completion
+ * wait command
+ */
+static void __iommu_wait_for_completion(struct amd_iommu *iommu)
+{
+ int ready = 0;
+ unsigned status = 0;
+ unsigned long i = 0;
+
+ while (!ready && (i < EXIT_LOOP_COUNT)) {
+ ++i;
+ /* wait for the bit to become one */
+ status = readl(iommu->mmio_base + MMIO_STATUS_OFFSET);
+ ready = status & MMIO_STATUS_COM_WAIT_INT_MASK;
+ }
+
+ /* set bit back to zero */
+ status &= ~MMIO_STATUS_COM_WAIT_INT_MASK;
+ writel(status, iommu->mmio_base + MMIO_STATUS_OFFSET);
+
+ if (unlikely((i == EXIT_LOOP_COUNT) && printk_ratelimit()))
+ printk(KERN_WARNING "AMD IOMMU: Completion wait loop failed\n");
+}
+
+/*
+ * This function queues a completion wait command into the command
+ * buffer of an IOMMU
+ */
+static int __iommu_completion_wait(struct amd_iommu *iommu)
+{
+ struct iommu_cmd cmd;
+
+ memset(&cmd, 0, sizeof(cmd));
+ cmd.data[0] = CMD_COMPL_WAIT_INT_MASK;
+ CMD_SET_TYPE(&cmd, CMD_COMPL_WAIT);
+
+ return __iommu_queue_command(iommu, &cmd);
+}
+
+/*
* This function is called whenever we need to ensure that the IOMMU has
* completed execution of all commands we sent. It sends a
* COMPLETION_WAIT command and waits for it to finish. The IOMMU informs
@@ -203,14 +243,8 @@ static int iommu_queue_command(struct amd_iommu *iommu, struct iommu_cmd *cmd)
*/
static int iommu_completion_wait(struct amd_iommu *iommu)
{
- int ret = 0, ready = 0;
- unsigned status = 0;
- struct iommu_cmd cmd;
- unsigned long flags, i = 0;
-
- memset(&cmd, 0, sizeof(cmd));
- cmd.data[0] = CMD_COMPL_WAIT_INT_MASK;
- CMD_SET_TYPE(&cmd, CMD_COMPL_WAIT);
+ int ret = 0;
+ unsigned long flags;
spin_lock_irqsave(&iommu->lock, flags);
@@ -219,24 +253,13 @@ static int iommu_completion_wait(struct amd_iommu *iommu)
iommu->need_sync = 0;
- ret = __iommu_queue_command(iommu, &cmd);
+ ret = __iommu_completion_wait(iommu);
if (ret)
goto out;
- while (!ready && (i < EXIT_LOOP_COUNT)) {
- ++i;
- /* wait for the bit to become one */
- status = readl(iommu->mmio_base + MMIO_STATUS_OFFSET);
- ready = status & MMIO_STATUS_COM_WAIT_INT_MASK;
- }
-
- /* set bit back to zero */
- status &= ~MMIO_STATUS_COM_WAIT_INT_MASK;
- writel(status, iommu->mmio_base + MMIO_STATUS_OFFSET);
+ __iommu_wait_for_completion(iommu);
- if (unlikely((i == EXIT_LOOP_COUNT) && printk_ratelimit()))
- printk(KERN_WARNING "AMD IOMMU: Completion wait loop failed\n");
out:
spin_unlock_irqrestore(&iommu->lock, flags);
--
1.5.6.4
Impact: detect when a driver uses a device assigned otherwise
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 21 +++++++++++++++++++++
1 files changed, 21 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 6d26544..2173283 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -787,6 +787,15 @@ free_dma_dom:
}
/*
+ * little helper function to check whether a given protection domain is a
+ * dma_ops domain
+ */
+static bool dma_ops_domain(struct protection_domain *domain)
+{
+ return domain->flags & PD_DMA_OPS_MASK;
+}
+
+/*
* Find out the protection domain structure for a given PCI device. This
* will give us the pointer to the page table root for example.
*/
@@ -1091,6 +1100,8 @@ static dma_addr_t map_single(struct device *dev, phys_addr_t paddr,
/* device not handled by any AMD IOMMU */
return (dma_addr_t)paddr;
+ BUG_ON(!dma_ops_domain(domain));
+
spin_lock_irqsave(&domain->lock, flags);
addr = __map_single(dev, iommu, domain->priv, paddr, size, dir, false,
dma_mask);
@@ -1121,6 +1132,8 @@ static void unmap_single(struct device *dev, dma_addr_t dma_addr,
/* device not handled by any AMD IOMMU */
return;
+ BUG_ON(!dma_ops_domain(domain));
+
spin_lock_irqsave(&domain->lock, flags);
__unmap_single(iommu, domain->priv, dma_addr, size, dir);
@@ -1175,6 +1188,8 @@ static int map_sg(struct device *dev, struct scatterlist *sglist,
if (!iommu || !domain)
return map_sg_no_iommu(dev, sglist, nelems, dir);
+ BUG_ON(!dma_ops_domain(domain));
+
spin_lock_irqsave(&domain->lock, flags);
for_each_sg(sglist, s, nelems, i) {
@@ -1228,6 +1243,8 @@ static void unmap_sg(struct device *dev, struct scatterlist *sglist,
!get_device_resources(dev, &iommu, &domain, &devid))
return;
+ BUG_ON(!dma_ops_domain(domain));
+
spin_lock_irqsave(&domain->lock, flags);
for_each_sg(sglist, s, nelems, i) {
@@ -1273,6 +1290,8 @@ static void *alloc_coherent(struct device *dev, size_t size,
return virt_addr;
}
+ BUG_ON(!dma_ops_domain(domain));
+
if (!dma_mask)
dma_mask = *dev->dma_mask;
@@ -1314,6 +1333,8 @@ static void free_coherent(struct device *dev, size_t size,
if (!iommu || !domain)
goto free_mem;
+ BUG_ON(!dma_ops_domain(domain));
+
spin_lock_irqsave(&domain->lock, flags);
__unmap_single(iommu, domain->priv, dma_addr, size, DMA_BIDIRECTIONAL);
--
1.5.6.4
This API can be used by KVM for accessing different types of IOMMUs to
do device passthrough to guests. Beside that this API can also be used
by device drivers to map non-linear host memory into dma-linear
addresses to prevent sgather-gather DMA. UIO may be another user for
this API.
Signed-off-by: Joerg Roedel <[email protected]>
Acked-by: Greg Kroah-Hartman <[email protected]>
---
drivers/base/iommu.c | 100 ++++++++++++++++++++++++++++++++++++++++++++++++++
1 files changed, 100 insertions(+), 0 deletions(-)
create mode 100644 drivers/base/iommu.c
diff --git a/drivers/base/iommu.c b/drivers/base/iommu.c
new file mode 100644
index 0000000..5e039d4
--- /dev/null
+++ b/drivers/base/iommu.c
@@ -0,0 +1,100 @@
+/*
+ * Copyright (C) 2007-2008 Advanced Micro Devices, Inc.
+ * Author: Joerg Roedel <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License version 2 as published
+ * by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
+ */
+
+#include <linux/bug.h>
+#include <linux/types.h>
+#include <linux/errno.h>
+#include <linux/iommu.h>
+
+static struct iommu_ops *iommu_ops;
+
+void register_iommu(struct iommu_ops *ops)
+{
+ if (iommu_ops)
+ BUG();
+
+ iommu_ops = ops;
+}
+
+bool iommu_found()
+{
+ return iommu_ops != NULL;
+}
+EXPORT_SYMBOL_GPL(iommu_found);
+
+struct iommu_domain *iommu_domain_alloc(void)
+{
+ struct iommu_domain *domain;
+ int ret;
+
+ domain = kmalloc(sizeof(*domain), GFP_KERNEL);
+ if (!domain)
+ return NULL;
+
+ ret = iommu_ops->domain_init(domain);
+ if (ret)
+ goto out_free;
+
+ return domain;
+
+out_free:
+ kfree(domain);
+
+ return NULL;
+}
+EXPORT_SYMBOL_GPL(iommu_domain_alloc);
+
+void iommu_domain_free(struct iommu_domain *domain)
+{
+ iommu_ops->domain_destroy(domain);
+ kfree(domain);
+}
+EXPORT_SYMBOL_GPL(iommu_domain_free);
+
+int iommu_attach_device(struct iommu_domain *domain, struct device *dev)
+{
+ return iommu_ops->attach_dev(domain, dev);
+}
+EXPORT_SYMBOL_GPL(iommu_attach_device);
+
+void iommu_detach_device(struct iommu_domain *domain, struct device *dev)
+{
+ iommu_ops->detach_dev(domain, dev);
+}
+EXPORT_SYMBOL_GPL(iommu_detach_device);
+
+int iommu_map_range(struct iommu_domain *domain, unsigned long iova,
+ phys_addr_t paddr, size_t size, int prot)
+{
+ return iommu_ops->map(domain, iova, paddr, size, prot);
+}
+EXPORT_SYMBOL_GPL(iommu_map_range);
+
+void iommu_unmap_range(struct iommu_domain *domain, unsigned long iova,
+ size_t size)
+{
+ iommu_ops->unmap(domain, iova, size);
+}
+EXPORT_SYMBOL_GPL(iommu_unmap_range);
+
+phys_addr_t iommu_iova_to_phys(struct iommu_domain *domain,
+ unsigned long iova)
+{
+ return iommu_ops->iova_to_phys(domain, iova);
+}
+EXPORT_SYMBOL_GPL(iommu_iova_to_phys);
--
1.5.6.4
Impact: add a function to remove all devices from a domain
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 14 ++++++++++++++
1 files changed, 14 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 47158e3..26b929b 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -1510,3 +1510,17 @@ static void detach_device(struct protection_domain *domain, u16 devid)
__detach_device(domain, devid);
write_unlock_irqrestore(&amd_iommu_devtable_lock, flags);
}
+
+static void cleanup_domain(struct protection_domain *domain)
+{
+ unsigned long flags;
+ u16 devid;
+
+ write_lock_irqsave(&amd_iommu_devtable_lock, flags);
+
+ for (devid = 0; devid <= amd_iommu_last_bdf; ++devid)
+ if (amd_iommu_pd_table[devid] == domain)
+ __detach_device(domain, devid);
+
+ write_unlock_irqrestore(&amd_iommu_devtable_lock, flags);
+}
--
1.5.6.4
Impact: add a generic function to lockup addresses in protection domains
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 31 +++++++++++++++++++++++++++++++
1 files changed, 31 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 6ee9377..b42578b 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -1702,3 +1702,34 @@ static void amd_iommu_unmap_range(struct iommu_domain *dom,
iommu_flush_domain(domain->id);
}
+
+static phys_addr_t amd_iommu_iova_to_phys(struct iommu_domain *dom,
+ unsigned long iova)
+{
+ struct protection_domain *domain = dom->priv;
+ unsigned long offset = iova & ~PAGE_MASK;
+ phys_addr_t paddr;
+ u64 *pte;
+
+ pte = &domain->pt_root[IOMMU_PTE_L2_INDEX(iova)];
+
+ if (!IOMMU_PTE_PRESENT(*pte))
+ return 0;
+
+ pte = IOMMU_PTE_PAGE(*pte);
+ pte = &pte[IOMMU_PTE_L1_INDEX(iova)];
+
+ if (!IOMMU_PTE_PRESENT(*pte))
+ return 0;
+
+ pte = IOMMU_PTE_PAGE(*pte);
+ pte = &pte[IOMMU_PTE_L0_INDEX(iova)];
+
+ if (!IOMMU_PTE_PRESENT(*pte))
+ return 0;
+
+ paddr = *pte & IOMMU_PAGE_MASK;
+ paddr |= offset;
+
+ return paddr;
+}
--
1.5.6.4
Impact: know how many devices are assigned to a domain
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/include/asm/amd_iommu_types.h | 1 +
arch/x86/kernel/amd_iommu.c | 3 ++-
2 files changed, 3 insertions(+), 1 deletions(-)
diff --git a/arch/x86/include/asm/amd_iommu_types.h b/arch/x86/include/asm/amd_iommu_types.h
index 4862a5b..1c769f4 100644
--- a/arch/x86/include/asm/amd_iommu_types.h
+++ b/arch/x86/include/asm/amd_iommu_types.h
@@ -203,6 +203,7 @@ struct protection_domain {
int mode; /* paging mode (0-6 levels) */
u64 *pt_root; /* page table root pointer */
unsigned long flags; /* flags to find out type of domain */
+ unsigned dev_cnt; /* devices assigned to this domain */
void *priv; /* private data */
};
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 2173283..61584d5 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -820,9 +820,10 @@ static void set_device_domain(struct amd_iommu *iommu,
u16 devid)
{
unsigned long flags;
-
u64 pte_root = virt_to_phys(domain->pt_root);
+ domain->dev_cnt += 1;
+
pte_root |= (domain->mode & DEV_ENTRY_MODE_MASK)
<< DEV_ENTRY_MODE_SHIFT;
pte_root |= IOMMU_PTE_IR | IOMMU_PTE_IW | IOMMU_PTE_P | IOMMU_PTE_TV;
--
1.5.6.4
Impact: add code to release a domain id
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 10 ++++++++++
1 files changed, 10 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 85a62e9..11de0f1 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -569,6 +569,16 @@ static u16 domain_id_alloc(void)
return id;
}
+static void domain_id_free(int id)
+{
+ unsigned long flags;
+
+ write_lock_irqsave(&amd_iommu_devtable_lock, flags);
+ if (id > 0 && id < MAX_DOMAIN_ID)
+ __clear_bit(id, amd_iommu_pd_alloc_bitmap);
+ write_unlock_irqrestore(&amd_iommu_devtable_lock, flags);
+}
+
/*
* Used to reserve address ranges in the aperture (e.g. for exclusion
* ranges.
--
1.5.6.4
Impact: change code to free pagetables from protection domains
The dma_ops_free_pagetable function can only free pagetables from
dma_ops domains. Change that to free pagetables of pure protection
domains.
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 8 +++++---
1 files changed, 5 insertions(+), 3 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index e96a2f3..85a62e9 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -585,12 +585,12 @@ static void dma_ops_reserve_addresses(struct dma_ops_domain *dom,
iommu_area_reserve(dom->bitmap, start_page, pages);
}
-static void dma_ops_free_pagetable(struct dma_ops_domain *dma_dom)
+static void free_pagetable(struct protection_domain *domain)
{
int i, j;
u64 *p1, *p2, *p3;
- p1 = dma_dom->domain.pt_root;
+ p1 = domain->pt_root;
if (!p1)
return;
@@ -611,6 +611,8 @@ static void dma_ops_free_pagetable(struct dma_ops_domain *dma_dom)
}
free_page((unsigned long)p1);
+
+ domain->pt_root = NULL;
}
/*
@@ -622,7 +624,7 @@ static void dma_ops_domain_free(struct dma_ops_domain *dom)
if (!dom)
return;
- dma_ops_free_pagetable(dom);
+ free_pagetable(&dom->domain);
kfree(dom->pte_pages);
--
1.5.6.4
Imapct: add a new struct member to 'struct protection_domain'
When using protection domains for dma_ops and KVM its better to know for
which subsystem it was allocated. Add a flags member to struct
protection domain for that purpose.
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/include/asm/amd_iommu_types.h | 14 +++++++++-----
arch/x86/kernel/amd_iommu.c | 1 +
2 files changed, 10 insertions(+), 5 deletions(-)
diff --git a/arch/x86/include/asm/amd_iommu_types.h b/arch/x86/include/asm/amd_iommu_types.h
index ac302a2..4862a5b 100644
--- a/arch/x86/include/asm/amd_iommu_types.h
+++ b/arch/x86/include/asm/amd_iommu_types.h
@@ -190,16 +190,20 @@
/* FIXME: move this macro to <linux/pci.h> */
#define PCI_BUS(x) (((x) >> 8) & 0xff)
+/* Protection domain flags */
+#define PD_DMA_OPS_MASK (1UL << 0) /* domain used for dma_ops */
+
/*
* This structure contains generic data for IOMMU protection domains
* independent of their use.
*/
struct protection_domain {
- spinlock_t lock; /* mostly used to lock the page table*/
- u16 id; /* the domain id written to the device table */
- int mode; /* paging mode (0-6 levels) */
- u64 *pt_root; /* page table root pointer */
- void *priv; /* private data */
+ spinlock_t lock; /* mostly used to lock the page table*/
+ u16 id; /* the domain id written to the device table */
+ int mode; /* paging mode (0-6 levels) */
+ u64 *pt_root; /* page table root pointer */
+ unsigned long flags; /* flags to find out type of domain */
+ void *priv; /* private data */
};
/*
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 7355da0..6d26544 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -724,6 +724,7 @@ static struct dma_ops_domain *dma_ops_domain_alloc(struct amd_iommu *iommu,
goto free_dma_dom;
dma_dom->domain.mode = PAGE_MODE_3_LEVEL;
dma_dom->domain.pt_root = (void *)get_zeroed_page(GFP_KERNEL);
+ dma_dom->domain.flags = PD_DMA_OPS_MASK;
dma_dom->domain.priv = dma_dom;
if (!dma_dom->domain.pt_root)
goto free_dma_dom;
--
1.5.6.4
Impact: add a generic function to attach devices to protection domains
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 35 +++++++++++++++++++++++++++++++++++
1 files changed, 35 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 60e268a..97eb60d 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -1600,3 +1600,38 @@ static void amd_iommu_detach_device(struct iommu_domain *dom,
iommu_completion_wait(iommu);
}
+static int amd_iommu_attach_device(struct iommu_domain *dom,
+ struct device *dev)
+{
+ struct protection_domain *domain = dom->priv;
+ struct protection_domain *old_domain;
+ struct amd_iommu *iommu;
+ struct pci_dev *pdev;
+ u16 devid;
+
+ if (dev->bus != &pci_bus_type)
+ return -EINVAL;
+
+ pdev = to_pci_dev(dev);
+
+ devid = calc_devid(pdev->bus->number, pdev->devfn);
+
+ if (devid >= amd_iommu_last_bdf ||
+ devid != amd_iommu_alias_table[devid])
+ return -EINVAL;
+
+ iommu = amd_iommu_rlookup_table[devid];
+ if (!iommu)
+ return -EINVAL;
+
+ old_domain = domain_for_device(devid);
+ if (old_domain)
+ __detach_device(old_domain, devid);
+
+ set_device_domain(iommu, domain, devid);
+
+ iommu_completion_wait(iommu);
+
+ return 0;
+}
+
--
1.5.6.4
Impact: refactoring of iommu_queue_inv_iommu_pages
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 26 ++++++++++++++++----------
1 files changed, 16 insertions(+), 10 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index db89001..80758cc 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -285,6 +285,21 @@ static int iommu_queue_inv_dev_entry(struct amd_iommu *iommu, u16 devid)
return ret;
}
+static void __iommu_build_inv_iommu_pages(struct iommu_cmd *cmd, u64 address,
+ u16 domid, int pde, int s)
+{
+ memset(cmd, 0, sizeof(*cmd));
+ address &= PAGE_MASK;
+ CMD_SET_TYPE(cmd, CMD_INV_IOMMU_PAGES);
+ cmd->data[1] |= domid;
+ cmd->data[2] = lower_32_bits(address);
+ cmd->data[3] = upper_32_bits(address);
+ if (s) /* size bit - we flush more than one 4kb page */
+ cmd->data[2] |= CMD_INV_IOMMU_PAGES_SIZE_MASK;
+ if (pde) /* PDE bit - we wan't flush everything not only the PTEs */
+ cmd->data[2] |= CMD_INV_IOMMU_PAGES_PDE_MASK;
+}
+
/*
* Generic command send function for invalidaing TLB entries
*/
@@ -294,16 +309,7 @@ static int iommu_queue_inv_iommu_pages(struct amd_iommu *iommu,
struct iommu_cmd cmd;
int ret;
- memset(&cmd, 0, sizeof(cmd));
- address &= PAGE_MASK;
- CMD_SET_TYPE(&cmd, CMD_INV_IOMMU_PAGES);
- cmd.data[1] |= domid;
- cmd.data[2] = lower_32_bits(address);
- cmd.data[3] = upper_32_bits(address);
- if (s) /* size bit - we flush more than one 4kb page */
- cmd.data[2] |= CMD_INV_IOMMU_PAGES_SIZE_MASK;
- if (pde) /* PDE bit - we wan't flush everything not only the PTEs */
- cmd.data[2] |= CMD_INV_IOMMU_PAGES_PDE_MASK;
+ __iommu_build_inv_iommu_pages(&cmd, address, domid, pde, s);
ret = iommu_queue_command(iommu, &cmd);
--
1.5.6.4
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 14 ++++++++++++++
1 files changed, 14 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index b42578b..6670aae 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -38,6 +38,8 @@ static DEFINE_RWLOCK(amd_iommu_devtable_lock);
static LIST_HEAD(iommu_pd_list);
static DEFINE_SPINLOCK(iommu_pd_list_lock);
+static struct iommu_ops amd_iommu_ops;
+
/*
* general struct to manage commands send to an IOMMU
*/
@@ -1481,6 +1483,8 @@ int __init amd_iommu_init_dma_ops(void)
/* Make the driver finally visible to the drivers */
dma_ops = &amd_iommu_dma_ops;
+ register_iommu(&amd_iommu_ops);
+
return 0;
free_domains:
@@ -1733,3 +1737,13 @@ static phys_addr_t amd_iommu_iova_to_phys(struct iommu_domain *dom,
return paddr;
}
+
+static struct iommu_ops amd_iommu_ops = {
+ .domain_init = amd_iommu_domain_init,
+ .domain_destroy = amd_iommu_domain_destroy,
+ .attach_dev = amd_iommu_attach_device,
+ .detach_dev = amd_iommu_detach_device,
+ .map = amd_iommu_map_range,
+ .unmap = amd_iommu_unmap_range,
+ .iova_to_phys = amd_iommu_iova_to_phys,
+};
--
1.5.6.4
Impact: function rename
The iommu_map function maps only one page. Make this clear in the
function name.
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 10 +++++-----
1 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 5662e22..4db4542 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -336,10 +336,10 @@ static void iommu_flush_tlb(struct amd_iommu *iommu, u16 domid)
* supporting all features of AMD IOMMU page tables like level skipping
* and full 64 bit address spaces.
*/
-static int iommu_map(struct protection_domain *dom,
- unsigned long bus_addr,
- unsigned long phys_addr,
- int prot)
+static int iommu_map_page(struct protection_domain *dom,
+ unsigned long bus_addr,
+ unsigned long phys_addr,
+ int prot)
{
u64 __pte, *pte, *page;
@@ -438,7 +438,7 @@ static int dma_ops_unity_map(struct dma_ops_domain *dma_dom,
for (addr = e->address_start; addr < e->address_end;
addr += PAGE_SIZE) {
- ret = iommu_map(&dma_dom->domain, addr, addr, e->prot);
+ ret = iommu_map_page(&dma_dom->domain, addr, addr, e->prot);
if (ret)
return ret;
/*
--
1.5.6.4
Impact: add a generic function to detach devices from protection domains
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 27 +++++++++++++++++++++++++++
1 files changed, 27 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index d5581bb..60e268a 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -1573,3 +1573,30 @@ static void amd_iommu_domain_destroy(struct iommu_domain *dom)
dom->priv = NULL;
}
+
+static void amd_iommu_detach_device(struct iommu_domain *dom,
+ struct device *dev)
+{
+ struct protection_domain *domain = dom->priv;
+ struct amd_iommu *iommu;
+ struct pci_dev *pdev;
+ u16 devid;
+
+ if (dev->bus != &pci_bus_type)
+ return;
+
+ pdev = to_pci_dev(dev);
+
+ devid = calc_devid(pdev->bus->number, pdev->devfn);
+
+ if (devid > 0)
+ detach_device(domain, devid);
+
+ iommu = amd_iommu_rlookup_table[devid];
+ if (!iommu)
+ return;
+
+ iommu_queue_inv_dev_entry(iommu, devid);
+ iommu_completion_wait(iommu);
+}
+
--
1.5.6.4
Impact: add a generic function to map pages into protection domains
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 28 ++++++++++++++++++++++++++++
1 files changed, 28 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 97eb60d..511f782 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -1635,3 +1635,31 @@ static int amd_iommu_attach_device(struct iommu_domain *dom,
return 0;
}
+static int amd_iommu_map_range(struct iommu_domain *dom,
+ unsigned long iova, phys_addr_t paddr,
+ size_t size, int iommu_prot)
+{
+ struct protection_domain *domain = dom->priv;
+ unsigned long i, npages = iommu_num_pages(paddr, size, PAGE_SIZE);
+ int prot = 0;
+ int ret;
+
+ if (iommu_prot & IOMMU_READ)
+ prot |= IOMMU_PROT_IR;
+ if (iommu_prot & IOMMU_WRITE)
+ prot |= IOMMU_PROT_IW;
+
+ iova &= PAGE_MASK;
+ paddr &= PAGE_MASK;
+
+ for (i = 0; i < npages; ++i) {
+ ret = iommu_map_page(domain, iova, paddr, prot);
+ if (ret)
+ return ret;
+
+ iova += PAGE_SIZE;
+ paddr += PAGE_SIZE;
+ }
+
+ return 0;
+}
--
1.5.6.4
Impact: add a generic function to unmap pages into protection domains
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 39 +++++++++++++++++++++++++++++++++++++++
1 files changed, 39 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 511f782..6ee9377 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -438,6 +438,28 @@ static int iommu_map_page(struct protection_domain *dom,
return 0;
}
+static void iommu_unmap_page(struct protection_domain *dom,
+ unsigned long bus_addr)
+{
+ u64 *pte;
+
+ pte = &dom->pt_root[IOMMU_PTE_L2_INDEX(bus_addr)];
+
+ if (!IOMMU_PTE_PRESENT(*pte))
+ return;
+
+ pte = IOMMU_PTE_PAGE(*pte);
+ pte = &pte[IOMMU_PTE_L1_INDEX(bus_addr)];
+
+ if (!IOMMU_PTE_PRESENT(*pte))
+ return;
+
+ pte = IOMMU_PTE_PAGE(*pte);
+ pte = &pte[IOMMU_PTE_L1_INDEX(bus_addr)];
+
+ *pte = 0;
+}
+
/*
* This function checks if a specific unity mapping entry is needed for
* this specific IOMMU.
@@ -1663,3 +1685,20 @@ static int amd_iommu_map_range(struct iommu_domain *dom,
return 0;
}
+
+static void amd_iommu_unmap_range(struct iommu_domain *dom,
+ unsigned long iova, size_t size)
+{
+
+ struct protection_domain *domain = dom->priv;
+ unsigned long i, npages = iommu_num_pages(iova, size, PAGE_SIZE);
+
+ iova &= PAGE_MASK;
+
+ for (i = 0; i < npages; ++i) {
+ iommu_unmap_page(domain, iova);
+ iova += PAGE_SIZE;
+ }
+
+ iommu_flush_domain(domain->id);
+}
--
1.5.6.4
Impact: add a generic function for allocation protection domains
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 28 ++++++++++++++++++++++++++++
1 files changed, 28 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 26b929b..cb7db66 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -22,6 +22,7 @@
#include <linux/bitops.h>
#include <linux/scatterlist.h>
#include <linux/iommu-helper.h>
+#include <linux/iommu.h>
#include <asm/proto.h>
#include <asm/iommu.h>
#include <asm/amd_iommu_types.h>
@@ -1524,3 +1525,30 @@ static void cleanup_domain(struct protection_domain *domain)
write_unlock_irqrestore(&amd_iommu_devtable_lock, flags);
}
+
+static int amd_iommu_domain_init(struct iommu_domain *dom)
+{
+ struct protection_domain *domain;
+
+ domain = kzalloc(sizeof(*domain), GFP_KERNEL);
+ if (!domain)
+ return -ENOMEM;
+
+ spin_lock_init(&domain->lock);
+ domain->mode = PAGE_MODE_3_LEVEL;
+ domain->id = domain_id_alloc();
+ if (!domain->id)
+ goto out_free;
+ domain->pt_root = (void *)get_zeroed_page(GFP_KERNEL);
+ if (!domain->pt_root)
+ goto out_free;
+
+ dom->priv = domain;
+
+ return 0;
+
+out_free:
+ kfree(domain);
+
+ return -ENOMEM;
+}
--
1.5.6.4
Impact: add a function to flush a domain id on every IOMMU
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 22 ++++++++++++++++++++++
1 files changed, 22 insertions(+), 0 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 80758cc..7355da0 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -351,6 +351,28 @@ static void iommu_flush_tlb(struct amd_iommu *iommu, u16 domid)
iommu_queue_inv_iommu_pages(iommu, address, domid, 0, 1);
}
+/*
+ * This function is used to flush the IO/TLB for a given protection domain
+ * on every IOMMU in the system
+ */
+static void iommu_flush_domain(u16 domid)
+{
+ unsigned long flags;
+ struct amd_iommu *iommu;
+ struct iommu_cmd cmd;
+
+ __iommu_build_inv_iommu_pages(&cmd, CMD_INV_IOMMU_ALL_PAGES_ADDRESS,
+ domid, 1, 1);
+
+ list_for_each_entry(iommu, &amd_iommu_list, list) {
+ spin_lock_irqsave(&iommu->lock, flags);
+ __iommu_queue_command(iommu, &cmd);
+ __iommu_completion_wait(iommu);
+ __iommu_wait_for_completion(iommu);
+ spin_unlock_irqrestore(&iommu->lock, flags);
+ }
+}
+
/****************************************************************************
*
* The functions below are used the create the page table mappings for
--
1.5.6.4
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/ia64/include/asm/kvm_host.h | 2 +-
arch/ia64/kvm/Makefile | 2 +-
arch/ia64/kvm/kvm-ia64.c | 3 +-
arch/x86/include/asm/kvm_host.h | 2 +-
arch/x86/kvm/Makefile | 2 +-
arch/x86/kvm/x86.c | 3 +-
include/linux/kvm_host.h | 6 ++--
virt/kvm/iommu.c | 45 +++++++++++++++++--------------------
virt/kvm/kvm_main.c | 2 +-
9 files changed, 33 insertions(+), 34 deletions(-)
diff --git a/arch/ia64/include/asm/kvm_host.h b/arch/ia64/include/asm/kvm_host.h
index 0560f3f..3486636 100644
--- a/arch/ia64/include/asm/kvm_host.h
+++ b/arch/ia64/include/asm/kvm_host.h
@@ -467,7 +467,7 @@ struct kvm_arch {
struct kvm_sal_data rdv_sal_data;
struct list_head assigned_dev_head;
- struct dmar_domain *intel_iommu_domain;
+ struct iommu_domain *iommu_domain;
struct hlist_head irq_ack_notifier_list;
unsigned long irq_sources_bitmap;
diff --git a/arch/ia64/kvm/Makefile b/arch/ia64/kvm/Makefile
index cb69dfc..0bb99b7 100644
--- a/arch/ia64/kvm/Makefile
+++ b/arch/ia64/kvm/Makefile
@@ -51,7 +51,7 @@ EXTRA_AFLAGS += -Ivirt/kvm -Iarch/ia64/kvm/
common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o ioapic.o \
coalesced_mmio.o irq_comm.o)
-ifeq ($(CONFIG_DMAR),y)
+ifeq ($(CONFIG_IOMMU_API),y)
common-objs += $(addprefix ../../../virt/kvm/, iommu.o)
endif
diff --git a/arch/ia64/kvm/kvm-ia64.c b/arch/ia64/kvm/kvm-ia64.c
index b4d24e2..483a15b 100644
--- a/arch/ia64/kvm/kvm-ia64.c
+++ b/arch/ia64/kvm/kvm-ia64.c
@@ -31,6 +31,7 @@
#include <linux/bitops.h>
#include <linux/hrtimer.h>
#include <linux/uaccess.h>
+#include <linux/iommu.h>
#include <linux/intel-iommu.h>
#include <asm/pgtable.h>
@@ -189,7 +190,7 @@ int kvm_dev_ioctl_check_extension(long ext)
r = KVM_COALESCED_MMIO_PAGE_OFFSET;
break;
case KVM_CAP_IOMMU:
- r = intel_iommu_found();
+ r = iommu_found();
break;
default:
r = 0;
diff --git a/arch/x86/include/asm/kvm_host.h b/arch/x86/include/asm/kvm_host.h
index 97215a4..730843d 100644
--- a/arch/x86/include/asm/kvm_host.h
+++ b/arch/x86/include/asm/kvm_host.h
@@ -360,7 +360,7 @@ struct kvm_arch{
struct list_head active_mmu_pages;
struct list_head assigned_dev_head;
struct list_head oos_global_pages;
- struct dmar_domain *intel_iommu_domain;
+ struct iommu_domain *iommu_domain;
struct kvm_pic *vpic;
struct kvm_ioapic *vioapic;
struct kvm_pit *vpit;
diff --git a/arch/x86/kvm/Makefile b/arch/x86/kvm/Makefile
index 00f46c2..d3ec292 100644
--- a/arch/x86/kvm/Makefile
+++ b/arch/x86/kvm/Makefile
@@ -7,7 +7,7 @@ common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o ioapic.o \
ifeq ($(CONFIG_KVM_TRACE),y)
common-objs += $(addprefix ../../../virt/kvm/, kvm_trace.o)
endif
-ifeq ($(CONFIG_DMAR),y)
+ifeq ($(CONFIG_IOMMU_API),y)
common-objs += $(addprefix ../../../virt/kvm/, iommu.o)
endif
diff --git a/arch/x86/kvm/x86.c b/arch/x86/kvm/x86.c
index ba10287..074a1ee 100644
--- a/arch/x86/kvm/x86.c
+++ b/arch/x86/kvm/x86.c
@@ -34,6 +34,7 @@
#include <linux/module.h>
#include <linux/mman.h>
#include <linux/highmem.h>
+#include <linux/iommu.h>
#include <linux/intel-iommu.h>
#include <asm/uaccess.h>
@@ -990,7 +991,7 @@ int kvm_dev_ioctl_check_extension(long ext)
r = !tdp_enabled;
break;
case KVM_CAP_IOMMU:
- r = intel_iommu_found();
+ r = iommu_found();
break;
default:
r = 0;
diff --git a/include/linux/kvm_host.h b/include/linux/kvm_host.h
index 673fabd..c428f2a 100644
--- a/include/linux/kvm_host.h
+++ b/include/linux/kvm_host.h
@@ -327,7 +327,7 @@ void kvm_unregister_irq_ack_notifier(struct kvm_irq_ack_notifier *kian);
int kvm_request_irq_source_id(struct kvm *kvm);
void kvm_free_irq_source_id(struct kvm *kvm, int irq_source_id);
-#ifdef CONFIG_DMAR
+#ifdef CONFIG_IOMMU_API
int kvm_iommu_map_pages(struct kvm *kvm, gfn_t base_gfn,
unsigned long npages);
int kvm_iommu_map_guest(struct kvm *kvm);
@@ -336,7 +336,7 @@ int kvm_assign_device(struct kvm *kvm,
struct kvm_assigned_dev_kernel *assigned_dev);
int kvm_deassign_device(struct kvm *kvm,
struct kvm_assigned_dev_kernel *assigned_dev);
-#else /* CONFIG_DMAR */
+#else /* CONFIG_IOMMU_API */
static inline int kvm_iommu_map_pages(struct kvm *kvm,
gfn_t base_gfn,
unsigned long npages)
@@ -365,7 +365,7 @@ static inline int kvm_deassign_device(struct kvm *kvm,
{
return 0;
}
-#endif /* CONFIG_DMAR */
+#endif /* CONFIG_IOMMU_API */
static inline void kvm_guest_enter(void)
{
diff --git a/virt/kvm/iommu.c b/virt/kvm/iommu.c
index d46de9a..d0bebaa 100644
--- a/virt/kvm/iommu.c
+++ b/virt/kvm/iommu.c
@@ -25,6 +25,7 @@
#include <linux/kvm_host.h>
#include <linux/pci.h>
#include <linux/dmar.h>
+#include <linux/iommu.h>
#include <linux/intel-iommu.h>
static int kvm_iommu_unmap_memslots(struct kvm *kvm);
@@ -37,7 +38,7 @@ int kvm_iommu_map_pages(struct kvm *kvm,
gfn_t gfn = base_gfn;
pfn_t pfn;
int i, r = 0;
- struct dmar_domain *domain = kvm->arch.intel_iommu_domain;
+ struct iommu_domain *domain = kvm->arch.iommu_domain;
/* check if iommu exists and in use */
if (!domain)
@@ -45,16 +46,15 @@ int kvm_iommu_map_pages(struct kvm *kvm,
for (i = 0; i < npages; i++) {
/* check if already mapped */
- if (intel_iommu_iova_to_phys(domain,
- gfn_to_gpa(gfn)))
+ if (iommu_iova_to_phys(domain, gfn_to_gpa(gfn)))
continue;
pfn = gfn_to_pfn(kvm, gfn);
- r = intel_iommu_map_address(domain,
- gfn_to_gpa(gfn),
- pfn_to_hpa(pfn),
- PAGE_SIZE,
- DMA_PTE_READ | DMA_PTE_WRITE);
+ r = iommu_map_range(domain,
+ gfn_to_gpa(gfn),
+ pfn_to_hpa(pfn),
+ PAGE_SIZE,
+ IOMMU_READ | IOMMU_WRITE);
if (r) {
printk(KERN_ERR "kvm_iommu_map_address:"
"iommu failed to map pfn=%lx\n", pfn);
@@ -88,7 +88,7 @@ int kvm_assign_device(struct kvm *kvm,
struct kvm_assigned_dev_kernel *assigned_dev)
{
struct pci_dev *pdev = NULL;
- struct dmar_domain *domain = kvm->arch.intel_iommu_domain;
+ struct iommu_domain *domain = kvm->arch.iommu_domain;
int r;
/* check if iommu exists and in use */
@@ -99,7 +99,7 @@ int kvm_assign_device(struct kvm *kvm,
if (pdev == NULL)
return -ENODEV;
- r = intel_iommu_attach_device(domain, pdev);
+ r = iommu_attach_device(domain, &pdev->dev);
if (r) {
printk(KERN_ERR "assign device %x:%x.%x failed",
pdev->bus->number,
@@ -119,7 +119,7 @@ int kvm_assign_device(struct kvm *kvm,
int kvm_deassign_device(struct kvm *kvm,
struct kvm_assigned_dev_kernel *assigned_dev)
{
- struct dmar_domain *domain = kvm->arch.intel_iommu_domain;
+ struct iommu_domain *domain = kvm->arch.iommu_domain;
struct pci_dev *pdev = NULL;
/* check if iommu exists and in use */
@@ -130,7 +130,7 @@ int kvm_deassign_device(struct kvm *kvm,
if (pdev == NULL)
return -ENODEV;
- intel_iommu_detach_device(domain, pdev);
+ iommu_detach_device(domain, &pdev->dev);
printk(KERN_DEBUG "deassign device: host bdf = %x:%x:%x\n",
assigned_dev->host_busnr,
@@ -144,13 +144,13 @@ int kvm_iommu_map_guest(struct kvm *kvm)
{
int r;
- if (!intel_iommu_found()) {
- printk(KERN_ERR "%s: intel iommu not found\n", __func__);
+ if (!iommu_found()) {
+ printk(KERN_ERR "%s: iommu not found\n", __func__);
return -ENODEV;
}
- kvm->arch.intel_iommu_domain = intel_iommu_alloc_domain();
- if (!kvm->arch.intel_iommu_domain)
+ kvm->arch.iommu_domain = iommu_domain_alloc();
+ if (!kvm->arch.iommu_domain)
return -ENOMEM;
r = kvm_iommu_map_memslots(kvm);
@@ -169,7 +169,7 @@ static void kvm_iommu_put_pages(struct kvm *kvm,
{
gfn_t gfn = base_gfn;
pfn_t pfn;
- struct dmar_domain *domain = kvm->arch.intel_iommu_domain;
+ struct iommu_domain *domain = kvm->arch.iommu_domain;
unsigned long i;
u64 phys;
@@ -178,16 +178,13 @@ static void kvm_iommu_put_pages(struct kvm *kvm,
return;
for (i = 0; i < npages; i++) {
- phys = intel_iommu_iova_to_phys(domain,
- gfn_to_gpa(gfn));
+ phys = iommu_iova_to_phys(domain, gfn_to_gpa(gfn));
pfn = phys >> PAGE_SHIFT;
kvm_release_pfn_clean(pfn);
gfn++;
}
- intel_iommu_unmap_address(domain,
- gfn_to_gpa(base_gfn),
- PAGE_SIZE * npages);
+ iommu_unmap_range(domain, gfn_to_gpa(base_gfn), PAGE_SIZE * npages);
}
static int kvm_iommu_unmap_memslots(struct kvm *kvm)
@@ -205,13 +202,13 @@ static int kvm_iommu_unmap_memslots(struct kvm *kvm)
int kvm_iommu_unmap_guest(struct kvm *kvm)
{
- struct dmar_domain *domain = kvm->arch.intel_iommu_domain;
+ struct iommu_domain *domain = kvm->arch.iommu_domain;
/* check if iommu exists and in use */
if (!domain)
return 0;
kvm_iommu_unmap_memslots(kvm);
- intel_iommu_free_domain(domain);
+ iommu_domain_free(domain);
return 0;
}
diff --git a/virt/kvm/kvm_main.c b/virt/kvm/kvm_main.c
index 4973ded..93a54fc 100644
--- a/virt/kvm/kvm_main.c
+++ b/virt/kvm/kvm_main.c
@@ -494,7 +494,7 @@ static int kvm_vm_ioctl_assign_device(struct kvm *kvm,
list_add(&match->list, &kvm->arch.assigned_dev_head);
if (assigned_dev->flags & KVM_DEV_ASSIGN_ENABLE_IOMMU) {
- if (!kvm->arch.intel_iommu_domain) {
+ if (!kvm->arch.iommu_domain) {
r = kvm_iommu_map_guest(kvm);
if (r)
goto out_list_del;
--
1.5.6.4
Signed-off-by: Joerg Roedel <[email protected]>
---
drivers/pci/intel-iommu.c | 15 +++++++++++++++
1 files changed, 15 insertions(+), 0 deletions(-)
diff --git a/drivers/pci/intel-iommu.c b/drivers/pci/intel-iommu.c
index 7128105..81e04ec 100644
--- a/drivers/pci/intel-iommu.c
+++ b/drivers/pci/intel-iommu.c
@@ -277,6 +277,8 @@ static int intel_iommu_strict;
static DEFINE_SPINLOCK(device_domain_lock);
static LIST_HEAD(device_domain_list);
+static struct iommu_ops intel_iommu_ops;
+
static int __init intel_iommu_setup(char *str)
{
if (!str)
@@ -2729,6 +2731,9 @@ int __init intel_iommu_init(void)
init_timer(&unmap_timer);
force_iommu = 1;
dma_ops = &intel_dma_ops;
+
+ register_iommu(&intel_iommu_ops);
+
return 0;
}
@@ -3119,3 +3124,13 @@ static phys_addr_t intel_iommu_iova_to_phys(struct iommu_domain *domain,
return phys;
}
+
+static struct iommu_ops intel_iommu_ops = {
+ .domain_init = intel_iommu_domain_init,
+ .domain_destroy = intel_iommu_domain_destroy,
+ .attach_dev = intel_iommu_attach_device,
+ .detach_dev = intel_iommu_detach_device,
+ .map = intel_iommu_map_range,
+ .unmap = intel_iommu_unmap_range,
+ .iova_to_phys = intel_iommu_iova_to_phys,
+};
--
1.5.6.4
Impact: bugfix in iommu_map_page function
Signed-off-by: Joerg Roedel <[email protected]>
---
arch/x86/kernel/amd_iommu.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/arch/x86/kernel/amd_iommu.c b/arch/x86/kernel/amd_iommu.c
index 4db4542..153db6e 100644
--- a/arch/x86/kernel/amd_iommu.c
+++ b/arch/x86/kernel/amd_iommu.c
@@ -344,7 +344,7 @@ static int iommu_map_page(struct protection_domain *dom,
u64 __pte, *pte, *page;
bus_addr = PAGE_ALIGN(bus_addr);
- phys_addr = PAGE_ALIGN(bus_addr);
+ phys_addr = PAGE_ALIGN(phys_addr);
/* only support 512GB address spaces for now */
if (bus_addr > IOMMU_MAP_SIZE_L3 || !(prot & IOMMU_PROT_MASK))
--
1.5.6.4
Joerg Roedel wrote:
> Hi,
>
> the two patchsets posted as reply to this email implement KVM device
> passthrough support for AMD IOMMU hardware. The changes to the previous
> posts are descibed below
>
> The first patchset is version 4 of the generic iommu api patchset which
> generalizes the VT-d functions exported to KVM into a common api where
> the AMD IOMMU code can plug into. In this version the patchset was
> rebased to the latest post of Han Weidong's patches.
>
> The second patchset finally implements the KVM device passthrough
> support in the AMD IOMMU code. Together with KVM-79 I successfully
> passed an 10GBit network card into an KVM guest. In this version the
> patchset was changed to remove any device before a protection domain is
> freed instead of printing a BUG. Also the patchset was rebased to the
> updated IOMMU-API patches.
>
> These two patchsets apply in order in top of the latest post of
> Han Weidong's Multiple device assignement support patches. Anybody who
> wants to try this out can pull the whole stuff from
>
> git://git.kernel.org/pub/scm/linux/kernel/git/joro/linux-2.6-iommu.git kvm-amd-iommu
>
> Please give these patches a good review.
>
>
Ack for the kvm bits.
--
error compiling committee.c: too many arguments to function
On Wed, Dec 10, 2008 at 11:36:16AM +0200, Avi Kivity wrote:
> Joerg Roedel wrote:
> >Hi,
> >
> >the two patchsets posted as reply to this email implement KVM device
> >passthrough support for AMD IOMMU hardware. The changes to the previous
> >posts are descibed below
> >
> >The first patchset is version 4 of the generic iommu api patchset which
> >generalizes the VT-d functions exported to KVM into a common api where
> >the AMD IOMMU code can plug into. In this version the patchset was
> >rebased to the latest post of Han Weidong's patches.
> >
> >The second patchset finally implements the KVM device passthrough
> >support in the AMD IOMMU code. Together with KVM-79 I successfully
> >passed an 10GBit network card into an KVM guest. In this version the
> >patchset was changed to remove any device before a protection domain is
> >freed instead of printing a BUG. Also the patchset was rebased to the
> >updated IOMMU-API patches.
> >
> >These two patchsets apply in order in top of the latest post of
> >Han Weidong's Multiple device assignement support patches. Anybody who
> >wants to try this out can pull the whole stuff from
> >
> >git://git.kernel.org/pub/scm/linux/kernel/git/joro/linux-2.6-iommu.git kvm-amd-iommu
> >
> >Please give these patches a good review.
> >
> >
>
> Ack for the kvm bits.
Thanks Avi.
So now its open how this will be merged alltogether. We can merge it in
three steps (first from Dave's tree, then Avi and at last my IOMMU
updates which has to happen in that order).
The other option is to merge this all with one pull-request I send to
Linus. It should not conflict with your updates Avi because all these
changes apply also cleanly to current linus/master.
Is this acceptable for everyone?
David, its important to hear your opinion on that because this single
pull-request would include all VT-d updates you have queued in your
tree? Is this ok for you?
Joerg
--
| AMD Saxony Limited Liability Company & Co. KG
Operating | Wilschdorfer Landstr. 101, 01109 Dresden, Germany
System | Register Court Dresden: HRA 4896
Research | General Partner authorized to represent:
Center | AMD Saxony LLC (Wilmington, Delaware, US)
| General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy
On Wed, 2008-12-10 at 15:11 +0100, Joerg Roedel wrote:
> So now its open how this will be merged alltogether. We can merge it in
> three steps (first from Dave's tree, then Avi and at last my IOMMU
> updates which has to happen in that order).
> The other option is to merge this all with one pull-request I send to
> Linus. It should not conflict with your updates Avi because all these
> changes apply also cleanly to current linus/master.
> Is this acceptable for everyone?
>
> David, its important to hear your opinion on that because this single
> pull-request would include all VT-d updates you have queued in your
> tree? Is this ok for you?
Yes, that's fine by me. Should I apply patches 1-14 of Weidong's patch
set from Monday first? Or just let you do that?
--
dwmw2
Joerg Roedel wrote:
> So now its open how this will be merged alltogether. We can merge it in
> three steps (first from Dave's tree, then Avi and at last my IOMMU
> updates which has to happen in that order).
> The other option is to merge this all with one pull-request I send to
> Linus. It should not conflict with your updates Avi because all these
> changes apply also cleanly to current linus/master.
> Is this acceptable for everyone?
>
Okay for me.
--
error compiling committee.c: too many arguments to function
On Wed, Dec 10, 2008 at 02:22:08PM +0000, David Woodhouse wrote:
> On Wed, 2008-12-10 at 15:11 +0100, Joerg Roedel wrote:
> > So now its open how this will be merged alltogether. We can merge it in
> > three steps (first from Dave's tree, then Avi and at last my IOMMU
> > updates which has to happen in that order).
> > The other option is to merge this all with one pull-request I send to
> > Linus. It should not conflict with your updates Avi because all these
> > changes apply also cleanly to current linus/master.
> > Is this acceptable for everyone?
> >
> > David, its important to hear your opinion on that because this single
> > pull-request would include all VT-d updates you have queued in your
> > tree? Is this ok for you?
>
> Yes, that's fine by me. Should I apply patches 1-14 of Weidong's patch
> set from Monday first? Or just let you do that?
I already did it on-top of your tree because Han Weidong's patches 1-17
were rebased to your tree and my IOMMU-API patches apply on-top of his
patches.
Joerg
--
| AMD Saxony Limited Liability Company & Co. KG
Operating | Wilschdorfer Landstr. 101, 01109 Dresden, Germany
System | Register Court Dresden: HRA 4896
Research | General Partner authorized to represent:
Center | AMD Saxony LLC (Wilmington, Delaware, US)
| General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy
On Wed, 2008-12-10 at 15:25 +0100, Joerg Roedel wrote:
> I already did it on-top of your tree because Han Weidong's patches 1-17
> were rebased to your tree and my IOMMU-API patches apply on-top of his
> patches.
OK, thanks. Looks like you still need this though, as Mike Day pointed
out:
diff --git a/drivers/pci/intel-iommu.c b/drivers/pci/intel-iommu.c
index 97df47c..a5f6e28 100644
--- a/drivers/pci/intel-iommu.c
+++ b/drivers/pci/intel-iommu.c
@@ -204,7 +204,7 @@ static inline bool dma_pte_present(struct dma_pte *pte)
}
/* devices under the same p2p bridge are owned in one domain */
-#define DOMAIN_FLAG_P2P_MULTIPLE_DEVICES (1 < 0)
+#define DOMAIN_FLAG_P2P_MULTIPLE_DEVICES (1 << 0)
/* domain represents a virtual machine, more than one devices
* across iommus may be owned in one domain, e.g. kvm guest.
--
David Woodhouse Open Source Technology Centre
[email protected] Intel Corporation
On Wed, Dec 10, 2008 at 06:24:35PM +0000, David Woodhouse wrote:
> On Wed, 2008-12-10 at 15:25 +0100, Joerg Roedel wrote:
> > I already did it on-top of your tree because Han Weidong's patches 1-17
> > were rebased to your tree and my IOMMU-API patches apply on-top of his
> > patches.
>
> OK, thanks. Looks like you still need this though, as Mike Day pointed
> out:
>
> diff --git a/drivers/pci/intel-iommu.c b/drivers/pci/intel-iommu.c
> index 97df47c..a5f6e28 100644
> --- a/drivers/pci/intel-iommu.c
> +++ b/drivers/pci/intel-iommu.c
> @@ -204,7 +204,7 @@ static inline bool dma_pte_present(struct dma_pte *pte)
> }
>
> /* devices under the same p2p bridge are owned in one domain */
> -#define DOMAIN_FLAG_P2P_MULTIPLE_DEVICES (1 < 0)
> +#define DOMAIN_FLAG_P2P_MULTIPLE_DEVICES (1 << 0)
>
> /* domain represents a virtual machine, more than one devices
> * across iommus may be owned in one domain, e.g. kvm guest.
Ok, I add it, thanks. Who is the author, Mike or you?
--
| AMD Saxony Limited Liability Company & Co. KG
Operating | Wilschdorfer Landstr. 101, 01109 Dresden, Germany
System | Register Court Dresden: HRA 4896
Research | General Partner authorized to represent:
Center | AMD Saxony LLC (Wilmington, Delaware, US)
| General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy
On Wed, 2008-12-10 at 19:42 +0100, Joerg Roedel wrote:
> Ok, I add it, thanks. Who is the author, Mike or you?
Might as well attribute it to Mike; he spotted it.
--
dwmw2
Joerg Roedel wrote:
> On Wed, Dec 10, 2008 at 02:22:08PM +0000, David Woodhouse wrote:
>> On Wed, 2008-12-10 at 15:11 +0100, Joerg Roedel wrote:
>>> So now its open how this will be merged alltogether. We can merge
>>> it in three steps (first from Dave's tree, then Avi and at last my
>>> IOMMU updates which has to happen in that order).
>>> The other option is to merge this all with one pull-request I send
>>> to Linus. It should not conflict with your updates Avi because all
>>> these changes apply also cleanly to current linus/master.
>>> Is this acceptable for everyone?
>>>
>>> David, its important to hear your opinion on that because this
>>> single pull-request would include all VT-d updates you have queued
>>> in your tree? Is this ok for you?
>>
>> Yes, that's fine by me. Should I apply patches 1-14 of Weidong's
>> patch set from Monday first? Or just let you do that?
>
> I already did it on-top of your tree because Han Weidong's patches
> 1-17 were rebased to your tree and my IOMMU-API patches apply on-top
> of his patches.
>
> Joerg
It's good. Thanks Dave and Joerg.
Regards,
Weidong-
On Wed, Dec 10, 2008 at 06:48:23PM +0000, David Woodhouse wrote:
> On Wed, 2008-12-10 at 19:42 +0100, Joerg Roedel wrote:
> > Ok, I add it, thanks. Who is the author, Mike or you?
>
> Might as well attribute it to Mike; he spotted it.
Ok, here is what I applied, fyi.
--
| AMD Saxony Limited Liability Company & Co. KG
Operating | Wilschdorfer Landstr. 101, 01109 Dresden, Germany
System | Register Court Dresden: HRA 4896
Research | General Partner authorized to represent:
Center | AMD Saxony LLC (Wilmington, Delaware, US)
| General Manager of AMD Saxony LLC: Dr. Hans-R. Deppe, Thomas McCoy