2015-11-20 02:29:00

by Chen Feng

[permalink] [raw]
Subject: [PATCH V5 RESEND 0/3] Add iommu support for hi6220 HiKey board

The patch sets add iommu support for Hi6220 SoC. Current testing and support
board is Hikey which is one of 96boards.
It is an arm64 open source board. For more information about this board,
please access https://www.96boards.org.

The Architecture of SMMU on Hi6220 SoC:

+------------------------------------------------------------------+
| |
| +---------+ +--------+ +-------------+ +-------+ |
| | ADE | | ISP | | V/J codec | | G3D | |
| +----|----+ +---|----+ +------|------+ +---|---| |
| | | | | |
| ---------v-----------v--------------v--------------v----- |
| Media Bus |
| --------------------------------|---------------|-------- |
| | | |
| +---v---------------v--------+ |
| | SMMU | |
| +----------|---------|-------+ |
| | | |
+--------------------------------------------|---------|-----------+
| |
+------------v---------v-----------+
| DDRC |
+----------------------------------+
Note:
The media system share the same smmu IP to access DDR memory. And all
media IP use the same page table. The hi6220 iommu driver also uses the
iova api to manage an iova allocator to ensure that the caller get different
iova address.
The caller can use the follow sample code to map phy and iova address.

eg:
struct iommu_domain *domain = iommu_domain_alloc(bus);
iommu_attach_device(domain, dev);
struct iova_domain *iovad = (struct iova_domain *)m_dev->archdata.iommu;
struct iova * t_iova = alloc_iova(iovad, size, limit_pfn, align);
iommu_map(domain, t_iova->pfn_lo << 12, phy_addr, size, port);

The patch sets are based on 4.4-RC1

V2: Fix tlb flush when unmap
V3: Fix format issue and iova address range
V5: Add cover-letter and resend to dt maillist

Chen Feng (3):
docs: iommu: Documentation for iommu in hi6220 SoC
iommu/hisilicon: Add hi6220-SoC smmu driver
arm64: dts: Add dts node for hi6220 smmu driver

.../bindings/iommu/hisi,hi6220-iommu.txt | 32 ++
arch/arm64/boot/dts/hisilicon/hi6220.dtsi | 13 +
drivers/iommu/Kconfig | 11 +
drivers/iommu/Makefile | 1 +
drivers/iommu/hi6220_iommu.c | 492 +++++++++++++++++++++
5 files changed, 549 insertions(+)
create mode 100644 Documentation/devicetree/bindings/iommu/hisi,hi6220-iommu.txt
create mode 100644 drivers/iommu/hi6220_iommu.c

--
1.9.1


2015-11-20 02:28:46

by Chen Feng

[permalink] [raw]
Subject: [PATCH V5 RESEND 1/3] docs: iommu: Documentation for iommu in hi6220 SoC

Documentation for hi6220 iommu driver.

Signed-off-by: Chen Feng <[email protected]>
---
.../bindings/iommu/hisi,hi6220-iommu.txt | 32 ++++++++++++++++++++++
1 file changed, 32 insertions(+)
create mode 100644 Documentation/devicetree/bindings/iommu/hisi,hi6220-iommu.txt

diff --git a/Documentation/devicetree/bindings/iommu/hisi,hi6220-iommu.txt b/Documentation/devicetree/bindings/iommu/hisi,hi6220-iommu.txt
new file mode 100644
index 0000000..44f9101
--- /dev/null
+++ b/Documentation/devicetree/bindings/iommu/hisi,hi6220-iommu.txt
@@ -0,0 +1,32 @@
+Hi6220 SoC SMMU Device Driver devicetree document
+The media system shared the same smmu IP to access DDR memory. And all
+media IP used the same page table.
+
+Below binding describes the system mmu for media system in hi6220 platform
+
+Required properties:
+- compatible: should contain "hisilicon,hi6220-smmu".
+- reg: A tuple of base address and size of System MMU registers.
+- clocks: a list of phandle + clock-specifier pairs, one for each entry
+ in clock-names.
+- clock-names: should contain:
+ * "smmu"
+ * "media-sc"
+ * "smmu-peri"
+- interrupts: An interrupt specifier for interrupt signal of System MMU.
+- #iommu-cells: The iommu-cells should be 0. Because no additional information
+ needs to be encoded in the specifier.
+
+Examples:
+ iommu@f4210000 {
+ compatible = "hisilicon,hi6220-smmu";
+ reg = <0x0 0xf4210000 0x0 0x1000>;
+ interrupts = <GIC_SPI 113 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&sys_ctrl HI6220_MMU_CLK>,
+ <&media_ctrl HI6220_MED_MMU>,
+ <&sys_ctrl HI6220_MEDIA_PLL_SRC>;
+ clock-names = "smmu",
+ "media-sc",
+ "smmu-peri";
+ #iommu-cells = <0>;
+ };
--
1.9.1

2015-11-20 02:28:58

by Chen Feng

[permalink] [raw]
Subject: [PATCH V5 RESEND 2/3] iommu/hisilicon: Add hi6220-SoC smmu driver

Add iommu driver for hi6220 SoC platform.The smmu on hi6220
SoC is for media system.And the media IP use the same page-table.
It supports only one-to-one mapping from iova to phys address.

Signed-off-by: Chen Feng <[email protected]>
---
drivers/iommu/Kconfig | 11 +
drivers/iommu/Makefile | 1 +
drivers/iommu/hi6220_iommu.c | 492 +++++++++++++++++++++++++++++++++++++++++++
3 files changed, 504 insertions(+)
create mode 100644 drivers/iommu/hi6220_iommu.c

diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
index b9094e9..7c13521 100644
--- a/drivers/iommu/Kconfig
+++ b/drivers/iommu/Kconfig
@@ -84,6 +84,17 @@ config IOMMU_PGTABLES_L2
def_bool y
depends on MSM_IOMMU && MMU && SMP && CPU_DCACHE_DISABLE=n

+config HI6220_IOMMU
+ bool "Hi6220 IOMMU Support"
+ depends on ARM64
+ select IOMMU_API
+ select IOMMU_IOVA
+ help
+ Enable IOMMU Driver for hi6220 SoC. The IOMMU API and IOMMU IOVA
+ is also selected.
+
+ If unsure, say N.
+
# AMD IOMMU support
config AMD_IOMMU
bool "AMD IOMMU support"
diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile
index 68faca02..cdb2b83 100644
--- a/drivers/iommu/Makefile
+++ b/drivers/iommu/Makefile
@@ -12,6 +12,7 @@ obj-$(CONFIG_AMD_IOMMU_V2) += amd_iommu_v2.o
obj-$(CONFIG_ARM_SMMU) += arm-smmu.o
obj-$(CONFIG_ARM_SMMU_V3) += arm-smmu-v3.o
obj-$(CONFIG_DMAR_TABLE) += dmar.o
+obj-$(CONFIG_HI6220_IOMMU) += hi6220_iommu.o
obj-$(CONFIG_INTEL_IOMMU) += intel-iommu.o
obj-$(CONFIG_INTEL_IOMMU_SVM) += intel-svm.o
obj-$(CONFIG_IPMMU_VMSA) += ipmmu-vmsa.o
diff --git a/drivers/iommu/hi6220_iommu.c b/drivers/iommu/hi6220_iommu.c
new file mode 100644
index 0000000..fd74f29
--- /dev/null
+++ b/drivers/iommu/hi6220_iommu.c
@@ -0,0 +1,492 @@
+/*
+ * Hisilicon Hi6220 IOMMU driver
+ *
+ * Copyright (c) 2015 Hisilicon Limited.
+ *
+ * Author: Chen Feng <[email protected]>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#define pr_fmt(fmt) "IOMMU: " fmt
+
+#include <linux/clk.h>
+#include <linux/dma-mapping.h>
+#include <linux/err.h>
+#include <linux/io.h>
+#include <linux/iommu.h>
+#include <linux/irq.h>
+#include <linux/iova.h>
+#include <linux/iopoll.h>
+#include <linux/interrupt.h>
+#include <linux/list.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
+#include <linux/sizes.h>
+
+#define SMMU_CTRL_OFFSET (0x0000)
+#define SMMU_ENABLE_OFFSET (0x0004)
+#define SMMU_PTBR_OFFSET (0x0008)
+#define SMMU_START_OFFSET (0x000C)
+#define SMMU_END_OFFSET (0x0010)
+#define SMMU_INTMASK_OFFSET (0x0014)
+#define SMMU_RINTSTS_OFFSET (0x0018)
+#define SMMU_MINTSTS_OFFSET (0x001C)
+#define SMMU_INTCLR_OFFSET (0x0020)
+#define SMMU_STATUS_OFFSET (0x0024)
+#define SMMU_AXIID_OFFSET (0x0028)
+#define SMMU_CNTCTRL_OFFSET (0x002C)
+#define SMMU_TRANSCNT_OFFSET (0x0030)
+#define SMMU_L0TLBHITCNT_OFFSET (0x0034)
+#define SMMU_L1TLBHITCNT_OFFSET (0x0038)
+#define SMMU_WRAPCNT_OFFSET (0x003C)
+#define SMMU_SEC_START_OFFSET (0x0040)
+#define SMMU_SEC_END_OFFSET (0x0044)
+#define SMMU_VERSION_OFFSET (0x0048)
+#define SMMU_IPTSRC_OFFSET (0x004C)
+#define SMMU_IPTPA_OFFSET (0x0050)
+#define SMMU_TRBA_OFFSET (0x0054)
+#define SMMU_BYS_START_OFFSET (0x0058)
+#define SMMU_BYS_END_OFFSET (0x005C)
+#define SMMU_RAM_OFFSET (0x1000)
+
+#define SMMU_CTRL_INVALID (BIT(10))
+#define SMMU_SR_REGS_NUM (15)
+#define SMMU_REGS_SGMT_END (0x60)
+#define PAGE_ENTRY_VALID (0x1)
+#define IOPAGE_SHIFT (12)
+#define IOVA_PFN(addr) ((addr) >> IOPAGE_SHIFT)
+#define IOVA_PAGE_SZ (SZ_4K)
+
+/**
+ * The iova address from 0 ~ 2G
+ */
+#define IOVA_START (0x0)
+#define IOVA_END (0x80000000)
+
+struct hi6220_smmu {
+ unsigned int irq;
+ void __iomem *reg_base;
+ struct clk *smmu_peri_clk;
+ struct clk *smmu_clk;
+ struct clk *media_sc_clk;
+ spinlock_t spinlock; /*spinlock for tlb invalid*/
+ dma_addr_t pgtable_phy;
+ void *pgtable_virt;
+ u32 pgtable_size;
+ u32 *sr_data;
+};
+
+struct hi6220_domain {
+ struct hi6220_smmu *smmu_dev;
+ struct iommu_domain io_domain;
+};
+
+static struct hi6220_smmu *smmu_dev_handle;
+static struct iova_domain iova_allocator;
+
+static struct hi6220_domain *to_hi6220_domain(struct iommu_domain *dom)
+{
+ return container_of(dom, struct hi6220_domain, io_domain);
+}
+
+static inline void __smmu_writel(struct hi6220_smmu *smmu_dev, u32 value,
+ unsigned long offset)
+{
+ writel(value, smmu_dev->reg_base + offset);
+}
+
+static inline u32 __smmu_readl(struct hi6220_smmu *smmu_dev,
+ unsigned long offset)
+{
+ return readl(smmu_dev->reg_base + offset);
+}
+
+/**
+ * store and load the regs
+ */
+static void __store_regs(struct hi6220_smmu *smmu_dev)
+{
+ int i;
+ u32 *data = smmu_dev->sr_data;
+
+ for (i = 0; i < SMMU_SR_REGS_NUM; i++)
+ data[i] = __smmu_readl(smmu_dev, i * 4);
+}
+
+static void __load_regs(struct hi6220_smmu *smmu_dev)
+{
+ int i;
+ u32 *data = smmu_dev->sr_data;
+
+ for (i = 0; i < SMMU_SR_REGS_NUM; i++)
+ __smmu_writel(smmu_dev, data[i], i * 4);
+}
+
+static inline void __set_smmu_pte(unsigned int *pte,
+ dma_addr_t phys_addr)
+{
+ if ((*pte & PAGE_ENTRY_VALID))
+ pr_err("set pte[%p]->%x already set!\n", pte, *pte);
+
+ *pte = (unsigned int)(phys_addr | PAGE_ENTRY_VALID);
+}
+
+static inline void __clear_smmu_pte(unsigned int *pte)
+{
+ if (!(*pte & PAGE_ENTRY_VALID))
+ pr_err("clear pte[%p] %x err!\n", pte, *pte);
+
+ *pte = 0;
+}
+
+static int __invalid_smmu_tlb(struct hi6220_domain *m_domain,
+ unsigned long iova, size_t size)
+{
+ unsigned long flags;
+ unsigned int smmu_ctrl = 0;
+ unsigned int start_pfn = 0;
+ unsigned int end_pfn = 0;
+ unsigned int tmp;
+ int ret;
+ void __iomem *reg_base;
+ struct hi6220_smmu *smmu_dev = m_domain->smmu_dev;
+ dma_addr_t smmu_pgtbl_phy = m_domain->smmu_dev->pgtable_phy;
+
+ reg_base = smmu_dev->reg_base;
+ start_pfn = IOVA_PFN(iova);
+ end_pfn = IOVA_PFN(iova + size);
+
+ spin_lock_irqsave(&smmu_dev->spinlock, flags);
+ __smmu_writel(smmu_dev, smmu_pgtbl_phy + start_pfn * sizeof(int),
+ SMMU_START_OFFSET);
+ __smmu_writel(smmu_dev, smmu_pgtbl_phy + end_pfn * sizeof(int),
+ SMMU_END_OFFSET);
+
+ smmu_ctrl = __smmu_readl(smmu_dev, SMMU_CTRL_OFFSET);
+ smmu_ctrl = smmu_ctrl | SMMU_CTRL_INVALID;
+ __smmu_writel(smmu_dev, smmu_ctrl, SMMU_CTRL_OFFSET);
+ spin_unlock_irqrestore(&smmu_dev->spinlock, flags);
+
+ ret = readl_poll_timeout_atomic(reg_base + SMMU_CNTCTRL_OFFSET, tmp,
+ !(tmp & SMMU_CTRL_INVALID), 5, 50);
+ if (ret) {
+ pr_err("invalid smmu tlb error!");
+ return ret;
+ }
+ return 0;
+}
+
+static int __smmu_enable(struct hi6220_smmu *smmu_dev)
+{
+ /**
+ * enable clock first
+ */
+ if (clk_prepare_enable(smmu_dev->media_sc_clk)) {
+ pr_err("clk_prepare_enable media_sc_clk is falied\n");
+ return -ENODEV;
+ }
+ if (clk_prepare_enable(smmu_dev->smmu_peri_clk)) {
+ pr_err("clk_prepare_enable smmu_peri_clk is falied\n");
+ return -ENODEV;
+ }
+ if (clk_prepare_enable(smmu_dev->smmu_clk)) {
+ pr_err("clk_prepare_enable smmu_clk is falied\n");
+ return -ENODEV;
+ }
+
+ /**
+ * load the store register when resume
+ */
+ if (smmu_dev->sr_data)
+ __load_regs(smmu_dev);
+
+ /*set axi id*/
+ __smmu_writel(smmu_dev, 0xC70000FF, SMMU_AXIID_OFFSET);
+ /*iommu page size 4K */
+ __smmu_writel(smmu_dev, 0x000001A6, SMMU_CTRL_OFFSET);
+ /*clear interrupt pa address*/
+ __smmu_writel(smmu_dev, 0x00000000, SMMU_IPTPA_OFFSET);
+ /*clear interrupt*/
+ __smmu_writel(smmu_dev, 0xFF, SMMU_INTCLR_OFFSET);
+
+ /*set page table phy_addr for ptable and preload*/
+ __smmu_writel(smmu_dev, smmu_dev->pgtable_phy, SMMU_PTBR_OFFSET);
+ __smmu_writel(smmu_dev, smmu_dev->pgtable_phy, SMMU_START_OFFSET);
+ /*set the page table memory size*/
+ __smmu_writel(smmu_dev, smmu_dev->pgtable_size, SMMU_END_OFFSET);
+ __smmu_writel(smmu_dev, 0x3, SMMU_ENABLE_OFFSET);
+
+ return 0;
+}
+
+/**
+ * interrupt happen when operator error
+ */
+static irqreturn_t hi6220_smmu_isr(int irq, void *data)
+{
+ int i;
+ int index;
+ unsigned int irq_stat;
+ struct hi6220_smmu *smmu_dev = (struct hi6220_smmu *)data;
+ unsigned int pgt = 0;
+ unsigned int pc_pgt = 0;
+
+ irq_stat = __smmu_readl(smmu_dev, SMMU_MINTSTS_OFFSET);
+
+ /**
+ * clear smmu interrupt
+ */
+ __smmu_writel(smmu_dev, 0xff, SMMU_INTCLR_OFFSET);
+ pgt = __smmu_readl(smmu_dev, SMMU_PTBR_OFFSET);
+ pc_pgt = __smmu_readl(smmu_dev, SMMU_IPTSRC_OFFSET);
+
+ /**
+ * dump key register of smmu
+ */
+ pr_err("\n irq status= %08x\n"
+ " SMMU_CTRL_OFFSET = %08x\n"
+ " SMMU_ENABLE_OFFSET = %08x\n"
+ " SMMU_PTBR_OFFSET = %08x\n"
+ " SMMU_START_OFFSET = %08x\n"
+ " SMMU_END_OFFSET = %08x\n"
+ " SMMU_IPTSRC_OFFSET = %08x\n",
+ irq_stat,
+ __smmu_readl(smmu_dev, SMMU_CTRL_OFFSET),
+ __smmu_readl(smmu_dev, SMMU_ENABLE_OFFSET),
+ __smmu_readl(smmu_dev, SMMU_PTBR_OFFSET),
+ __smmu_readl(smmu_dev, SMMU_START_OFFSET),
+ __smmu_readl(smmu_dev, SMMU_END_OFFSET),
+ __smmu_readl(smmu_dev, SMMU_IPTSRC_OFFSET));
+ /**
+ * test bit0 to bit5
+ */
+ if (irq_stat & 0x1)
+ pr_err("page_size L1 TLB_size ddr_size configure error\n");
+ if (irq_stat & BIT(1)) {
+ index = (pc_pgt - pgt) / 4;
+ pr_err("ptable entry error:IOMMU_VA=0x%x\n", index * SZ_4K);
+ }
+ if (irq_stat & BIT(2))
+ pr_err("AXI master0 receive error response\n");
+ if (irq_stat & BIT(3))
+ pr_err("AXI master1 receive error response\n");
+ if (irq_stat & BIT(4))
+ pr_err("AXI master0 access timeout\n");
+ if (irq_stat & BIT(5))
+ pr_err("AXI master1 access timeout\n");
+
+ /**
+ * dump all smmu register for error check
+ */
+ for (i = 0; i < SMMU_REGS_SGMT_END; i += 4)
+ pr_err("[%08x] ", __smmu_readl(smmu_dev, i));
+
+ return IRQ_HANDLED;
+}
+
+static struct iommu_domain *hi6220_domain_alloc(unsigned type)
+{
+ struct hi6220_domain *m_domain;
+ struct hi6220_smmu *smmu_dev;
+
+ if (type != IOMMU_DOMAIN_UNMANAGED)
+ return NULL;
+
+ smmu_dev = smmu_dev_handle;
+ if (!smmu_dev)
+ return NULL;
+
+ m_domain = kzalloc(sizeof(*m_domain), GFP_KERNEL);
+ if (!m_domain)
+ return NULL;
+
+ m_domain->smmu_dev = smmu_dev_handle;
+ m_domain->io_domain.geometry.aperture_start = IOVA_START;
+ m_domain->io_domain.geometry.aperture_end = IOVA_END;
+ m_domain->io_domain.geometry.force_aperture = true;
+
+ return &m_domain->io_domain;
+}
+
+static void hi6220_domain_free(struct iommu_domain *domain)
+{
+ struct hi6220_domain *hi6220_domain = to_hi6220_domain(domain);
+
+ kfree(hi6220_domain);
+}
+
+static int hi6220_smmu_attach_dev(struct iommu_domain *domain,
+ struct device *dev)
+{
+ dev->archdata.iommu = &iova_allocator;
+
+ return 0;
+}
+
+static void hi6220_smmu_detach_dev(struct iommu_domain *domain,
+ struct device *dev)
+{
+ dev->archdata.iommu = NULL;
+}
+
+static int hi6220_smmu_map(struct iommu_domain *domain,
+ unsigned long iova, phys_addr_t pa,
+ size_t size, int smmu_prot)
+{
+ struct hi6220_domain *m_domain = to_hi6220_domain(domain);
+ struct hi6220_smmu *smmu_dev = m_domain->smmu_dev;
+ unsigned int *page_table = (unsigned int *)smmu_dev->pgtable_virt;
+
+ __set_smmu_pte(page_table + IOVA_PFN(iova), pa);
+
+ __invalid_smmu_tlb(m_domain, iova, size);
+
+ return 0;
+}
+
+static size_t hi6220_smmu_unmap(struct iommu_domain *domain, unsigned long iova,
+ size_t size)
+{
+ struct hi6220_domain *m_domain = to_hi6220_domain(domain);
+ struct hi6220_smmu *smmu_dev = m_domain->smmu_dev;
+ int *page_table = (unsigned int *)smmu_dev->pgtable_virt;
+
+ __clear_smmu_pte(page_table + IOVA_PFN(iova));
+
+ __invalid_smmu_tlb(m_domain, iova, size);
+
+ return IOVA_PAGE_SZ;
+}
+
+static const struct iommu_ops hi6220_smmu_ops = {
+ .domain_alloc = hi6220_domain_alloc,
+ .domain_free = hi6220_domain_free,
+ .attach_dev = hi6220_smmu_attach_dev,
+ .detach_dev = hi6220_smmu_detach_dev,
+ .map = hi6220_smmu_map,
+ .unmap = hi6220_smmu_unmap,
+ .map_sg = default_iommu_map_sg,
+ .pgsize_bitmap = IOVA_PAGE_SZ,
+};
+
+static int hi6220_smmu_probe(struct platform_device *pdev)
+{
+ int ret;
+ int irq;
+ struct hi6220_smmu *smmu_dev = NULL;
+ struct resource *res = NULL;
+
+ smmu_dev = devm_kzalloc(&pdev->dev, sizeof(*smmu_dev), GFP_KERNEL);
+
+ res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ smmu_dev->reg_base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(smmu_dev->reg_base))
+ return PTR_ERR(smmu_dev->reg_base);
+
+ smmu_dev->pgtable_size = (IOVA_END - IOVA_START) / SZ_4K * sizeof(int);
+ smmu_dev->media_sc_clk = devm_clk_get(&pdev->dev, "media-sc");
+ smmu_dev->smmu_peri_clk = devm_clk_get(&pdev->dev, "smmu-peri");
+ smmu_dev->smmu_clk = devm_clk_get(&pdev->dev, "smmu");
+ if (IS_ERR(smmu_dev->media_sc_clk) || IS_ERR(smmu_dev->smmu_peri_clk) ||
+ IS_ERR(smmu_dev->media_sc_clk)) {
+ pr_err("clk is not ready!\n");
+ return -EINVAL;
+ }
+
+ irq = platform_get_irq(pdev, 0);
+ ret = devm_request_irq(&pdev->dev, irq, hi6220_smmu_isr, 0,
+ dev_name(&pdev->dev), smmu_dev);
+ if (ret) {
+ pr_err("Unabled to register handler of irq %d\n", irq);
+ return ret;
+ }
+
+ spin_lock_init(&smmu_dev->spinlock);
+
+ __smmu_enable(smmu_dev);
+
+ iova_cache_get();
+ init_iova_domain(&iova_allocator, IOVA_PAGE_SZ,
+ IOVA_PFN(IOVA_START), IOVA_PFN(IOVA_END));
+
+ smmu_dev->sr_data = vmalloc(sizeof(u32) * SMMU_SR_REGS_NUM);
+ if (!smmu_dev->sr_data) {
+ iova_cache_put();
+ return -ENOMEM;
+ }
+
+ dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(32));
+ smmu_dev->pgtable_virt = dma_zalloc_coherent(&pdev->dev,
+ smmu_dev->pgtable_size,
+ &smmu_dev->pgtable_phy,
+ GFP_KERNEL);
+
+ if (!smmu_dev->pgtable_virt) {
+ pr_err("alloc pagetable mem failed\n");
+ vfree(smmu_dev->sr_data);
+ iova_cache_put();
+ return -ENOMEM;
+ }
+
+ platform_set_drvdata(pdev, smmu_dev);
+ bus_set_iommu(&platform_bus_type, &hi6220_smmu_ops);
+ smmu_dev_handle = smmu_dev;
+
+ return 0;
+}
+
+static int hi6220_smmu_suspend(struct platform_device *pdev,
+ pm_message_t state)
+{
+ struct hi6220_smmu *smmu_dev = dev_get_drvdata(&pdev->dev);
+
+ __store_regs(smmu_dev);
+
+ if (smmu_dev->smmu_clk)
+ clk_disable_unprepare(smmu_dev->smmu_clk);
+ if (smmu_dev->media_sc_clk)
+ clk_disable_unprepare(smmu_dev->media_sc_clk);
+ if (smmu_dev->smmu_peri_clk)
+ clk_disable_unprepare(smmu_dev->smmu_peri_clk);
+
+ return 0;
+}
+
+static int hi6220_smmu_resume(struct platform_device *pdev)
+{
+ struct hi6220_smmu *smmu_dev = dev_get_drvdata(&pdev->dev);
+
+ __smmu_enable(smmu_dev);
+ return 0;
+}
+
+static const struct of_device_id of_smmu_match_tbl[] = {
+ {
+ .compatible = "hisilicon,hi6220-smmu",
+ },
+ { }
+};
+
+static struct platform_driver hi6220_smmu_driver = {
+ .driver = {
+ .name = "smmu-hi6220",
+ .of_match_table = of_smmu_match_tbl,
+ },
+ .probe = hi6220_smmu_probe,
+ .suspend = hi6220_smmu_suspend,
+ .resume = hi6220_smmu_resume,
+};
+
+static int __init hi6220_smmu_init(void)
+{
+ int ret;
+
+ ret = platform_driver_register(&hi6220_smmu_driver);
+ return ret;
+}
+
+subsys_initcall(hi6220_smmu_init);
--
1.9.1

2015-11-20 02:28:57

by Chen Feng

[permalink] [raw]
Subject: [PATCH V5 RESEND 3/3] arm64: dts: Add dts node for hi6220 smmu driver

Add iommu node for hi6220 SoC platform

Signed-off-by: Chen Feng <[email protected]>
---
arch/arm64/boot/dts/hisilicon/hi6220.dtsi | 13 +++++++++++++
1 file changed, 13 insertions(+)

diff --git a/arch/arm64/boot/dts/hisilicon/hi6220.dtsi b/arch/arm64/boot/dts/hisilicon/hi6220.dtsi
index 82d2488..589424a 100644
--- a/arch/arm64/boot/dts/hisilicon/hi6220.dtsi
+++ b/arch/arm64/boot/dts/hisilicon/hi6220.dtsi
@@ -208,5 +208,18 @@
clock-names = "uartclk", "apb_pclk";
status = "disabled";
};
+
+ iommu: iommu@f4210000 {
+ compatible = "hisilicon,hi6220-smmu";
+ reg = <0x0 0xf4210000 0x0 0x1000>;
+ interrupts = <GIC_SPI 113 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&sys_ctrl HI6220_MMU_CLK>,
+ <&media_ctrl HI6220_MED_MMU>,
+ <&sys_ctrl HI6220_MEDIA_PLL_SRC>;
+ clock-names = "smmu",
+ "media-sc",
+ "smmu-peri";
+ #iommu-cells = <0>;
+ };
};
};
--
1.9.1

2015-11-20 14:21:56

by Rob Herring

[permalink] [raw]
Subject: Re: [PATCH V5 RESEND 1/3] docs: iommu: Documentation for iommu in hi6220 SoC

On Fri, Nov 20, 2015 at 10:25:07AM +0800, Chen Feng wrote:
> Documentation for hi6220 iommu driver.
>
> Signed-off-by: Chen Feng <[email protected]>

Acked-by: Rob Herring <[email protected]>

> ---
> .../bindings/iommu/hisi,hi6220-iommu.txt | 32 ++++++++++++++++++++++
> 1 file changed, 32 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/iommu/hisi,hi6220-iommu.txt
>
> diff --git a/Documentation/devicetree/bindings/iommu/hisi,hi6220-iommu.txt b/Documentation/devicetree/bindings/iommu/hisi,hi6220-iommu.txt
> new file mode 100644
> index 0000000..44f9101
> --- /dev/null
> +++ b/Documentation/devicetree/bindings/iommu/hisi,hi6220-iommu.txt
> @@ -0,0 +1,32 @@
> +Hi6220 SoC SMMU Device Driver devicetree document
> +The media system shared the same smmu IP to access DDR memory. And all
> +media IP used the same page table.
> +
> +Below binding describes the system mmu for media system in hi6220 platform
> +
> +Required properties:
> +- compatible: should contain "hisilicon,hi6220-smmu".
> +- reg: A tuple of base address and size of System MMU registers.
> +- clocks: a list of phandle + clock-specifier pairs, one for each entry
> + in clock-names.
> +- clock-names: should contain:
> + * "smmu"
> + * "media-sc"
> + * "smmu-peri"
> +- interrupts: An interrupt specifier for interrupt signal of System MMU.
> +- #iommu-cells: The iommu-cells should be 0. Because no additional information
> + needs to be encoded in the specifier.
> +
> +Examples:
> + iommu@f4210000 {
> + compatible = "hisilicon,hi6220-smmu";
> + reg = <0x0 0xf4210000 0x0 0x1000>;
> + interrupts = <GIC_SPI 113 IRQ_TYPE_LEVEL_HIGH>;
> + clocks = <&sys_ctrl HI6220_MMU_CLK>,
> + <&media_ctrl HI6220_MED_MMU>,
> + <&sys_ctrl HI6220_MEDIA_PLL_SRC>;
> + clock-names = "smmu",
> + "media-sc",
> + "smmu-peri";
> + #iommu-cells = <0>;
> + };
> --
> 1.9.1
>
> --
> To unsubscribe from this list: send the line "unsubscribe devicetree" in
> the body of a message to [email protected]
> More majordomo info at http://vger.kernel.org/majordomo-info.html

2015-11-27 11:49:28

by Joerg Roedel

[permalink] [raw]
Subject: Re: [PATCH V5 RESEND 0/3] Add iommu support for hi6220 HiKey board

On Fri, Nov 20, 2015 at 10:25:06AM +0800, Chen Feng wrote:
> eg:
> struct iommu_domain *domain = iommu_domain_alloc(bus);
> iommu_attach_device(domain, dev);
> struct iova_domain *iovad = (struct iova_domain *)m_dev->archdata.iommu;
> struct iova * t_iova = alloc_iova(iovad, size, limit_pfn, align);
> iommu_map(domain, t_iova->pfn_lo << 12, phy_addr, size, port);

Any reason the media subsystem should use the iommu-api directly, and
not the dma-api? You can make your iommu available by adapting the
common dma-iommu implementation from Robin Murphy to your iommu to make
it usable through the dma-api.


Joerg

2015-11-27 12:02:13

by Joerg Roedel

[permalink] [raw]
Subject: Re: [PATCH V5 RESEND 2/3] iommu/hisilicon: Add hi6220-SoC smmu driver

On Fri, Nov 20, 2015 at 10:25:08AM +0800, Chen Feng wrote:
> +config HI6220_IOMMU
> + bool "Hi6220 IOMMU Support"
> + depends on ARM64
> + select IOMMU_API
> + select IOMMU_IOVA
> + help
> + Enable IOMMU Driver for hi6220 SoC. The IOMMU API and IOMMU IOVA
> + is also selected.

The last sentence is of little help for the user. Better put the reasons
in here when a user should select this option.

> + /*set axi id*/

Coding style nit: Please write these oneline comments with spaces, like
this:

/* set axi id */

> +static int hi6220_smmu_attach_dev(struct iommu_domain *domain,
> + struct device *dev)
> +{
> + dev->archdata.iommu = &iova_allocator;
> +
> + return 0;
> +}
> +
> +static void hi6220_smmu_detach_dev(struct iommu_domain *domain,
> + struct device *dev)
> +{
> + dev->archdata.iommu = NULL;
> +}

This basically means that this driver only supports one domain, right?
That is not compatible with the iommu-api requirements.

You need to create an iommu-group per smmu in your system and put all
devices translated by this smmu in that group. And then you must change
your code to allow attaching/detaching this iommu-group to different
domains.


Joerg

2015-11-28 02:09:35

by Chen Feng

[permalink] [raw]
Subject: Re: [PATCH V5 RESEND 0/3] Add iommu support for hi6220 HiKey board



On 2015/11/27 19:49, Joerg Roedel wrote:
> On Fri, Nov 20, 2015 at 10:25:06AM +0800, Chen Feng wrote:
>> eg:
>> struct iommu_domain *domain = iommu_domain_alloc(bus);
>> iommu_attach_device(domain, dev);
>> struct iova_domain *iovad = (struct iova_domain *)m_dev->archdata.iommu;
>> struct iova * t_iova = alloc_iova(iovad, size, limit_pfn, align);
>> iommu_map(domain, t_iova->pfn_lo << 12, phy_addr, size, port);
>
> Any reason the media subsystem should use the iommu-api directly, and
> not the dma-api? You can make your iommu available by adapting the
> common dma-iommu implementation from Robin Murphy to your iommu to make
> it usable through the dma-api.
>

ok, I will change this to dma-api. Since v1 version, the dma-api was not ok for arm64.
I will do this with dma-api function.
>
> Joerg
>
>
> .
>

2015-11-28 02:19:42

by Chen Feng

[permalink] [raw]
Subject: Re: [PATCH V5 RESEND 2/3] iommu/hisilicon: Add hi6220-SoC smmu driver



On 2015/11/27 20:02, Joerg Roedel wrote:
> On Fri, Nov 20, 2015 at 10:25:08AM +0800, Chen Feng wrote:
>> +config HI6220_IOMMU
>> + bool "Hi6220 IOMMU Support"
>> + depends on ARM64
>> + select IOMMU_API
>> + select IOMMU_IOVA
>> + help
>> + Enable IOMMU Driver for hi6220 SoC. The IOMMU API and IOMMU IOVA
>> + is also selected.
>
> The last sentence is of little help for the user. Better put the reasons
> in here when a user should select this option.
>
>> + /*set axi id*/
>
> Coding style nit: Please write these oneline comments with spaces, like
> this:
>
> /* set axi id */
>

learned,Thanks.
>> +static int hi6220_smmu_attach_dev(struct iommu_domain *domain,
>> + struct device *dev)
>> +{
>> + dev->archdata.iommu = &iova_allocator;
>> +
>> + return 0;
>> +}
>> +
>> +static void hi6220_smmu_detach_dev(struct iommu_domain *domain,
>> + struct device *dev)
>> +{
>> + dev->archdata.iommu = NULL;
>> +}
>
> This basically means that this driver only supports one domain, right?
yes.
> That is not compatible with the iommu-api requirements.
>
> You need to create an iommu-group per smmu in your system and put all
> devices translated by this smmu in that group. And then you must change
> your code to allow attaching/detaching this iommu-group to different
> domains.
>
I read the code,I am confused about these concepts.
In my opinion,
IOMMU-Domain: The masters in one domain share the same iova space. Is that right?
IOMMU-GROUP: As you mentioned up,all devices translated by this smmu should be into one group.
I can do this. But if there is only on domain in the system, how can I attaching/detaching the group
to different domains.

I think I have something wrong with these concepts.

Could you help to give me some suggestions.

>
> Joerg
>
>
> .
>

2015-11-28 12:20:08

by Joerg Roedel

[permalink] [raw]
Subject: Re: [PATCH V5 RESEND 2/3] iommu/hisilicon: Add hi6220-SoC smmu driver

On Sat, Nov 28, 2015 at 10:19:14AM +0800, chenfeng wrote:
> On 2015/11/27 20:02, Joerg Roedel wrote:
> > You need to create an iommu-group per smmu in your system and put all
> > devices translated by this smmu in that group. And then you must change
> > your code to allow attaching/detaching this iommu-group to different
> > domains.
> >
> I read the code,I am confused about these concepts.
> In my opinion,
> IOMMU-Domain: The masters in one domain share the same iova space. Is that right?

Yes, an iommu-domain is an abstraction of a single page-table and
describes one iova address space. The SMMU in your hardware supports
only on page-table, so all devices behind it share a single iova address
space, und must be in the same domain.

But the way you implemented it there could be only one domain per SMMU.
With the IOMMU-API we must support to change the domain of an SMMU.
Think of one domain for DMA-API usage and another for VFIO, for example.

> IOMMU-GROUP: As you mentioned up,all devices translated by this smmu
> should be into one group. I can do this. But if there is only on
> domain in the system, how can I attaching/detaching the group to
> different domains.

IOMMU-groups describe the isolation capabilities of the IOMMU between
devices. If devices are in a different IOMMU-group, this means that
the hardware can isolate them from each other.

Attaching the devices in a group to a new domain means in your case,
that you set a different page-table for them. Since you can only do that
for all devices behind a SMMU, these devices must be in on IOMMU-group.



Joerg