Received: by 2002:a25:824b:0:0:0:0:0 with SMTP id d11csp391215ybn; Tue, 1 Oct 2019 23:12:03 -0700 (PDT) X-Google-Smtp-Source: APXvYqwFZbO7x9mOSpmAIs1gc7wBAVcIlMMdWtMUK7jyLDMyd+ToGZX3bYTa28uvr6qIndrYUZgH X-Received: by 2002:a50:f603:: with SMTP id c3mr1983428edn.208.1569996723297; Tue, 01 Oct 2019 23:12:03 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1569996723; cv=none; d=google.com; s=arc-20160816; b=dtS+JZXoAI0Nz7Dq0Th74ezqPGGf2x74IlusVxRW0NJ0VZoKkl5ZIp5Wf1v3czxJr7 uxvxrYAfRFpLwC28y2JDRAWGGw8El0bjzgd/xnRM3kxUpIJiBXACYfLhkOuBdENzugfg l5qDzNBXeIOcMcJ6RsKCZ9XAt1p5Rf4VBT5JumlMpOMsfrmVOsdY2s5LVx6BQ6zm9Hqc q3S4yqJRFCaAOXpWh/eE16nMwF5cti4i7Vdjv5eGMegk34tamMBFoTGOy0fuRF1QKOAS xxuQNT9O0hGzvZtsyzPq7lS9TmLu9tJFPu8/FzfvIhhyDT50/RYkTTLU1243gZ9fkSSd wBcw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:sender:references:in-reply-to:message-id:date :subject:cc:to:from; bh=W6IfRLTol8Q/HcmDvS/M0BOEHI/SO9se98o0lDtNhVs=; b=nlhPx/8wU8QveSCRPcii0+7QWIL1sOQVv0QVcxDQFD91DYPEBEJXIM+njTDVKfLNX2 1yisaDaUKUD/RgXwijZEFzRwHPlyDhuV6LoWqGPBxxgrWuXC26OR7MxUMZSrzHlhDeEB JZ1Z1yJ9pycVkBn9/47jApD2WvztUT1CtEbtbfHAWfwGosDQJqct9sLBa/QyymWQ5f8/ 5M8RUy5MmCP3rmr6s7vo0vPWxmc4EzbL5Q5ZU8Sie2GxZ8NTsUbz8LZ+IAcaCxzFW5cF mU/ZHKsyHQHnEKBiRN/65xSEsc2U8idnZf1WKwwScHXskzqGx5d5RL6e7LlROsf/xGLM ohjg== ARC-Authentication-Results: i=1; mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from vger.kernel.org (vger.kernel.org. [209.132.180.67]) by mx.google.com with ESMTP id v57si10261054edc.171.2019.10.01.23.11.39; Tue, 01 Oct 2019 23:12:03 -0700 (PDT) Received-SPF: pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) client-ip=209.132.180.67; Authentication-Results: mx.google.com; spf=pass (google.com: best guess record for domain of linux-kernel-owner@vger.kernel.org designates 209.132.180.67 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=fail (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1729044AbfJAX3X (ORCPT + 99 others); Tue, 1 Oct 2019 19:29:23 -0400 Received: from mga17.intel.com ([192.55.52.151]:26348 "EHLO mga17.intel.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1729005AbfJAX3V (ORCPT ); Tue, 1 Oct 2019 19:29:21 -0400 X-Amp-Result: SKIPPED(no attachment in message) X-Amp-File-Uploaded: False Received: from orsmga002.jf.intel.com ([10.7.209.21]) by fmsmga107.fm.intel.com with ESMTP/TLS/DHE-RSA-AES256-GCM-SHA384; 01 Oct 2019 16:29:17 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.64,572,1559545200"; d="scan'208";a="203402508" Received: from jacob-builder.jf.intel.com ([10.7.199.155]) by orsmga002.jf.intel.com with ESMTP; 01 Oct 2019 16:29:17 -0700 From: Jacob Pan To: iommu@lists.linux-foundation.org, LKML , Joerg Roedel , David Woodhouse , Alex Williamson , Jean-Philippe Brucker Cc: "Yi Liu" , "Tian, Kevin" , Raj Ashok , "Christoph Hellwig" , "Lu Baolu" , Jonathan Cameron , Eric Auger , Jerry Snitselaar , Jean-Philippe Brucker , Jacob Pan Subject: [PATCH v3 2/4] iommu: Add I/O ASID allocator Date: Tue, 1 Oct 2019 16:33:23 -0700 Message-Id: <1569972805-27664-3-git-send-email-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.7.4 In-Reply-To: <1569972805-27664-1-git-send-email-jacob.jun.pan@linux.intel.com> References: <1569972805-27664-1-git-send-email-jacob.jun.pan@linux.intel.com> Sender: linux-kernel-owner@vger.kernel.org Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org From: Jean-Philippe Brucker Some devices might support multiple DMA address spaces, in particular those that have the PCI PASID feature. PASID (Process Address Space ID) allows to share process address spaces with devices (SVA), partition a device into VM-assignable entities (VFIO mdev) or simply provide multiple DMA address space to kernel drivers. Add a global PASID allocator usable by different drivers at the same time. Name it I/O ASID to avoid confusion with ASIDs allocated by arch code, which are usually a separate ID space. The IOASID space is global. Each device can have its own PASID space, but by convention the IOMMU ended up having a global PASID space, so that with SVA, each mm_struct is associated to a single PASID. The allocator is primarily used by IOMMU subsystem but in rare occasions drivers would like to allocate PASIDs for devices that aren't managed by an IOMMU, using the same ID space as IOMMU. Signed-off-by: Jean-Philippe Brucker Signed-off-by: Jacob Pan --- drivers/iommu/Kconfig | 4 ++ drivers/iommu/Makefile | 1 + drivers/iommu/ioasid.c | 151 +++++++++++++++++++++++++++++++++++++++++++++++++ include/linux/ioasid.h | 48 ++++++++++++++++ 4 files changed, 204 insertions(+) create mode 100644 drivers/iommu/ioasid.c create mode 100644 include/linux/ioasid.h diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig index e3842eabcfdd..fd50ddffffbf 100644 --- a/drivers/iommu/Kconfig +++ b/drivers/iommu/Kconfig @@ -3,6 +3,10 @@ config IOMMU_IOVA tristate +# The IOASID library may also be used by non-IOMMU_API users +config IOASID + tristate + # IOMMU_API always gets selected by whoever wants it. config IOMMU_API bool diff --git a/drivers/iommu/Makefile b/drivers/iommu/Makefile index 4f405f926e73..35d17094fe3b 100644 --- a/drivers/iommu/Makefile +++ b/drivers/iommu/Makefile @@ -7,6 +7,7 @@ obj-$(CONFIG_IOMMU_DMA) += dma-iommu.o obj-$(CONFIG_IOMMU_IO_PGTABLE) += io-pgtable.o obj-$(CONFIG_IOMMU_IO_PGTABLE_ARMV7S) += io-pgtable-arm-v7s.o obj-$(CONFIG_IOMMU_IO_PGTABLE_LPAE) += io-pgtable-arm.o +obj-$(CONFIG_IOASID) += ioasid.o obj-$(CONFIG_IOMMU_IOVA) += iova.o obj-$(CONFIG_OF_IOMMU) += of_iommu.o obj-$(CONFIG_MSM_IOMMU) += msm_iommu.o diff --git a/drivers/iommu/ioasid.c b/drivers/iommu/ioasid.c new file mode 100644 index 000000000000..6fbea76a47cf --- /dev/null +++ b/drivers/iommu/ioasid.c @@ -0,0 +1,151 @@ +// SPDX-License-Identifier: GPL-2.0 +/* + * I/O Address Space ID allocator. There is one global IOASID space, split into + * subsets. Users create a subset with DECLARE_IOASID_SET, then allocate and + * free IOASIDs with ioasid_alloc and ioasid_free. + */ +#include +#include +#include +#include +#include + +struct ioasid_data { + ioasid_t id; + struct ioasid_set *set; + void *private; + struct rcu_head rcu; +}; + +static DEFINE_XARRAY_ALLOC(ioasid_xa); + +/** + * ioasid_set_data - Set private data for an allocated ioasid + * @ioasid: the ID to set data + * @data: the private data + * + * For IOASID that is already allocated, private data can be set + * via this API. Future lookup can be done via ioasid_find. + */ +int ioasid_set_data(ioasid_t ioasid, void *data) +{ + struct ioasid_data *ioasid_data; + int ret = 0; + + xa_lock(&ioasid_xa); + ioasid_data = xa_load(&ioasid_xa, ioasid); + if (ioasid_data) + rcu_assign_pointer(ioasid_data->private, data); + else + ret = -ENOENT; + xa_unlock(&ioasid_xa); + + /* + * Wait for readers to stop accessing the old private data, so the + * caller can free it. + */ + if (!ret) + synchronize_rcu(); + + return ret; +} +EXPORT_SYMBOL_GPL(ioasid_set_data); + +/** + * ioasid_alloc - Allocate an IOASID + * @set: the IOASID set + * @min: the minimum ID (inclusive) + * @max: the maximum ID (inclusive) + * @private: data private to the caller + * + * Allocate an ID between @min and @max. The @private pointer is stored + * internally and can be retrieved with ioasid_find(). + * + * Return: the allocated ID on success, or %INVALID_IOASID on failure. + */ +ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min, ioasid_t max, + void *private) +{ + ioasid_t id; + struct ioasid_data *data; + + data = kzalloc(sizeof(*data), GFP_KERNEL); + if (!data) + return INVALID_IOASID; + + data->set = set; + data->private = private; + + if (xa_alloc(&ioasid_xa, &id, data, XA_LIMIT(min, max), GFP_KERNEL)) { + pr_err("Failed to alloc ioasid from %d to %d\n", min, max); + goto exit_free; + } + data->id = id; + + return id; +exit_free: + kfree(data); + return INVALID_IOASID; +} +EXPORT_SYMBOL_GPL(ioasid_alloc); + +/** + * ioasid_free - Free an IOASID + * @ioasid: the ID to remove + */ +void ioasid_free(ioasid_t ioasid) +{ + struct ioasid_data *ioasid_data; + + ioasid_data = xa_erase(&ioasid_xa, ioasid); + + kfree_rcu(ioasid_data, rcu); +} +EXPORT_SYMBOL_GPL(ioasid_free); + +/** + * ioasid_find - Find IOASID data + * @set: the IOASID set + * @ioasid: the IOASID to find + * @getter: function to call on the found object + * + * The optional getter function allows to take a reference to the found object + * under the rcu lock. The function can also check if the object is still valid: + * if @getter returns false, then the object is invalid and NULL is returned. + * + * If the IOASID exists, return the private pointer passed to ioasid_alloc. + * Private data can be NULL if not set. Return an error if the IOASID is not + * found, or if @set is not NULL and the IOASID does not belong to the set. + */ +void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid, + bool (*getter)(void *)) +{ + void *priv; + struct ioasid_data *ioasid_data; + + rcu_read_lock(); + ioasid_data = xa_load(&ioasid_xa, ioasid); + if (!ioasid_data) { + priv = ERR_PTR(-ENOENT); + goto unlock; + } + if (set && ioasid_data->set != set) { + /* data found but does not belong to the set */ + priv = ERR_PTR(-EACCES); + goto unlock; + } + /* Now IOASID and its set is verified, we can return the private data */ + priv = rcu_dereference(ioasid_data->private); + if (getter && !getter(priv)) + priv = NULL; +unlock: + rcu_read_unlock(); + + return priv; +} +EXPORT_SYMBOL_GPL(ioasid_find); + +MODULE_AUTHOR("Jean-Philippe Brucker "); +MODULE_AUTHOR("Jacob Pan "); +MODULE_DESCRIPTION("IO Address Space ID (IOASID) allocator"); +MODULE_LICENSE("GPL"); diff --git a/include/linux/ioasid.h b/include/linux/ioasid.h new file mode 100644 index 000000000000..17337a13f06e --- /dev/null +++ b/include/linux/ioasid.h @@ -0,0 +1,48 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef __LINUX_IOASID_H +#define __LINUX_IOASID_H + +#include +#include + +#define INVALID_IOASID ((ioasid_t)-1) +typedef unsigned int ioasid_t; + +struct ioasid_set { + int dummy; +}; + +#define DECLARE_IOASID_SET(name) struct ioasid_set name = { 0 } + +#if IS_ENABLED(CONFIG_IOASID) +ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min, ioasid_t max, + void *private); +void ioasid_free(ioasid_t ioasid); +void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid, + bool (*getter)(void *)); +int ioasid_set_data(ioasid_t ioasid, void *data); + +#else /* !CONFIG_IOASID */ +static inline ioasid_t ioasid_alloc(struct ioasid_set *set, ioasid_t min, + ioasid_t max, void *private) +{ + return INVALID_IOASID; +} + +static inline void ioasid_free(ioasid_t ioasid) +{ +} + +static inline void *ioasid_find(struct ioasid_set *set, ioasid_t ioasid, + bool (*getter)(void *)) +{ + return NULL; +} + +static inline int ioasid_set_data(ioasid_t ioasid, void *data) +{ + return -ENOTSUPP; +} + +#endif /* CONFIG_IOASID */ +#endif /* __LINUX_IOASID_H */ -- 2.7.4