Received: by 2002:a05:6358:4e97:b0:b3:742d:4702 with SMTP id ce23csp4277759rwb; Tue, 16 Aug 2022 18:45:58 -0700 (PDT) X-Google-Smtp-Source: AA6agR7X5qkjrJMvPrkPt0U1tXoOS4J/pltQubNT1UTIToBTbfJy0VKryTzEyJKZJzcqPEniV1bu X-Received: by 2002:a63:484e:0:b0:429:411b:1d8c with SMTP id x14-20020a63484e000000b00429411b1d8cmr10114015pgk.623.1660700757739; Tue, 16 Aug 2022 18:45:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1660700757; cv=none; d=google.com; s=arc-20160816; b=RPxkRzF9jtJI8g3ItLLwTDPHctenaZ3CaMyspko8ZNzLbAZ1zkOPzF1CaidFdVHMKK Zd95etOGnTEBONmje9RrmpYyiBHeQ+6yzBQpZCiyZJ/SwSRW1P7tvaeP6kC0J8rH/oqj mrjddC/1Fy5+45a38IPo9PLnwttQO7l0cT8vHLB5Ke1dfCG0bmzb/thWCEyEjTCf3Jp4 3qdEYErRzirs5+dYWfbe8YIwNyhz3fphKRfmYx4/a7hoXe7RMw/murOwPINp8aWHHXWx 7A/SI/2gA1Jbj+0urN/QfdF290N4P/RLkdNg4s45z5pyLqLzZdiXyHeIivJl3r+PyePJ v6hg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=hr/+sFNtm4kgbOLCIwtfJQwK1LLTyujFADAixplPhJo=; b=cXqWysAWUUMNtLqlAkIEtH+uNbTsYUocZsNlypvmAMZ6xpFg7dVO8hHBy8pV4RsFcq EB5NTfrVd0aCppcFQ3ITZEvc5Rvji/WNj+xHfQ5B1jqaZKJNygC3rVz8NXXs1ABsbv7w C1SXfh075qeH1pJbTk/AibudtovuVUd9ur5JMj7vzXdbC4Ri34cwmNMv4mefWOAv7I/f zyALGMSwZgo6g2IqsRQ6jNIqpw+oOMHLppuM+FVIJXpmy1u4WvS5s7LCrsyB2PMcq0eW gFEHREeIQOqqlJntkKDqNK6OUJRjcRUZ2JnuvDnCFoDJDOimA+PGPhxkmrSWeUzeRHTr efJg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SHXl4Z5B; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id 40-20020a17090a0fab00b001f4f3ed7a4fsi500802pjz.83.2022.08.16.18.45.37; Tue, 16 Aug 2022 18:45:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=SHXl4Z5B; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238167AbiHQB0V (ORCPT + 99 others); Tue, 16 Aug 2022 21:26:21 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:52902 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238150AbiHQB0O (ORCPT ); Tue, 16 Aug 2022 21:26:14 -0400 Received: from mga03.intel.com (mga03.intel.com [134.134.136.65]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 4728275FE2; Tue, 16 Aug 2022 18:26:13 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1660699573; x=1692235573; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=2miVSApkJpe5AYi5mZA+1ip1HHx9olblFpumTVmuV2Y=; b=SHXl4Z5Bf3Q+5Fd8Y/7B18SVRwbZK8Hf2JsfuQnhyirzBA8U4rHFvn9t yq+qiJZjYW5a2hCWFWa9EN4rxsX5ArAXzBAuNh2Jh4xPC8wRGGNxNcMyv S1oWZQTpx0P4TEZ04Es5gAeXmEYMUlUBFU51kxJ+xGINyriO3yGd9oUk0 weym33d4LzZGR7sup10Abg1MCXQjyKPgZzZYsVUCN0Zj/cy5Sr78PP3ip HaNQhuVvqGgFgwrVgMewXc1A2aZ66g4+hYRtD/4DZV+1w/Sit9g5Q8y8Z 16qIsqdTTjF162e++cY1JFxd6Bn7ebWIS1PVO489uBNNcd3n9Xc93TV3v g==; X-IronPort-AV: E=McAfee;i="6400,9594,10441"; a="293649206" X-IronPort-AV: E=Sophos;i="5.93,242,1654585200"; d="scan'208";a="293649206" Received: from fmsmga003.fm.intel.com ([10.253.24.29]) by orsmga103.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 16 Aug 2022 18:26:13 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.93,242,1654585200"; d="scan'208";a="696587911" Received: from allen-box.sh.intel.com ([10.239.159.48]) by FMSMGA003.fm.intel.com with ESMTP; 16 Aug 2022 18:26:08 -0700 From: Lu Baolu To: Joerg Roedel , Jason Gunthorpe , Christoph Hellwig , Bjorn Helgaas , Kevin Tian , Ashok Raj , Will Deacon , Robin Murphy , Jean-Philippe Brucker , Dave Jiang , Vinod Koul Cc: Eric Auger , Liu Yi L , Jacob jun Pan , Zhangfei Gao , Zhu Tony , iommu@lists.linux.dev, linux-pci@vger.kernel.org, linux-kernel@vger.kernel.org, Lu Baolu , Jean-Philippe Brucker Subject: [PATCH v11 05/13] iommu: Add attach/detach_dev_pasid iommu interface Date: Wed, 17 Aug 2022 09:20:16 +0800 Message-Id: <20220817012024.3251276-6-baolu.lu@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20220817012024.3251276-1-baolu.lu@linux.intel.com> References: <20220817012024.3251276-1-baolu.lu@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-4.3 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,SPF_HELO_NONE, SPF_NONE,T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Attaching an IOMMU domain to a PASID of a device is a generic operation for modern IOMMU drivers which support PASID-granular DMA address translation. Currently visible usage scenarios include (but not limited): - SVA (Shared Virtual Address) - kernel DMA with PASID - hardware-assist mediated device This adds set_dev_pasid domain ops for this purpose and also adds some interfaces for device drivers to attach/detach/retrieve a domain for a PASID of a device. If multiple devices share a single group, it's fine as long the fabric always routes every TLP marked with a PASID to the host bridge and only the host bridge. For example, ACS achieves this universally and has been checked when pci_enable_pasid() is called. As we can't reliably tell the source apart in a group, all the devices in a group have to be considered as the same source, and mapped to the same PASID table. Signed-off-by: Lu Baolu Reviewed-by: Jean-Philippe Brucker Reviewed-by: Kevin Tian Reviewed-by: Yi Liu --- include/linux/iommu.h | 26 +++++++++ drivers/iommu/iommu.c | 123 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 149 insertions(+) diff --git a/include/linux/iommu.h b/include/linux/iommu.h index 2f237c3cd680..f1e8953b1e2e 100644 --- a/include/linux/iommu.h +++ b/include/linux/iommu.h @@ -266,6 +266,7 @@ struct iommu_ops { * struct iommu_domain_ops - domain specific operations * @attach_dev: attach an iommu domain to a device * @detach_dev: detach an iommu domain from a device + * @set_dev_pasid: set an iommu domain to a pasid of device * @map: map a physically contiguous memory region to an iommu domain * @map_pages: map a physically contiguous set of pages of the same size to * an iommu domain. @@ -286,6 +287,8 @@ struct iommu_ops { struct iommu_domain_ops { int (*attach_dev)(struct iommu_domain *domain, struct device *dev); void (*detach_dev)(struct iommu_domain *domain, struct device *dev); + int (*set_dev_pasid)(struct iommu_domain *domain, struct device *dev, + ioasid_t pasid); int (*map)(struct iommu_domain *domain, unsigned long iova, phys_addr_t paddr, size_t size, int prot, gfp_t gfp); @@ -680,6 +683,12 @@ int iommu_group_claim_dma_owner(struct iommu_group *group, void *owner); void iommu_group_release_dma_owner(struct iommu_group *group); bool iommu_group_dma_owner_claimed(struct iommu_group *group); +int iommu_attach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid); +void iommu_detach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid); +struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid); #else /* CONFIG_IOMMU_API */ struct iommu_ops {}; @@ -1047,6 +1056,23 @@ static inline bool iommu_group_dma_owner_claimed(struct iommu_group *group) { return false; } + +static inline int iommu_attach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + return -ENODEV; +} + +static inline void iommu_detach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ +} + +static inline struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid) +{ + return NULL; +} #endif /* CONFIG_IOMMU_API */ /** diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index 1d28a74a0511..6f2cbccc0570 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -39,6 +39,7 @@ struct iommu_group { struct kobject kobj; struct kobject *devices_kobj; struct list_head devices; + struct xarray pasid_array; struct mutex mutex; void *iommu_data; void (*iommu_data_release)(void *iommu_data); @@ -663,6 +664,7 @@ struct iommu_group *iommu_group_alloc(void) mutex_init(&group->mutex); INIT_LIST_HEAD(&group->devices); INIT_LIST_HEAD(&group->entry); + xa_init(&group->pasid_array); ret = ida_alloc(&iommu_group_ida, GFP_KERNEL); if (ret < 0) { @@ -3258,3 +3260,124 @@ bool iommu_group_dma_owner_claimed(struct iommu_group *group) return user; } EXPORT_SYMBOL_GPL(iommu_group_dma_owner_claimed); + +static int __iommu_set_group_pasid(struct iommu_domain *domain, + struct iommu_group *group, ioasid_t pasid) +{ + struct iommu_domain *ops_domain; + struct group_device *device; + int ret = 0; + + if (domain == group->blocking_domain) + ops_domain = xa_load(&group->pasid_array, pasid); + else + ops_domain = domain; + + list_for_each_entry(device, &group->devices, list) { + ret = ops_domain->ops->set_dev_pasid(domain, device->dev, pasid); + if (ret) + break; + } + + return ret; +} + +/* + * iommu_attach_device_pasid() - Attach a domain to pasid of device + * @domain: the iommu domain. + * @dev: the attached device. + * @pasid: the pasid of the device. + * + * Return: 0 on success, or an error. + */ +int iommu_attach_device_pasid(struct iommu_domain *domain, + struct device *dev, ioasid_t pasid) +{ + struct iommu_group *group; + void *curr; + int ret; + + if (!domain->ops->set_dev_pasid) + return -EOPNOTSUPP; + + group = iommu_group_get(dev); + if (!group) + return -ENODEV; + + mutex_lock(&group->mutex); + curr = xa_cmpxchg(&group->pasid_array, pasid, NULL, domain, GFP_KERNEL); + if (curr) { + ret = xa_err(curr) ? : -EBUSY; + goto out_unlock; + } + + ret = __iommu_set_group_pasid(domain, group, pasid); + if (ret) { + __iommu_set_group_pasid(group->blocking_domain, group, pasid); + xa_erase(&group->pasid_array, pasid); + } +out_unlock: + mutex_unlock(&group->mutex); + iommu_group_put(group); + + return ret; +} +EXPORT_SYMBOL_GPL(iommu_attach_device_pasid); + +/* + * iommu_detach_device_pasid() - Detach the domain from pasid of device + * @domain: the iommu domain. + * @dev: the attached device. + * @pasid: the pasid of the device. + * + * The @domain must have been attached to @pasid of the @dev with + * iommu_attach_device_pasid(). + */ +void iommu_detach_device_pasid(struct iommu_domain *domain, struct device *dev, + ioasid_t pasid) +{ + struct iommu_group *group = iommu_group_get(dev); + + mutex_lock(&group->mutex); + __iommu_set_group_pasid(group->blocking_domain, group, pasid); + WARN_ON(xa_erase(&group->pasid_array, pasid) != domain); + mutex_unlock(&group->mutex); + + iommu_group_put(group); +} +EXPORT_SYMBOL_GPL(iommu_detach_device_pasid); + +/* + * iommu_get_domain_for_dev_pasid() - Retrieve domain for @pasid of @dev + * @dev: the queried device + * @pasid: the pasid of the device + * + * This is a variant of iommu_get_domain_for_dev(). It returns the existing + * domain attached to pasid of a device. It's only for internal use of the + * IOMMU subsystem. The caller must take care to avoid any possible + * use-after-free case. + * + * Return: attached domain on success, NULL otherwise. + */ +struct iommu_domain * +iommu_get_domain_for_dev_pasid(struct device *dev, ioasid_t pasid) +{ + struct iommu_domain *domain; + struct iommu_group *group; + + if (!pasid_valid(pasid)) + return NULL; + + group = iommu_group_get(dev); + if (!group) + return NULL; + /* + * The xarray protects its internal state with RCU. Hence the domain + * obtained is either NULL or fully formed. + */ + domain = xa_load(&group->pasid_array, pasid); + iommu_group_put(group); + + return domain; +} +EXPORT_SYMBOL_GPL(iommu_get_domain_for_dev_pasid); -- 2.25.1