Received: by 2002:a05:6358:3188:b0:123:57c1:9b43 with SMTP id q8csp4671780rwd; Tue, 23 May 2023 10:43:40 -0700 (PDT) X-Google-Smtp-Source: ACHHUZ4r7h1bexXuCiGEjD96cid+TvUddDfGIZ5VHiMbEZETGw9SFmhVRFnyJu83z4OzigB1AYei X-Received: by 2002:a17:90a:a092:b0:255:ae09:1f1f with SMTP id r18-20020a17090aa09200b00255ae091f1fmr2386789pjp.21.1684863820219; Tue, 23 May 2023 10:43:40 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1684863820; cv=none; d=google.com; s=arc-20160816; b=oYe9fTnFkt1qxVRg7kG2CD6xtNTLPV7S2huAE+q0E+9LoO36JovzIUojrUeKo1HcUQ H8/tHvPoVVd33SptLtv2FNVrwXrJPLC7r2erPmnV5BmS9fvU5G0EvID27i3PYl8LGSAP Kj3gB5tu4gTfi2ZlQiCBn8cDNmFH1kT84nIu/HJ7vj5BoQT+sFSJhxDOB1Eu2fN5R6U0 h1nFYs3qTZrxv67zuKYDio+4X00MBa7oiuV7S5SS7dzTu32sbFTJrEd17+yneMsBVurK Y3yxWs0DOHWjT+3k+pu4c/tph9AQj5U46iAjxhsVT6zB/yd72VsiocbOxiwVnVXf738f mCnw== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=SkSs33qPaSX2Sbm7TerQvYbFQUA+GSKh9/aG7QBbc4c=; b=PS/4ii4RAwY/EOyiM7aBlsjU4ZEEKWygG9uPbtmD9KEioaImot3rOLJLy7GsZeiZEg 2gUMAq4YDmhUCIg7zKaEXSuGVLslf8C5b+bqJIWeoHAypXEM0d6tPKgyx8iTAQ+2aUPe e3cy4ZcXGZN/wEaphESrU6c4pko4Iwi1cW/bAFaHB2BF594PqyU2FnghlHgaO2eK3emS 2lIfM8i+IAGEqaP8+0xLpN/g3pWsfllanz0V18k9ACe4ciKgG/ij2n12V3eJRZEBpaCH JtXF+dSn6ttvulWnPyP/bwq5DpnWZSamqmSfmDvjb/SHc3iTTg96biI0V6EjALzQX6Gw vTvw== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=UI5NeBep; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id bb18-20020a17090b009200b002534549dac8si8926779pjb.38.2023.05.23.10.43.26; Tue, 23 May 2023 10:43:40 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=UI5NeBep; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S238222AbjEWRap (ORCPT + 99 others); Tue, 23 May 2023 13:30:45 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:32818 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S238244AbjEWRac (ORCPT ); Tue, 23 May 2023 13:30:32 -0400 Received: from mga18.intel.com (mga18.intel.com [134.134.136.126]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 296CD129; Tue, 23 May 2023 10:30:25 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1684863025; x=1716399025; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=fj/9L9Yv5iMRlM4PmVWEzhn4LRc1g/zAg1PPYUQN4Hg=; b=UI5NeBepsrU0B+ABzwQytmTy2wgwrx9SJ1UGQFwS5K3M7mudteExbv3x vHGBzf7awD98t0+lubIDmW8eB0gbBz6r3i1DFWCbGueaTHHWlxvh5vFB5 0CUkiRxA3TkaUBn1JAdQtA4pXVUBW+iCz/oKeY9Gmy5szJKg4E8nivpZs McBPRybUXZA0nyAQ6lb+9X2YQbEetRVYpX8KXqPe5tDMPUstuO3fZ9RLO OchhMV1J42tmKZ7sszUtZ/37GBgXYFwQ2Xp5sTFV+as5tDBJP4bruSKz/ o78UzxRkaolLpWO6tOGyAS/jyYVWK6Pe4mN2ASJ6rUDqnDRkzTqwGePih w==; X-IronPort-AV: E=McAfee;i="6600,9927,10719"; a="337892837" X-IronPort-AV: E=Sophos;i="6.00,187,1681196400"; d="scan'208";a="337892837" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by orsmga106.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 23 May 2023 10:30:24 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10719"; a="1034170033" X-IronPort-AV: E=Sophos;i="6.00,187,1681196400"; d="scan'208";a="1034170033" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.97.184]) by fmsmga005.fm.intel.com with ESMTP; 23 May 2023 10:30:23 -0700 From: Jacob Pan To: LKML , iommu@lists.linux.dev, Jason Gunthorpe , "Lu Baolu" , Joerg Roedel , "Robin Murphy" , Jean-Philippe Brucker , dmaengine@vger.kernel.org, vkoul@kernel.org Cc: "Will Deacon" , David Woodhouse , Raj Ashok , "Tian, Kevin" , Yi Liu , "Yu, Fenghua" , Dave Jiang , Tony Luck , "Zanussi, Tom" , narayan.ranganathan@intel.com, Jacob Pan Subject: [PATCH v7 4/4] dmaengine/idxd: Re-enable kernel workqueue under DMA API Date: Tue, 23 May 2023 10:34:51 -0700 Message-Id: <20230523173451.2932113-5-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230523173451.2932113-1-jacob.jun.pan@linux.intel.com> References: <20230523173451.2932113-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.0 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH, DKIM_SIGNED,DKIM_VALID,DKIM_VALID_EF,SPF_HELO_NONE,SPF_NONE, T_SCC_BODY_TEXT_LINE,URIBL_BLOCKED autolearn=ham autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Kernel workqueues were disabled due to flawed use of kernel VA and SVA API. Now that we have the support for attaching PASID to the device's default domain and the ability to reserve global PASIDs from SVA APIs, we can re-enable the kernel work queues and use them under DMA API. We also use non-privileged access for in-kernel DMA to be consistent with the IOMMU settings. Consequently, interrupt for user privilege is enabled for work completion IRQs. Link:https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.com/ Reviewed-by: Dave Jiang Reviewed-by: Fenghua Yu Reviewed-by: Lu Baolu Signed-off-by: Jacob Pan --- drivers/dma/idxd/device.c | 30 ++++---------------- drivers/dma/idxd/dma.c | 5 ++-- drivers/dma/idxd/init.c | 60 ++++++++++++++++++++++++++++++++++++--- drivers/dma/idxd/sysfs.c | 7 ----- 4 files changed, 64 insertions(+), 38 deletions(-) diff --git a/drivers/dma/idxd/device.c b/drivers/dma/idxd/device.c index 5abbcc61c528..66b6665a45cb 100644 --- a/drivers/dma/idxd/device.c +++ b/drivers/dma/idxd/device.c @@ -299,21 +299,6 @@ void idxd_wqs_unmap_portal(struct idxd_device *idxd) } } -static void __idxd_wq_set_priv_locked(struct idxd_wq *wq, int priv) -{ - struct idxd_device *idxd = wq->idxd; - union wqcfg wqcfg; - unsigned int offset; - - offset = WQCFG_OFFSET(idxd, wq->id, WQCFG_PRIVL_IDX); - spin_lock(&idxd->dev_lock); - wqcfg.bits[WQCFG_PRIVL_IDX] = ioread32(idxd->reg_base + offset); - wqcfg.priv = priv; - wq->wqcfg->bits[WQCFG_PRIVL_IDX] = wqcfg.bits[WQCFG_PRIVL_IDX]; - iowrite32(wqcfg.bits[WQCFG_PRIVL_IDX], idxd->reg_base + offset); - spin_unlock(&idxd->dev_lock); -} - static void __idxd_wq_set_pasid_locked(struct idxd_wq *wq, int pasid) { struct idxd_device *idxd = wq->idxd; @@ -1423,15 +1408,14 @@ int drv_enable_wq(struct idxd_wq *wq) } /* - * In the event that the WQ is configurable for pasid and priv bits. - * For kernel wq, the driver should setup the pasid, pasid_en, and priv bit. - * However, for non-kernel wq, the driver should only set the pasid_en bit for - * shared wq. A dedicated wq that is not 'kernel' type will configure pasid and + * In the event that the WQ is configurable for pasid, the driver + * should setup the pasid, pasid_en bit. This is true for both kernel + * and user shared workqueues. There is no need to setup priv bit in + * that in-kernel DMA will also do user privileged requests. + * A dedicated wq that is not 'kernel' type will configure pasid and * pasid_en later on so there is no need to setup. */ if (test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags)) { - int priv = 0; - if (wq_pasid_enabled(wq)) { if (is_idxd_wq_kernel(wq) || wq_shared(wq)) { u32 pasid = wq_dedicated(wq) ? idxd->pasid : 0; @@ -1439,10 +1423,6 @@ int drv_enable_wq(struct idxd_wq *wq) __idxd_wq_set_pasid_locked(wq, pasid); } } - - if (is_idxd_wq_kernel(wq)) - priv = 1; - __idxd_wq_set_priv_locked(wq, priv); } rc = 0; diff --git a/drivers/dma/idxd/dma.c b/drivers/dma/idxd/dma.c index eb35ca313684..07623fb0f52f 100644 --- a/drivers/dma/idxd/dma.c +++ b/drivers/dma/idxd/dma.c @@ -75,9 +75,10 @@ static inline void idxd_prep_desc_common(struct idxd_wq *wq, hw->xfer_size = len; /* * For dedicated WQ, this field is ignored and HW will use the WQCFG.priv - * field instead. This field should be set to 1 for kernel descriptors. + * field instead. This field should be set to 0 for kernel descriptors + * since kernel DMA on VT-d supports "user" privilege only. */ - hw->priv = 1; + hw->priv = 0; hw->completion_addr = compl; } diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index 1aa823974cda..bd7b9bd40f0a 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -550,14 +550,65 @@ static struct idxd_device *idxd_alloc(struct pci_dev *pdev, struct idxd_driver_d static int idxd_enable_system_pasid(struct idxd_device *idxd) { - return -EOPNOTSUPP; + struct pci_dev *pdev = idxd->pdev; + struct device *dev = &pdev->dev; + struct iommu_domain *domain; + union gencfg_reg gencfg; + ioasid_t pasid; + int ret; + + /* + * Attach a global PASID to the DMA domain so that we can use ENQCMDS + * to submit work on buffers mapped by DMA API. + */ + domain = iommu_get_domain_for_dev(dev); + if (!domain) + return -EPERM; + + pasid = iommu_alloc_global_pasid_dev(dev); + if (pasid == IOMMU_PASID_INVALID) + return -ENOSPC; + + /* + * DMA domain is owned by the driver, it should support all valid + * types such as DMA-FQ, identity, etc. + */ + ret = iommu_attach_device_pasid(domain, dev, pasid); + if (ret) { + dev_err(dev, "failed to attach device pasid %d, domain type %d", + pasid, domain->type); + iommu_free_global_pasid(pasid); + return ret; + } + + /* Since we set user privilege for kernel DMA, enable completion IRQ */ + gencfg.bits = ioread32(idxd->reg_base + IDXD_GENCFG_OFFSET); + gencfg.user_int_en = 1; + iowrite32(gencfg.bits, idxd->reg_base + IDXD_GENCFG_OFFSET); + idxd->pasid = pasid; + + return ret; } static void idxd_disable_system_pasid(struct idxd_device *idxd) { + struct pci_dev *pdev = idxd->pdev; + struct device *dev = &pdev->dev; + struct iommu_domain *domain; + union gencfg_reg gencfg; + + domain = iommu_get_domain_for_dev(dev); + if (!domain) + return; + + iommu_detach_device_pasid(domain, dev, idxd->pasid); + iommu_free_global_pasid(idxd->pasid); - iommu_sva_unbind_device(idxd->sva); + gencfg.bits = ioread32(idxd->reg_base + IDXD_GENCFG_OFFSET); + gencfg.user_int_en = 0; + iowrite32(gencfg.bits, idxd->reg_base + IDXD_GENCFG_OFFSET); idxd->sva = NULL; + idxd->pasid = IOMMU_PASID_INVALID; } static int idxd_enable_sva(struct pci_dev *pdev) @@ -600,8 +651,9 @@ static int idxd_probe(struct idxd_device *idxd) } else { set_bit(IDXD_FLAG_USER_PASID_ENABLED, &idxd->flags); - if (idxd_enable_system_pasid(idxd)) - dev_warn(dev, "No in-kernel DMA with PASID.\n"); + rc = idxd_enable_system_pasid(idxd); + if (rc) + dev_warn(dev, "No in-kernel DMA with PASID. %d\n", rc); else set_bit(IDXD_FLAG_PASID_ENABLED, &idxd->flags); } diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c index 293739ac5596..63f6966c51aa 100644 --- a/drivers/dma/idxd/sysfs.c +++ b/drivers/dma/idxd/sysfs.c @@ -948,13 +948,6 @@ static ssize_t wq_name_store(struct device *dev, if (strlen(buf) > WQ_NAME_SIZE || strlen(buf) == 0) return -EINVAL; - /* - * This is temporarily placed here until we have SVM support for - * dmaengine. - */ - if (wq->type == IDXD_WQT_KERNEL && device_pasid_enabled(wq->idxd)) - return -EOPNOTSUPP; - input = kstrndup(buf, count, GFP_KERNEL); if (!input) return -ENOMEM; -- 2.25.1