Received: by 2002:a05:6358:11c7:b0:104:8066:f915 with SMTP id i7csp1732625rwl; Fri, 31 Mar 2023 16:09:50 -0700 (PDT) X-Google-Smtp-Source: AKy350bQxiLkqffM0J17nHnJRvpZBuZjpb49Bk7lYbMoATwIkFpgVU79HiJE1FBNkECk+g6w64G/ X-Received: by 2002:a17:902:c944:b0:1a0:53ba:ff1f with SMTP id i4-20020a170902c94400b001a053baff1fmr8153409pla.0.1680304190677; Fri, 31 Mar 2023 16:09:50 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680304190; cv=none; d=google.com; s=arc-20160816; b=zNjPU/8N5TSkaEvSj33rl7m+9nznm+IILhkt2JxZKsatYzDafsgPZpO70Z8U8TqBYo vZLPyn6wlM1ZRH5XED+qoPnQw50Qk0bx3c12vvgc49ymqyAyQd8UIJKtAL4Ce7hl4I7P OtKAKhWEzRB2xgnsxNH00B3064CipQBk42oqcGL4vCnwewVlZUElWQ4XWVKHBUaJlhgp 6c9KX0oIRyLTs03nub467P+WJuFqGdSAFhGsC5/dgus0DnTp1QyLktA9DUKL4jEvtNgO 7XxGuf8sjXO6uO6qYG3JyJuw7Izj+x1JkpIEP6Hp22S18h69KYausEvYC1fgbDj/iAYX Aqeg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :dkim-signature; bh=FxyuR9Y8tUn/mPb6zthKOhRc5pCPo854BP6xHwuSKN0=; b=M7Txm4LoOjK93tQb4EdmH4aH1b4aGsS5vf/77amE71Qg5EWYlhkZlfQBO/AkE7wS2F p3IeEAlWfjSHyHusxSxQha9aPtFGDLGpaC8jxjdX04ykDBhggfjuPoovvtDGtT1jznWc kLemai36NjmUfRuYaVuure3n+cyuSJp0fMVpOYZdIa+xu1BWN5eWWXCkDSz9GZVhXCqB prBeEMw0h/dXE4GeWRCaKRRs/UEjR8QLEYkZ8U3fAKOFalCQ3FKfABI7mjP/WIVfb4+y zyjusoB6CTgS31LQTQEHQvECS1HOJXeeM3/YESJDUleEfPq7Sj1sxBIgkeVHrcNR+PSk RXNg== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Zulfu2H4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id q11-20020a170902788b00b0018fae988814si3242544pll.167.2023.03.31.16.09.39; Fri, 31 Mar 2023 16:09:50 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=Zulfu2H4; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S233022AbjCaXH4 (ORCPT + 99 others); Fri, 31 Mar 2023 19:07:56 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:36080 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S233104AbjCaXHh (ORCPT ); Fri, 31 Mar 2023 19:07:37 -0400 Received: from mga09.intel.com (mga09.intel.com [134.134.136.24]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id E679E1A959; Fri, 31 Mar 2023 16:07:35 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1680304055; x=1711840055; h=from:to:cc:subject:date:message-id:in-reply-to: references:mime-version:content-transfer-encoding; bh=jxr3v2l61hZEreTEFWYJm0GwUqpLGEMRB3P581VOVLk=; b=Zulfu2H4JDocbgX/MYqi82Qd9e83ASoHz8ZjGj/pDgbDj2QXSUSlLEYD /aZPvL9LKiOzdvzfXrr3XREszAbF9vhQ39NXg6ZeYS09lEShy3XysIiRu KCMfpDdQKvdMGsiIXZjQZP3YYV7pWnR5K7EGtbzGMKgj8+FHmqkl4J4JO By9CwwBSI4QXtS3Si3HPMP/+SOKNX7cgZhNwe7DwErlbs2e6HAETEVCbx ql2ysJXpXMBCCZFBmo9xipP6j6hGOUMS0ngFBDzPoIf+2evOuaMKSe33V lK8Ojt7yqkZxKF29YkJ87g5b7VFYdfoaG8Ie4JWh+U6jkJ6PrWUCRQ/zz Q==; X-IronPort-AV: E=McAfee;i="6600,9927,10666"; a="343245167" X-IronPort-AV: E=Sophos;i="5.98,307,1673942400"; d="scan'208";a="343245167" Received: from orsmga001.jf.intel.com ([10.7.209.18]) by orsmga102.jf.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 31 Mar 2023 16:07:34 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10666"; a="717888355" X-IronPort-AV: E=Sophos;i="5.98,307,1673942400"; d="scan'208";a="717888355" Received: from srinivas-otcpl-7600.jf.intel.com (HELO jacob-builder.jf.intel.com) ([10.54.39.106]) by orsmga001.jf.intel.com with ESMTP; 31 Mar 2023 16:07:33 -0700 From: Jacob Pan To: LKML , iommu@lists.linux.dev, "Robin Murphy" , Jason Gunthorpe , "Lu Baolu" , Joerg Roedel , dmaengine@vger.kernel.org, vkoul@kernel.org Cc: "Will Deacon" , David Woodhouse , Raj Ashok , "Tian, Kevin" , Yi Liu , "Yu, Fenghua" , Dave Jiang , Tony Luck , "Zanussi, Tom" , Jacob Pan Subject: [PATCH v3 7/7] dmaengine/idxd: Re-enable kernel workqueue under DMA API Date: Fri, 31 Mar 2023 16:11:37 -0700 Message-Id: <20230331231137.1947675-8-jacob.jun.pan@linux.intel.com> X-Mailer: git-send-email 2.25.1 In-Reply-To: <20230331231137.1947675-1-jacob.jun.pan@linux.intel.com> References: <20230331231137.1947675-1-jacob.jun.pan@linux.intel.com> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-2.4 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,RCVD_IN_DNSWL_MED,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Kernel workqueues were disabled due to flawed use of kernel VA and SVA API. Now That we have the support for attaching PASID to the device's default domain and the ability to reserve global PASIDs from SVA APIs, we can re-enable the kernel work queues and use them under DMA API. We also use non-privileged access for in-kernel DMA to be consistent with the IOMMU settings. Consequently, interrupt for user privilege is enabled for work completion IRQs. Link:https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.com/ Reviewed-by: Dave Jiang Signed-off-by: Jacob Pan --- drivers/dma/idxd/device.c | 30 ++++----------------- drivers/dma/idxd/init.c | 56 ++++++++++++++++++++++++++++++++++++--- drivers/dma/idxd/sysfs.c | 7 ----- 3 files changed, 57 insertions(+), 36 deletions(-) diff --git a/drivers/dma/idxd/device.c b/drivers/dma/idxd/device.c index 6fca8fa8d3a8..f6b133d61a04 100644 --- a/drivers/dma/idxd/device.c +++ b/drivers/dma/idxd/device.c @@ -299,21 +299,6 @@ void idxd_wqs_unmap_portal(struct idxd_device *idxd) } } -static void __idxd_wq_set_priv_locked(struct idxd_wq *wq, int priv) -{ - struct idxd_device *idxd = wq->idxd; - union wqcfg wqcfg; - unsigned int offset; - - offset = WQCFG_OFFSET(idxd, wq->id, WQCFG_PRIVL_IDX); - spin_lock(&idxd->dev_lock); - wqcfg.bits[WQCFG_PRIVL_IDX] = ioread32(idxd->reg_base + offset); - wqcfg.priv = priv; - wq->wqcfg->bits[WQCFG_PRIVL_IDX] = wqcfg.bits[WQCFG_PRIVL_IDX]; - iowrite32(wqcfg.bits[WQCFG_PRIVL_IDX], idxd->reg_base + offset); - spin_unlock(&idxd->dev_lock); -} - static void __idxd_wq_set_pasid_locked(struct idxd_wq *wq, int pasid) { struct idxd_device *idxd = wq->idxd; @@ -1324,15 +1309,14 @@ int drv_enable_wq(struct idxd_wq *wq) } /* - * In the event that the WQ is configurable for pasid and priv bits. - * For kernel wq, the driver should setup the pasid, pasid_en, and priv bit. - * However, for non-kernel wq, the driver should only set the pasid_en bit for - * shared wq. A dedicated wq that is not 'kernel' type will configure pasid and + * In the event that the WQ is configurable for pasid, the driver + * should setup the pasid, pasid_en bit. This is true for both kernel + * and user shared workqueues. There is no need to setup priv bit in + * that in-kernel DMA will also do user privileged requests. + * A dedicated wq that is not 'kernel' type will configure pasid and * pasid_en later on so there is no need to setup. */ if (test_bit(IDXD_FLAG_CONFIGURABLE, &idxd->flags)) { - int priv = 0; - if (wq_pasid_enabled(wq)) { if (is_idxd_wq_kernel(wq) || wq_shared(wq)) { u32 pasid = wq_dedicated(wq) ? idxd->pasid : 0; @@ -1340,10 +1324,6 @@ int drv_enable_wq(struct idxd_wq *wq) __idxd_wq_set_pasid_locked(wq, pasid); } } - - if (is_idxd_wq_kernel(wq)) - priv = 1; - __idxd_wq_set_priv_locked(wq, priv); } rc = 0; diff --git a/drivers/dma/idxd/init.c b/drivers/dma/idxd/init.c index e6ee267da0ff..6f7778e1e936 100644 --- a/drivers/dma/idxd/init.c +++ b/drivers/dma/idxd/init.c @@ -506,14 +506,61 @@ static struct idxd_device *idxd_alloc(struct pci_dev *pdev, struct idxd_driver_d static int idxd_enable_system_pasid(struct idxd_device *idxd) { - return -EOPNOTSUPP; + struct pci_dev *pdev = idxd->pdev; + struct device *dev = &pdev->dev; + struct iommu_domain *domain; + union gencfg_reg gencfg; + ioasid_t pasid; + int ret; + + /* + * Attach a global PASID to the DMA domain so that we can use ENQCMDS + * to submit work on buffers mapped by DMA API. + */ + domain = iommu_get_domain_for_dev(dev); + if (!domain) + return -EPERM; + + pasid = iommu_alloc_global_pasid(0, dev->iommu->max_pasids); + if (!pasid_valid(pasid)) + return -ENOSPC; + + ret = iommu_attach_device_pasid(domain, dev, pasid); + if (ret) { + dev_err(dev, "failed to attach device pasid %d, domain type %d", + pasid, domain->type); + iommu_free_global_pasid(pasid); + return ret; + } + + /* Since we set user privilege for kernel DMA, enable completion IRQ */ + gencfg.bits = ioread32(idxd->reg_base + IDXD_GENCFG_OFFSET); + gencfg.user_int_en = 1; + iowrite32(gencfg.bits, idxd->reg_base + IDXD_GENCFG_OFFSET); + idxd->pasid = pasid; + + return ret; } static void idxd_disable_system_pasid(struct idxd_device *idxd) { + struct pci_dev *pdev = idxd->pdev; + struct device *dev = &pdev->dev; + struct iommu_domain *domain; + union gencfg_reg gencfg; + + domain = iommu_get_domain_for_dev(dev); + if (!domain || domain->type == IOMMU_DOMAIN_BLOCKED) + return; + + iommu_detach_device_pasid(domain, dev, idxd->pasid); + iommu_free_global_pasid(idxd->pasid); - iommu_sva_unbind_device(idxd->sva); + gencfg.bits = ioread32(idxd->reg_base + IDXD_GENCFG_OFFSET); + gencfg.user_int_en = 0; + iowrite32(gencfg.bits, idxd->reg_base + IDXD_GENCFG_OFFSET); idxd->sva = NULL; + idxd->pasid = IOMMU_PASID_INVALID; } static int idxd_probe(struct idxd_device *idxd) @@ -535,8 +582,9 @@ static int idxd_probe(struct idxd_device *idxd) } else { set_bit(IDXD_FLAG_USER_PASID_ENABLED, &idxd->flags); - if (idxd_enable_system_pasid(idxd)) - dev_warn(dev, "No in-kernel DMA with PASID.\n"); + rc = idxd_enable_system_pasid(idxd); + if (rc) + dev_warn(dev, "No in-kernel DMA with PASID. %d\n", rc); else set_bit(IDXD_FLAG_PASID_ENABLED, &idxd->flags); } diff --git a/drivers/dma/idxd/sysfs.c b/drivers/dma/idxd/sysfs.c index 18cd8151dee0..c5561c00a503 100644 --- a/drivers/dma/idxd/sysfs.c +++ b/drivers/dma/idxd/sysfs.c @@ -944,13 +944,6 @@ static ssize_t wq_name_store(struct device *dev, if (strlen(buf) > WQ_NAME_SIZE || strlen(buf) == 0) return -EINVAL; - /* - * This is temporarily placed here until we have SVM support for - * dmaengine. - */ - if (wq->type == IDXD_WQT_KERNEL && device_pasid_enabled(wq->idxd)) - return -EOPNOTSUPP; - input = kstrndup(buf, count, GFP_KERNEL); if (!input) return -ENOMEM; -- 2.25.1