Received: by 2002:a05:6358:11c7:b0:104:8066:f915 with SMTP id i7csp1543566rwl; Wed, 5 Apr 2023 19:52:57 -0700 (PDT) X-Google-Smtp-Source: AKy350bJawT4oEkHX045/UExttvQz0vqUIFnHP9z4uGvNxcSAV0OGf1HodM05PN9M2eb8Tq698K0 X-Received: by 2002:a17:903:32c3:b0:19e:b088:5900 with SMTP id i3-20020a17090332c300b0019eb0885900mr9981531plr.38.1680749577139; Wed, 05 Apr 2023 19:52:57 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1680749577; cv=none; d=google.com; s=arc-20160816; b=jcrLWr78FL4o41g+NtX8tjjcR7DTJHnPwm1iYWenNWhOXWkH64uBHo2AuufYuSrFjd yqFerb5B0hi6j4oLsuzWL/roGpfTIB8fYOlaHU7CHvWRzqGo3iEimqz/+rTRbjJcbK/D rYjnipC3gco2gpMVLYKrup6ENQqIhm6jqNjhRsIMrOXEj0ldTnpGgmkbDVlnarKg0zAA ik4GHZSxcCX53MLiACcNFQ31vVjerTkp3RYCehvCAs2nhAjzZhZlH/coaNylzuKqKbqA KhBE5ZfhpfE9A8Hmv5DLzh2ld+pCemYyB25Ldhc9+wlZIr5TgyUfc2j2AD2gf/aZ4h8/ LnoA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=list-id:precedence:content-transfer-encoding:in-reply-to:from :references:to:content-language:subject:cc:user-agent:mime-version :date:message-id:dkim-signature; bh=5G2T+aSASSNejpBY8G2QaiSeT0KZJIXl678sY5giK/Q=; b=o2aepG31aBLZGiPu9AaTceTo5lKtmVnoeEJGBKNvUJJadFB2bMiv5hC99hY7iAnyC5 i1l/Qg6Uu4xtiW4J8NOEwNFplHgbE0VfTYJppIP3uFOB2jrPKwwZ8cBmogYuI0YQVjeY DQweqzblEnqpHpZ4RgAVnDBlSBeFErgobu2/tFej7ICSJGFtJJkb89z23YVI1OUQDEly +9P91AwJpLz2+hS9zLdI+XufP0XfFvkXjPy0rV5rIxRpYkT+KeuodVIQMLsClU2pJ9mL /G4rWptFT+bZ1At4oPwDeHFlo5pBIpoIv4N7r30K1ndVet4AH3SzDgw2X6rHYnXQqzhJ nV7g== ARC-Authentication-Results: i=1; mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=lEkqMSbT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Return-Path: Received: from out1.vger.email (out1.vger.email. [2620:137:e000::1:20]) by mx.google.com with ESMTP id r17-20020a170902e3d100b0019cf7f260adsi546892ple.173.2023.04.05.19.52.45; Wed, 05 Apr 2023 19:52:57 -0700 (PDT) Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20; Authentication-Results: mx.google.com; dkim=pass header.i=@intel.com header.s=Intel header.b=lEkqMSbT; spf=pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) smtp.mailfrom=linux-kernel-owner@vger.kernel.org; dmarc=pass (p=NONE sp=NONE dis=NONE) header.from=intel.com Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S235154AbjDFCux (ORCPT + 99 others); Wed, 5 Apr 2023 22:50:53 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:39900 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S235119AbjDFCuY (ORCPT ); Wed, 5 Apr 2023 22:50:24 -0400 Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 0D27F9009; Wed, 5 Apr 2023 19:49:45 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=intel.com; i=@intel.com; q=dns/txt; s=Intel; t=1680749385; x=1712285385; h=message-id:date:mime-version:cc:subject:to:references: from:in-reply-to:content-transfer-encoding; bh=wNLHlYy6uV4G115SGXQXDrIz1CoHj/OQl8UlKp5dXLs=; b=lEkqMSbTxooxglfJ8+0E9V84q+3C0Ip3/mHcIleCPVEq1kGlwX6ZYV70 joo9cZ//RiAtYOd1jI4kJmB6Cl+exBGaIpL+OGBqg77ou4gYWb0B4p65+ XN0ba6s+4n4bdzG2p3aVKwRfzYHB5xoPugzkqIeOSEb9pZ4ODAukqPqDV SjPPVk9byVNhICDkMEeP1QOqqYUouIwy5bbzPheL92tGSpqjAaqOJFglq +9gOIZnRrEyL9zffeXbDnvtsSj4M0YP0jM9kVxOvBD2Xxn0hitmBHltdI MqovND55rnh3paZvVzNX8jZ1VeqjI4TDMkC3ckLknTeY8R+bpjflIMe3f w==; X-IronPort-AV: E=McAfee;i="6600,9927,10671"; a="370457043" X-IronPort-AV: E=Sophos;i="5.98,322,1673942400"; d="scan'208";a="370457043" Received: from fmsmga005.fm.intel.com ([10.253.24.32]) by fmsmga101.fm.intel.com with ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 05 Apr 2023 19:49:44 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=McAfee;i="6600,9927,10671"; a="1016701974" X-IronPort-AV: E=Sophos;i="5.98,322,1673942400"; d="scan'208";a="1016701974" Received: from allen-box.sh.intel.com (HELO [10.239.159.48]) ([10.239.159.48]) by fmsmga005.fm.intel.com with ESMTP; 05 Apr 2023 19:49:40 -0700 Message-ID: <7312331a-71a1-deea-6d17-bd6d968e87d4@linux.intel.com> Date: Thu, 6 Apr 2023 10:49:50 +0800 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101 Thunderbird/102.9.0 Cc: baolu.lu@linux.intel.com, LKML , iommu@lists.linux.dev, Robin Murphy , Joerg Roedel , dmaengine@vger.kernel.org, vkoul@kernel.org, Will Deacon , David Woodhouse , Raj Ashok , "Tian, Kevin" , Yi Liu , "Yu, Fenghua" , Dave Jiang , Tony Luck , "Zanussi, Tom" Subject: Re: [PATCH v3 7/7] dmaengine/idxd: Re-enable kernel workqueue under DMA API Content-Language: en-US To: Jason Gunthorpe , Jacob Pan References: <20230331231137.1947675-1-jacob.jun.pan@linux.intel.com> <20230331231137.1947675-8-jacob.jun.pan@linux.intel.com> From: Baolu Lu In-Reply-To: Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Status: No, score=-3.8 required=5.0 tests=DKIMWL_WL_HIGH,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,NICE_REPLY_A,RCVD_IN_DNSWL_MED, RCVD_IN_MSPIKE_H3,RCVD_IN_MSPIKE_WL,SPF_HELO_NONE,SPF_NONE autolearn=unavailable autolearn_force=no version=3.4.6 X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on lindbergh.monkeyblade.net Precedence: bulk List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On 4/5/23 8:15 PM, Jason Gunthorpe wrote: > On Fri, Mar 31, 2023 at 04:11:37PM -0700, Jacob Pan wrote: >> static void idxd_disable_system_pasid(struct idxd_device *idxd) >> { >> + struct pci_dev *pdev = idxd->pdev; >> + struct device *dev = &pdev->dev; >> + struct iommu_domain *domain; >> + union gencfg_reg gencfg; >> + >> + domain = iommu_get_domain_for_dev(dev); >> + if (!domain || domain->type == IOMMU_DOMAIN_BLOCKED) >> + return; >> + >> + iommu_detach_device_pasid(domain, dev, idxd->pasid); > This sequence is kinda weird, we shouldn't pass in domain to > detach_device_pasid, IMHO. We already know the domain because we store > it in an xarray, it just creates weirdness if the user passes in the > wrong domain. The initial idea was that the driver has a domain and it wants to attach the domain to a pasid of the device. During attaching, iommu core will save the domain in its xarray. After use, driver want to detach the domain from the pasid by calling iommu_detach_device_pasid(). The iommu core will compare the input domain and the one it saved. A warning will be triggered if they are different. WARN_ON(xa_erase(&group->pasid_array, pasid) != domain); Logically speaking, @domain for detach_device_pasid is unnecessary, because the pasid array is essentially per-device (as we discussed before. the pci_enable_pasid() ensures this), although it is currently placed in the group structure. In that case, the driver can and should own everything about the pasid and domain. The roles of the iommu core and the individual driver are only to handle requests of installing or withdrawing a domain on/from a device's pasid. Best regards, baolu