Received-SPF: softfail (google.com: domain of transitioning linux-kernel-owner@vger.kernel.org does not designate 23.128.96.19 as permitted sender) client-ip=23.128.96.19;
Date:   Mon, 2 May 2022 10:17:47 -0300
From:   Jason Gunthorpe <jgg@nvidia.com>
To:     Lu Baolu <baolu.lu@linux.intel.com>
Cc:     Joerg Roedel <joro@8bytes.org>,
        Alex Williamson <alex.williamson@redhat.com>,
        Kevin Tian <kevin.tian@intel.com>,
        Jacob jun Pan <jacob.jun.pan@intel.com>,
        Liu Yi L <yi.l.liu@intel.com>,
        iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH 3/5] iommu/vt-d: Check domain force_snooping against
 attached devices
Message-ID: <20220502131747.GJ8364@nvidia.com>
References: <20220501112434.874236-1-baolu.lu@linux.intel.com>
 <20220501112434.874236-4-baolu.lu@linux.intel.com>
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <20220501112434.874236-4-baolu.lu@linux.intel.com>
MIME-Version: 1.0
X-MS-Exchange-AntiSpam-MessageData-ChunkCount: 1
X-MS-Exchange-AntiSpam-MessageData-0: =?us-ascii?Q?p8CHKvlnBeU09sijha3BOZHDUoi98kwXruqEGxLUXAfVyIGbKLRrI3MHjwiv?=
 =?us-ascii?Q?O2fmjd+8Aig5vkoW02AeDS7wPNVFHjo+/PudVmStDa0elPdTJlErc+Y9Z/FO?=
 =?us-ascii?Q?1j6dA9tFKtlnADaUCu8TVr36qYmZzk0NZiIaKZ4nz01pyvg07ZfALBKqV1R3?=
 =?us-ascii?Q?G3AGvWLt/pZm60K9wZiDi5yrSXXeCLt03bMVz0cEUoOygg19H/bmQqBVCCzH?=
 =?us-ascii?Q?yctiRkdpLTZG2PCwe6+wELZfM4cZN/O47jBhEH1FsSNNtUlNswWxh2sVNM6i?=
 =?us-ascii?Q?d7cfm10Pfo5Wy3otjvlnSQlUbHx+Bpf2OOVV+PuH5zff5IdP+hI02vXG0TaS?=
 =?us-ascii?Q?W/hvxe2rxECYwscxj8dujlh1S2aOoj91wgcePG4qxMAGHfBfNVcLceWmkvkF?=
 =?us-ascii?Q?1n5LTlRUxcAbQNk682cB4XfNlBljyVdtOsvfHAncyNmeiZfrpsLl++w33KHa?=
 =?us-ascii?Q?HjyPh9zkfgA6x7Bk21efubQKpqps2O3KOgT4Wr45EpWrDPXm9q98rEXigCv+?=
 =?us-ascii?Q?Z8/0hvwyJxokCQtVwFzVMpHoEyfnTzLJiTkUfVrAh4nJ5uU6kTFqlwJzVGM7?=
 =?us-ascii?Q?8OAqk9omIRaUlZng7DadMZnP8I9jBzekXMYkAnGf4J6Hp/tEUAomt59nO16m?=
 =?us-ascii?Q?TQzFhfUw0RcLSqoBhR99Cu5K5NSTHwLPkBEyQRzat1m3fe04nZsnoSh24HY/?=
 =?us-ascii?Q?qqbjMb6fAsBR91N8MeSu2iEuhf80Dk8tV+dUp18TEmlvTbnHPB0TmNtYSzBn?=
 =?us-ascii?Q?TgweXdB8oJ0XJsc2a55UKN1wswpHR1k0MmxVxsszSTnU1cDTs7C/dDyZnfMN?=
 =?us-ascii?Q?sypxuW0rjDFJxnZs1wIeD8OBoqPfOeR6lBiqHWu76gDDH+CPacbBxILi4YbB?=
 =?us-ascii?Q?Za8pNrhnO4fcB2xfij0gDScG90axCO3e4gxyoTxta+ksy3aKkONehXU5ai4F?=
 =?us-ascii?Q?sKzMLOBhdGpEp1K8VMqFeFNHb0Je3S1f8dO79NkE0Jny9QuW2jfIXRt5RYgS?=
 =?us-ascii?Q?S3ISHvQYkMRRSozOj9767qwm6U1qK7eMHbznXLzrvuyR4cKqrV3zocVN0gqQ?=
 =?us-ascii?Q?tLtMAOqQVX5MO8q+Ig98hrldcBGbS9pWxwXk2uNOvwku6UqOn/MbjkBzA34r?=
 =?us-ascii?Q?6Y3pHAE3W31kZ46CkBvTT/4CWVyIDlvdJruezqSWqU4qsCW1qjpRdcKmd6sp?=
 =?us-ascii?Q?ibHWa9kWOmNPK23YfMzyI+RltalW1JxcsH1BK1X5oSoReHxlr8F0YElnjAe5?=
 =?us-ascii?Q?zaljLc0GOVo6ZwBL9ybov8/4PKnDw6JVx5ecSSVStTt7KQYe0l88FDlejgaO?=
 =?us-ascii?Q?4OckfHEagnipgkgvzv4FsEvOOybo091rNr4y/UsuLET3eCEOGQ4VqSmPLEBc?=
 =?us-ascii?Q?Abob7taF24WaKGHvl6OnpYB6fzW4raw+8qetO1lGLjYqi81zfA3z78umK5Zk?=
 =?us-ascii?Q?JYVOq8ErskJmSfg6Ht449+Uuffs8FMlnBX/gQUPWsR6fRyBxJbByaTtiOqEG?=
 =?us-ascii?Q?rY6N87kVAyiGo0CW8w40ObBbEh9Rxp1090TJaQHCvkT1gsCmFmKXzgFmMS82?=
 =?us-ascii?Q?kVlv5KDu8kz/fuhscNWvF5nYVWdXZQIqg3B+KPGCRn3CGsrTTaOGfRJDYt0b?=
 =?us-ascii?Q?tbsjygzJvyJYArWETlUefLLeMRNiHbFadzoYKe+pDuCDT9UtyYYERRZGjBT0?=
 =?us-ascii?Q?KuBKzv29xDJ4p401TGqQUzdFV0JZWN0WsdTJbLj7z+/xGZQ6FkY/AAoEC7EG?=
 =?us-ascii?Q?Knzas90YSg=3D=3D?=
X-OriginatorOrg: Nvidia.com
X-MS-Exchange-CrossTenant-Network-Message-Id: 1e1ffe83-f0e5-4c7d-4d7a-08da2c3e274c
X-MS-Exchange-CrossTenant-AuthSource: MN2PR12MB4192.namprd12.prod.outlook.com
X-MS-Exchange-CrossTenant-AuthAs: Internal
X-MS-Exchange-CrossTenant-OriginalArrivalTime: 02 May 2022 13:17:48.6502
 (UTC)
X-MS-Exchange-CrossTenant-FromEntityHeader: Hosted
X-MS-Exchange-CrossTenant-Id: 43083d15-7273-40c1-b7db-39efd9ccc17a
X-MS-Exchange-CrossTenant-MailboxType: HOSTED
X-MS-Exchange-CrossTenant-UserPrincipalName: eDlMqMVKaw3OFQTuqVQBCCkXCmDh6uus4mNyq9YiAUffIIiPhc9KjBnYiDpB0ufY
X-MS-Exchange-Transport-CrossTenantHeadersStamped: CH2PR12MB3798
X-Spam-Status: No, score=-2.6 required=5.0 tests=BAYES_00,DKIMWL_WL_HIGH,
	DKIM_SIGNED,DKIM_VALID,DKIM_VALID_AU,HEADER_FROM_DIFFERENT_DOMAINS,
	MAILING_LIST_MULTI,RDNS_NONE,SPF_HELO_NONE,T_SCC_BODY_TEXT_LINE
	autolearn=unavailable autolearn_force=no version=3.4.6
X-Spam-Checker-Version: SpamAssassin 3.4.6 (2021-04-09) on
	lindbergh.monkeyblade.net
Precedence: bulk
List-ID: <linux-kernel.vger.kernel.org>
X-Mailing-List: linux-kernel@vger.kernel.org

On Sun, May 01, 2022 at 07:24:32PM +0800, Lu Baolu wrote:
> +static bool domain_support_force_snooping(struct dmar_domain *domain)
> +{
> +	struct device_domain_info *info;
> +	unsigned long flags;
> +	bool support = true;
> +
> +	spin_lock_irqsave(&device_domain_lock, flags);
> +	if (list_empty(&domain->devices))
> +		goto out;

Why? list_for_each_entry will just do nothing..

> +	list_for_each_entry(info, &domain->devices, link) {
> +		if (!ecap_sc_support(info->iommu->ecap)) {
> +			support = false;
> +			break;
> +		}
> +	}
> +out:
> +	spin_unlock_irqrestore(&device_domain_lock, flags);
> +	return support;
> +}
> +
> +static void domain_set_force_snooping(struct dmar_domain *domain)
> +{
> +	struct device_domain_info *info;
> +	unsigned long flags;
> +
> +	/*
> +	 * Second level page table supports per-PTE snoop control. The
> +	 * iommu_map() interface will handle this by setting SNP bit.
> +	 */
> +	if (!domain_use_first_level(domain))
> +		return;
> +
> +	spin_lock_irqsave(&device_domain_lock, flags);
> +	if (list_empty(&domain->devices))
> +		goto out_unlock;
> +
> +	list_for_each_entry(info, &domain->devices, link)
> +		intel_pasid_setup_page_snoop_control(info->iommu, info->dev,
> +						     PASID_RID2PASID);
> +
> +out_unlock:
> +	spin_unlock_irqrestore(&device_domain_lock, flags);
> +}
> +
>  static bool intel_iommu_enforce_cache_coherency(struct iommu_domain *domain)
>  {
>  	struct dmar_domain *dmar_domain = to_dmar_domain(domain);
>  
> -	if (!domain_update_iommu_snooping(NULL))
> +	if (!domain_support_force_snooping(dmar_domain))
>  		return false;

Maybe exit early if force_snooping = true?

> +	domain_set_force_snooping(dmar_domain);
>  	dmar_domain->force_snooping = true;
> +
>  	return true;
>  }
>  
> diff --git a/drivers/iommu/intel/pasid.c b/drivers/iommu/intel/pasid.c
> index f8d215d85695..815c744e6a34 100644
> +++ b/drivers/iommu/intel/pasid.c
> @@ -762,3 +762,21 @@ int intel_pasid_setup_pass_through(struct intel_iommu *iommu,
>  
>  	return 0;
>  }
> +
> +/*
> + * Set the page snoop control for a pasid entry which has been set up.
> + */

So the 'first level' is only used with pasid?

> +void intel_pasid_setup_page_snoop_control(struct intel_iommu *iommu,
> +					  struct device *dev, u32 pasid)
> +{
> +	struct pasid_entry *pte;
> +	u16 did;
> +
> +	pte = intel_pasid_get_entry(dev, pasid);
> +	if (WARN_ON(!pte || !pasid_pte_is_present(pte)))
> +		return;
> +
> +	pasid_set_pgsnp(pte);

Doesn't this need to be done in other places too, like when a new attach
is made? Patch 5 removed it, but should that be made if
domain->force_snooping?

Jason