Received-SPF: pass (google.com: domain of linux-kernel-owner@vger.kernel.org designates 2620:137:e000::1:20 as permitted sender) client-ip=2620:137:e000::1:20;
Message-ID: <42825be5-e24c-0f95-f49d-5f50d608506d@linux.intel.com>
Date:   Tue, 28 Jun 2022 13:41:28 +0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.9.1
Cc:     baolu.lu@linux.intel.com, Eric Auger <eric.auger@redhat.com>,
        "Liu, Yi L" <yi.l.liu@intel.com>,
        "Pan, Jacob jun" <jacob.jun.pan@intel.com>,
        "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org>,
        "iommu@lists.linux.dev" <iommu@lists.linux.dev>,
        "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
        Jean-Philippe Brucker <jean-philippe@linaro.org>
Subject: Re: [PATCH v9 04/11] iommu: Add sva iommu_domain support
Content-Language: en-US
To:     "Tian, Kevin" <kevin.tian@intel.com>,
        Joerg Roedel <joro@8bytes.org>,
        Jason Gunthorpe <jgg@nvidia.com>,
        Christoph Hellwig <hch@infradead.org>,
        "Raj, Ashok" <ashok.raj@intel.com>, Will Deacon <will@kernel.org>,
        Robin Murphy <robin.murphy@arm.com>,
        Jean-Philippe Brucker <jean-philippe@linaro.com>,
        "Jiang, Dave" <dave.jiang@intel.com>, Vinod Koul <vkoul@kernel.org>
References: <20220621144353.17547-1-baolu.lu@linux.intel.com>
 <20220621144353.17547-5-baolu.lu@linux.intel.com>
 <BN9PR11MB5276FA1A1A8C20786D958C048CB99@BN9PR11MB5276.namprd11.prod.outlook.com>
From:   Baolu Lu <baolu.lu@linux.intel.com>
In-Reply-To: <BN9PR11MB5276FA1A1A8C20786D958C048CB99@BN9PR11MB5276.namprd11.prod.outlook.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
Precedence: bulk

Hi Kevin,

On 2022/6/27 16:29, Tian, Kevin wrote:
>> From: Lu Baolu <baolu.lu@linux.intel.com>
>> Sent: Tuesday, June 21, 2022 10:44 PM
>>
>> The sva iommu_domain represents a hardware pagetable that the IOMMU
>> hardware could use for SVA translation. This adds some infrastructure
>> to support SVA domain in the iommu common layer. It includes:
>>
>> - Extend the iommu_domain to support a new IOMMU_DOMAIN_SVA
>> domain
>>    type. The IOMMU drivers that support SVA should provide the sva
>>    domain specific iommu_domain_ops.
>> - Add a helper to allocate an SVA domain. The iommu_domain_free()
>>    is still used to free an SVA domain.
>> - Add helpers to attach an SVA domain to a device and the reverse
>>    operation.
>>
>> Some buses, like PCI, route packets without considering the PASID value.
>> Thus a DMA target address with PASID might be treated as P2P if the
>> address falls into the MMIO BAR of other devices in the group. To make
>> things simple, the attach/detach interfaces only apply to devices
>> belonging to the singleton groups, and the singleton is immutable in
>> fabric i.e. not affected by hotplug.
>>
>> The iommu_attach/detach_device_pasid() can be used for other purposes,
>> such as kernel DMA with pasid, mediation device, etc.
> 
> I'd split this into two patches. One for adding iommu_attach/
> detach_device_pasid() and set/block_dev_pasid ops, and the
> other for adding SVA.

Yes. Make sense.

> 
>>   struct iommu_domain {
>>   	unsigned type;
>>   	const struct iommu_domain_ops *ops;
>>   	unsigned long pgsize_bitmap;	/* Bitmap of page sizes in use */
>> -	iommu_fault_handler_t handler;
>> -	void *handler_token;
>>   	struct iommu_domain_geometry geometry;
>>   	struct iommu_dma_cookie *iova_cookie;
>> +	union {
>> +		struct {	/* IOMMU_DOMAIN_DMA */
>> +			iommu_fault_handler_t handler;
>> +			void *handler_token;
>> +		};
> 
> why is it DMA domain specific? What about unmanaged
> domain? Unrecoverable fault can happen on any type
> including SVA. Hence I think above should be domain type
> agnostic.

The report_iommu_fault() should be replaced by the new
iommu_report_device_fault(). Jean has already started this work.

https://lore.kernel.org/linux-iommu/Yo4Nw9QyllT1RZbd@myrica/

Currently this is only for DMA domains, hence Robin suggested to make it
exclude with the SVA domain things.

https://lore.kernel.org/linux-iommu/f3170016-4d7f-e78e-db48-68305f683349@arm.com/

> 
>> +		struct {	/* IOMMU_DOMAIN_SVA */
>> +			struct mm_struct *mm;
>> +		};
>> +	};
>>   };
>>
> 
> 
> 
>> +
>> +struct iommu_domain *iommu_sva_domain_alloc(struct device *dev,
>> +					    struct mm_struct *mm)
>> +{
>> +	const struct iommu_ops *ops = dev_iommu_ops(dev);
>> +	struct iommu_domain *domain;
>> +
>> +	domain = ops->domain_alloc(IOMMU_DOMAIN_SVA);
>> +	if (!domain)
>> +		return NULL;
>> +
>> +	domain->type = IOMMU_DOMAIN_SVA;
> 
> It's a bit weird that the type has been specified when calling
> ops->domain_alloc while it still leaves to the caller to set the
> type. But this is not caused by this series. could be cleaned
> up separately.

Yes. Robin has patches to refactor the domain allocation interface,
let's wait and see what it looks like finally.

> 
>> +
>> +	mutex_lock(&group->mutex);
>> +	curr = xa_cmpxchg(&group->pasid_array, pasid, NULL, domain,
>> GFP_KERNEL);
>> +	if (curr)
>> +		goto out_unlock;
> 
> Need check xa_is_err(old).

Either

(1) old entry is a valid pointer, or
(2) xa_is_err(curr)

are failure cases. Hence, "curr == NULL" is the only check we need. Did
I miss anything?

Best regards,
baolu