Received-SPF: pass (google.com: domain of linux-kernel+bounces-70823-linux.lists.archive=gmail.com@vger.kernel.org designates 139.178.88.99 as permitted sender) client-ip=139.178.88.99;
Message-ID: <a0b16fc8-4812-4126-8dab-445aa974307a@linux.intel.com>
Date: Mon, 19 Feb 2024 15:24:35 +0800
Precedence: bulk
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: [PATCH 1/2] iommu/vt-d: Use rbtree to track iommu probed devices
To: Baolu Lu <baolu.lu@linux.intel.com>, Joerg Roedel <joro@8bytes.org>,
 Will Deacon <will@kernel.org>, Robin Murphy <robin.murphy@arm.com>,
 Jason Gunthorpe <jgg@ziepe.ca>, Kevin Tian <kevin.tian@intel.com>
Cc: Huang Jiaqing <jiaqing.huang@intel.com>, iommu@lists.linux.dev,
 linux-kernel@vger.kernel.org
References: <20240215072249.4465-1-baolu.lu@linux.intel.com>
 <20240215072249.4465-2-baolu.lu@linux.intel.com>
 <eace2ec0-0b8b-450d-b05f-7b7ca3e473a7@linux.intel.com>
 <942dd645-7475-4c43-99c6-304601fb09eb@linux.intel.com>
 <ba137a34-68d1-4f61-a6dd-30841610c079@linux.intel.com>
 <11a2a48f-aad9-42b0-a9a1-29ea1c0f7f45@linux.intel.com>
From: Ethan Zhao <haifeng.zhao@linux.intel.com>
In-Reply-To: <11a2a48f-aad9-42b0-a9a1-29ea1c0f7f45@linux.intel.com>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 2/19/2024 2:47 PM, Baolu Lu wrote:
> On 2024/2/19 13:33, Ethan Zhao wrote:
>> On 2/19/2024 12:04 PM, Baolu Lu wrote:
>>> On 2024/2/19 10:45, Ethan Zhao wrote:
>>>>> @@ -4264,25 +4313,34 @@ static struct iommu_device 
>>>>> *intel_iommu_probe_device(struct device *dev)
>>>>>       }
>>>>>       dev_iommu_priv_set(dev, info);
>>>>> +    ret = device_rbtree_insert(iommu, info);
>>>>> +    if (ret)
>>>>> +        goto free;
>>>>>       if (sm_supported(iommu) && !dev_is_real_dma_subdevice(dev)) {
>>>>>           ret = intel_pasid_alloc_table(dev);
>>>>>           if (ret) {
>>>>>               dev_err(dev, "PASID table allocation failed\n");
>>>>> -            kfree(info);
>>>>> -            return ERR_PTR(ret);
>>>>> +            goto clear_rbtree;
>>>>>           }
>>>>>       }
>>>>>       intel_iommu_debugfs_create_dev(info);
>>>>>       return &iommu->iommu;
>>>>> +clear_rbtree:
>>>>> +    device_rbtree_remove(info);
>>>>> +free:
>>>>> +    kfree(info);
>>>>> +
>>>>> +    return ERR_PTR(ret);
>>>>>   }
>>>>>   static void intel_iommu_release_device(struct device *dev)
>>>>>   {
>>>>>       struct device_domain_info *info = dev_iommu_priv_get(dev);
>>>>> +    device_rbtree_remove(info);
>>>>
>>>> Perhpas too early here to remove dev from the rbtree, if it is 
>>>> wanted in
>>>> devTLB invalidation steps in intel_pasid_tear_down_entry().
>>>
>>> Perhaps the caller of device_rbtree_find() should not depend on the
>>
>> I didn't catch up here. seems have to maintain the lifecycle as PCI 
>> subsystem
>> does, or there would be mutli instances for the same BDF(e.g. the 
>> device is
>> removed then plugged, again and again.....in the same slot) in the 
>> rbtree ?
>
> There should not be multiple instances for a same BDF. The lifecycle of
> a device is managed by the device and driver core. The iommu subsystem
> registers a notification to the core and take actions on device ADD and
> REMOVE events.

Move device_rbtree_remove(info) to end of this REMOVED notifier callback, or such
beautiful code wouldn't work :)

struct device *dev = device_rbtree_find(iommu, ite_sid);
if (!dev || !pci_device_is_present(to_pci_dev(dev)))
   return -ETIMEDOUT;

Thanks,
Ethan

  

>
> Best regards,
> baolu
>